tf. Implementation of linear regression by keras

One 、 Univariate linear regression

Univariate linear regression algorithm （ such as ,x Represents Education ,f（x） Representative income ）
Can be listed ：
f(x) = ax + b This formula reflects the relationship between education and income .
We use f(x) This function maps the input characteristics to the output values .

Two 、 Problem analysis

Predict the goal ： Make the prediction function f(x) The overall error between and the true value is the smallest .
How to define the minimum error ？
At this time, we need to use the loss function .
For example, use the mean square deviation as the loss function . That is, the square of the difference between the predicted value and the real value is taken as the mean .y Represents actual income , Find the right a and b, bring (f(x)-y)^2 Value , The smaller the better. , The smaller it is, the closer the predicted income is to the actual income .

Be careful Now the solution is the parameter a and b, The problem turns into How to solve the appropriate parameters a and b？
At this time, the optimization algorithm of model training will be involved .
For example, an optimization algorithm called gradient descent method is often used in linear regression .

3、 ... and 、 Gradient descent method

Gradient descent method is an algorithm dedicated to finding the extreme point of a function .
So-called “ Study ” That is to improve the model parameters , In order to minimize the loss through a large number of training steps . With this concept , Applying the gradient descent method to find the extreme point of the loss function constitutes the model learning based on the input data .
The output of the gradient is a vector composed of several partial derivatives , Each component of it corresponds to the partial derivative of the function to the corresponding component of the input vector ：

The output vector of the gradient indicates the direction in which the loss function grows fastest at each position , It can be regarded as indicating which direction to move in each position of the function , Function value can be increased .

Four 、 Learning rate

Learning rate It is a kind of super parameter or a kind of model Manually configurable The setting of needs to specify the correct value for it . If The learning rate is too small , It may take many rounds to find the minimum point of the loss function ; If it is too big , Then the algorithm may “ skip ” Minimum point and due to periodic “ jumping ” And the minimum point can never be found .

5、 ... and 、 Linear regression example code

The code running platform is jupyter-notebook, Code blocks in the article , According to jupyter-notebook Written in the order of division in , Run article code , Glue directly into jupyter-notebook that will do .

1. Import third-party library

import matplotlib.pyplot as plt
import tensorflow as tf
import pandas as pd

2. Reading data

data = pd.read_csv("/content/drive/MyDrive/learing_data/Income.csv")
data

3. Draw a scatter chart of education and income

%matplotlib inline
plt.scatter(data.Education,data.Income)

You can see that education and income seem to be in a linear relationship , A straight line can be fitted .

4. Build a linear network model

model = tf.keras.models.Sequential()
#  Build a full connection layer , Input is 1 Dimensional （ Because only 1 Column input data ）, The output is also 1 dimension .
model.add(tf.keras.layers.Dense(1,input_shape=(1,)))  #  Equivalent to building equations  f(x) = ax + b
#  View the network model structure 
model.summary()

You can see from above Param yes 2, in other words Fully connected layer (Dense layer ) There are two parameters , One parameter is The weight (a, Equivalent to f(x)=ax+b Medium a), Another parameter is bias (b, Equivalent to f(x)=ax+b Medium b).

5. model training

x = data.Education
y = data.Income
#  Use adam Gradient descent method as optimizer , Use mean square error mse As a loss function 
model.compile(optimizer="adam",loss="mse")
model.fit(x,y,epochs=50)

6. Model to predict

model.predict(x)# Forecast existing value .
model.predict(pd.Series([20]))