Hello everyone, I am Xiaobai. This is a machine learning course that combines theory with practice . We will use Python to complete the practical part. So if you haven’t learned Python , you can go to my Python course first. The theoretical part is because of this It’s not a mathematics class , so there won’t be too difficult mathematics in it, so don’t worry . Well, let’s start without talking nonsense. Hello everyone, I’m Xiaobai. Welcome to this course. The first one is artificial intelligence. AI is full of artificial intelligence. You should often hear artificial intelligence. So what is artificial intelligence? Artificial intelligence or you can call it artificial intelligence. In a simple sentence , we want machines Can have the same intelligence as humans. Once the machine has intelligence , it can help us deal with many complicated things. Then what is machine learning ML ? How to let the machine have intelligence? One of the ways is to let the machine learn . Now the question is how to let the machine learn. We can first think about how humans learn. Can humans learn from the past history and the past ? The same is true for machines . The past history and past experience of machines are actually the data stored in the past. So machine learning , in a simple sentence , is to find out from the data stored in the past. Out of the rules OK Machine learning is to find out the rules from the data in a simple sentence. Then what is deep learning DL ? Learning is to find the rules from the data . There are many ways to find the rules from the data. One of the methods is deep learning . The method of deep learning is to learn from the data by imitating a neural network of the human brain. Let’s find out the rules OK , so what is a neural network? I will introduce it to you later in the course . Okay, so let’s summarize the artificial intelligence, machine learning and deep learning we just mentioned. We can use this picture to do it. One said that artificial intelligence is the ultimate goal we want to achieve . We want machines to be as intelligent as humans , so that they can help us deal with many complicated things . How to achieve artificial intelligence? One of the ways is to let machines learn . In a simple sentence, machine learning is to find rules from data. There are many ways to find rules from data . One of the most powerful methods is through deep learning . The method of deep learning is to imitate the brain. A similar neural network to find out the rules from the data OK, so this is the relationship between artificial intelligence, machine learning and deep learning . Next, let’s understand how the machine learns . The machine has been mentioned before. Learning is equivalent to finding out the rules from the data . Then we will further discuss how the machine or computer finds out the rules from the data. In fact, it can be achieved by using some mathematical skills and programs to make the machine find out the rules from the data. Out of the rules, let’s take a look at what the process of machine learning looks like . Suppose we have some picture data in our hands . These pictures are pictures of dogs and cats. We want the machine to learn from the data of these pictures. Learn how to distinguish between dogs and cats OK , that is to say, we want the machine to find out the rules for distinguishing dogs and cats from these materials, so what can we do? You can first think about how humans learn to distinguish between dogs and cats Suppose you want to teach a child today. This child does not know what is a cat and what is a dog . You have to teach him how to tell the difference. At the beginning, you may take a picture and ask the child whether it is just a cat or a dog. You can see that the eyes of the child are quite different. Because he doesn’t know what you’re talking about, he just wants to go to bed quickly , so he casually answered him, saying it’s just a dog, obviously he got the wrong answer , then we, as teachers, give him a cross and tell him The animal that this child looks like is a cat. Then we took another picture and asked the child if it was just a cat or a dog. You can see that the child’s eyes revealed a little confidence compared to the last time . He replied It’s a pity that he said it’s just a cat. It’s a pity that he got the wrong answer again . As teachers, we gave him a cross again and told the children that the animal that looks like this is a dog . Just like this, after constantly looking at the photos, let the children answer . As long as the child answers the wrong answer, we will ask him to correct it . Over time, the child has seen too many pictures of cats and dogs, and he will probably know what a cat looks like and what a dog looks like . So you can ask him if it is just a cat or a dog. He will tell you very firmly and confidently that this is just a dog. The same learning method can be applied to the machine. We can input a picture into the machine or input a picture into the computer and ask him if it is a cat or not. What about dogs? After entering into the computer, some programs will be triggered . Behind these programs, some mathematical skills are actually used. Here , mathematical skills and programs can be combined . We can also call it a model, so here we can say that we You can input a picture into the model and ask him whether it is a cat or a dog . At the beginning, the model is the same as an ignorant child. He does not know what is a cat and what is a dog , so he guesses randomly. It’s just a dog. Obviously he got the wrong answer . We have to give him a cross at this time and tell him that the one that looks like this is a cat. Please correct the model and that the one that looks like this is a cat. Then we took another picture and entered it . Input it into the model in the machine and ask him if he is a cat or a dog . This time he answered that it was a cat. Obviously he answered wrong again. Give him a cross and tell him that the one that looks like this is a dog. Please correct the model and it will pass. Continuous training like this, constantly show the machine a lot of pictures of cats and dogs . If he gets the wrong answer, ask him to correct it until the model has a certain accuracy rate. Input a picture of a dog and this model tells you that this is a dog . After training the machine or training the model, suppose you have a new picture in the future and you want to ask whether it is a cat or a dog , you can input it into In the model, he will tell you that this is just a dog . Next, let’s look at an example. Suppose we have some data on house sales . These data are the same as the square meter of the house and its corresponding sale price. We want the machine to start from Find out the rules from these materials, find out the rule between the number of square meters and its corresponding selling price , that is to say, we want to guess what the selling price should be based on the number of square meters . In this way, we can say that the number of square meters is called The characteristic feature price is called the label label. We guess the price based on the square number, so the square number is called the feature price is called the label . Then the training process is similar to the previous one. We can input the square number, which is the feature, into the model . The model is ignorant at the beginning. So he will make random guesses. Suppose he guesses 4 million , let’s take a look at the information we have. According to the information we have in hand, the price of this 50- ping house is 5 million , which means our label is 5 million , which is obviously far from his guess . So at this time, we tell the model that the house is The selling price is 5 million. Please correct it. Next , enter the next data of 66 pings . Then enter the model and he guesses 9 million. Let’s look at the data in hand . It is 6.5 million and the label is 6.5 million. Constantly revising, constantly looking at the data , until the final prediction of the model is consistent with our real data, that is, when the error value of the label is not large , we say that the model training is completed. After the model training is completed , suppose you have a house and want to sell it It’s 72 pings. You don’t know how much he should sell . You can input the feature of 72 pings into the model and ask him to predict how much he should be able to sell . You may have doubts at this point. The price of a house should not just be It can be determined by how many pings it has. We may also need to look at its location, its layout and other comprehensive factors to evaluate how much a house is worth. There is nothing wrong with it , so you can collect more complete information. In addition to the number of pings, we have also collected his location and the number of rooms , so now the machine needs to estimate the selling price based on the number of pings, location, and number of rooms . Now we will talk about the number of pings, location and number of rooms The room is called a feature, and the price is also a label . Because we estimate the price by the number of square meters, the location and the room, these three are the characteristic price and the label . The learning process is the same as before. We input the characteristic square meter, location and the number of rooms. Into the model, the model is ignorant at the beginning, so he will guess randomly . Suppose we guess 4 million at the beginning. Let’s take a look at the data in hand, which is the label. The label is 5 million, so please correct the model and then enter the second data, the same. Ask the model to speculate that he speculates 9 million. Look at the label, it is 6.5 million, and then ask the model to correct it . Just keep training until the error value of the model reaches a certain range . We say that the training of this model is completed. In the future, if you have a house It is 72 pings, there are 6 rooms in Washington that you want to sell. If you want to guess how much he should be able to sell , you can input it into this model. He estimates that it can sell for 7.3 million . This is probably the process of machine learning. Next , we will go directly Implementation In this class, we are going to use colab . Colab is a python writing environment provided by Google for free. It allows us to write python programs very quickly and easily on the browser , but because it is provided by Google , you need to create a Google first. Account and log in . After logging in, you can see 9 dots here . Click to find cloud hard drive. We can see a new addition in the upper left corner , and then click here. If you have found Google colab here, you can Click it. If you can’t find us, click to connect to more applications . Well, we can find colab here. Click to enter and click to install . Click to install. He may ask you to log in. Then you log in. After the installation is complete, we will Just click OK to complete OK, then we will add more here, you can see that there is an additional Google colab, click in, after clicking in, you can see that it has helped us create an environment for writing Python . First, let’s see The upper left corner, the upper left corner is the name of the file , you can modify it , assuming I change it to the environment to build OK, then the modification is over, and because I am more used to the background being black , so I go to the tool to find the settings Then here is a theme in the website . We select Dark and click to save the background and it will become black. Of course , you can also set your favorite color . On the editor side, you can also set the size and spacing of the text, etc. Waiting, I leave it to everyone to play by yourself . Before we write the program, we need to do the connection action. We can go to the upper right corner to find a connection point , and he will allocate some resources for us to use . Wait for him After the connection is completed, we can see the block in the middle . The block in the middle is where we can write programs. First of all , we can enter the program code directly in this grid. Suppose I enter print 87 , and I will enlarge it a bit for you to see. It’s clearer. Well, after printing 87 , there’s an execution button here, click it , and it will start to execute , and the execution result will be displayed below. In colab, the code can be divided into pieces. We can put the mouse Put it a little bit above this grid, and you can see that it jumps out a code or text. If you put it a little bit below, it will jump out. Suppose I click to add a code , and it will have an extra grid . That 's what we You can write python programs in this grid . Suppose I write print 88 and press execute . You can see the execution result will be displayed below . Let’s take a look. If I click to add text , it will jump out of a grid where I can write text . Suppose I Write a hello, me, hello, everyone . In addition to writing text , you can also choose some fonts, fonts, etc. You can see that there is an additional text grid here . OK, we can add a lot of code grids, and we can also add a lot. If you don’t want the grid of the text , you can delete it here . There is a symbol of a trash can, click on it and delete it. Delete Delete Delete OK . Colab has another important point , which is that it can provide free GPU. Let us use it with TPU. GPU and TPU can help us speed up a lot in computing . We can find the editor on the top and then there is a notebook setting . You can see that there is a hardware acceleration here. You can choose GPU or TPU Then in the following classes, we will also use GPU for acceleration , so suppose I choose GPU here and press save , you can see that he will help us re-allocate the connection, and we have to wait for him . After he is connected, he will Ask us if we want to delete the previous execution stage. Suppose I press cancel first . After connecting the GPU, we want to see which GPU he uses for us . We can type it in the grid of the code ! Press nvidia-smi to execute, and you can see that the GPU he is using for us now is this Tesla T4 OK . Then, our environment construction is complete. The first mathematical technique I want to introduce to you is called simplicity. Linear regression is simple linear regression in English . You can see that there are two words in front of its name, so you can imagine that it is very simple . Don’t worry too much. Let’s first describe the situation we encountered today . Suppose today you are a new The boss of a start-up company , you want to hire your first employee , but you are not sure how much salary you should pay him , so you go to the market and collect some information about people in the same position . These data are their years of work. With the corresponding salary , let’s take the data here and draw it in a graph, so that the x-axis is the seniority and the y-axis is the monthly salary. Each cross here It means every piece of information you have collected . From this picture, we can easily see that the seniority is directly proportional to the monthly salary, that is, the higher the seniority, the higher the monthly salary . Well, now the problem comes. You are a new company. The boss of a start-up company , you want to hire your first employee . Your first employee comes and he tells you that his seniority is two and a half years, which is 2.5 . So how much salary should you offer him? You can do it at this time. Simple linear regression is used. Since seniority is directly proportional to monthly salary , we can probably use a straight line to represent these data. Simple linear regression is to represent the data with a most suitable straight line . Suppose now that we have found this The most suitable straight line , then we can bring the 2.5 seniority into this straight line to see how much salary we can give this employee . After bringing it in, we can see that we can probably give this employee a salary of 51K . Now The problem becomes how to find the most suitable straight line? Let’s first look at how a straight line can be represented mathematically . We can write y=w*x+b to represent a straight line . Then , this formula Applying it to the present example, y will be equal to the monthly salary and x will be equal to the seniority. So the problem now becomes that we have to find the most suitable w and the most suitable b to represent these data. We have to find the most suitable w and the most suitable b to represent this straight line . Well, let’s implement it directly to see what kind of straight line will be produced by different w and different b. First, let’s read in the data first . We can use the Pandas module To do the reading action. Here I call it pd Pandas. This module is a very useful data processing tool . We don’t need to install it in colab because the default is already installed , so we can directly import it for use . Next The URL of the data we want to use can be found in the course file . I use a variable to represent it. Here, pay special attention to the file we want to read . It is a CSV file , so we can use Pandas under it . read CSV to do the reading action. We only need to write the URL into the parameter and it can be read . Here, I also use a variable to represent it. We can directly display the read result and execute it . This is what we do The data to be used, seniority and his corresponding salary , there are a total of 0-32, that is, 33 data. We want to use a straight line to represent these data . We have said that a straight line can be written in mathematics as y equals w times x plus b . In our example, we want to use seniority to predict salary , so seniority will be x and salary will be y. We first separate x from y, and we want to get the column of years experience. If you want to get this column, you can put brackets years experience behind it so that it will separate out that column. The same is true for x and y. I want to get the column of salary , so I will display it to see if there is no problem with x and y. Then Well, let’s first draw the pictures corresponding to these materials to see. I ’m opening another grid. If you want to draw pictures, you can use a very useful suite called matplotlib. We use the pyplot under it. I call it plt, which is the same as matplotlib. The presets in colab are also installed , so we can directly import and use them. I can use the scatter under it to draw the picture. We only need to pass the x and y that we just separated into it, and it will help us draw the picture. Then Finally, I want to display it and write a show to execute it. You can see that it will display our data bit by bit in the form of a point. If you want to change the style of this point , suppose I want to put I want to change its color into a cross . Then I can add some parameters . I can write another marker equal to the original mark . I want to change it into a cross, so I can write it. x Then I also want to change the color. I can set the color to turn it into red. I write Red so that it can be executed. You can see the mark and it will turn into a red cross. As for what other marks and colors can be used here , if you are interested I leave it to you to research by yourself. Here I can add a title to the picture. I can set its title assuming that my title is seniority corresponding to his salary . Let’s execute it. You can see that there seems to be a little mistake in this part of the title. It has 4 frames. The reason for this error is that matplotlib does not support Chinese by default. If we want to display Chinese , we can add Chinese fonts by ourselves . Well , let’s add it . First, we need to download one first. Chinese font Here I create a new grid to do the downloading action . We can use the wget tool to do the downloading , but it is not installed in colab by default , so we need to install it first. We want to install the package in colab Module, you can enter pip install what you want to install after the exclamation mark. After installation, import it. Then we can use the download below it to do the download action . Just write the URL of the thing you want to download in the parameters. That’s ok. Then this URL can be found in the course files . This is the font we want to download , so we can execute it directly . It will be installed first and then imported. After the download is complete, let’s take a look. You can find a file on the left and open it . This is what we just did. The downloaded font is called Chinesefont.ttf . Let me close it first. After downloading the font , we can add it to matplotlib. If we want to add fonts, we will import matplotlib. I call it mpl and then import it from matplotlib. The font_manager under it is imported into the fontManager. After importing everything here, you can use the addfont under the fontManager to add fonts . The font we just downloaded is called Chinesefont.ttf, so we write Chinesefont.ttf here. After adding the font, we It is also necessary to set this font to be used now . To set this font to be used , we can use the rc under mpl to set something. What we want to set is the font, so the first parameter writes font and the second parameter specifies to use The font is Chinesefont , so it should be fine. Let’s add the font first , then set the font to be used and execute it again . You can see that the Chinese can be displayed normally . In addition to setting the title of the chart , we can also set it. The x-axis and y-axis labels of the chart To set the x-axis label, we can use the x label to do the setting , then our x-axis is the seniority, so I write the seniority, and then the y-axis , the y-axis is the y label, and the y-axis is the monthly salary Then its unit is thousand, so we write the monthly salary Execute the brackets again, and you can see that the labels of the x-axis and y-axis are now up with seniority and monthly salary . Next, let’s draw the straight line. Here I’ll add another grid. As I mentioned just now, a straight line in mathematics can be Written as y=w*x+b , pay special attention to the fact that we want to use the value of x to predict y, so this y is different from our y, this y is the predicted value , and this y is the real data, so Let's go back to the bottom and we can use x to multiply a w plus b to represent our predicted y , so I call it y_pred to represent the predicted y. Now, suppose I first set w to 0 and b also set Let’s take a look at what the straight line drawn by w=0 b=0 will look like. To draw a straight line, we can use the plot under plt to draw it . The first parameter is to write the value of x. The second parameter is to write Let’s see the value of y. This time, our y value is the predicted result of y_pred. We also need to display it and write show OK. Let’s execute it and see. You can see that this is the straight line drawn when w is equal to 0 and b is equal to 0. Then I You can also change its color. Suppose I set its color to blue , which I think is good. Then I will draw the above information and these crosses into it, and I will put all these here. Copy it, copy it here to execute it again , this is our prediction line , these are real data , our goal is to find a straight line that can best represent these data , now this straight line is w=0 b=0 So we need to find the most suitable w and b to represent these data . Before that, we can add some legends to the chart. Our plot is to draw a straight line, and the scatter is to draw these crosses. We can Add a label behind it , the legend is the prediction line , and then we can also add a label on the crossed side , which is the real data. After adding this label , we can write it here plot.legend will display the legend, let’s execute it, and you can see that the graph here is displayed. This straight line is the prediction line , and the crosses are the real data. Let’s first write these here as a Function This way, let us bring in different w and b . I call it plot_pred , and then it can pass in w and b to indent it. Let’s try to bring in different w and b to draw it. What will the result look like? Here we can call plot_pred. Assume that I put in 0 and 0 first and execute it once. You can see that the result is the same as just now . Suppose I modify the value of b and I change it to 10 and execute it again . Seeing this line, it seems that it has not changed , but in fact it has changed . It is because the values of the x-axis and y-axis are not fixed , so the graph looks like it has not changed . So this Let’s fix the values of the x-axis and y-axis first. If we want to fix it, we can write plt.xlim like this. We set its maximum and minimum ranges. If the x-axis is about 0-10, here it is There are a little more , so I set it to 0-12 , here we can write it like this, write a list in it , and then set its minimum and maximum values , and the value of y is the same , we set its maximum and minimum. On the other side, it is possible to get negative numbers , so I let it be -60 at the minimum, and 140 at the maximum. Then we execute it again, and you can see that the values of the x-axis and y-axis will be fixed now. Execute once and the values of the x-axis and y-axis will be fixed . The y-axis is -60 to 140 and the x-axis is 0 to 12. Let’s try again. The result of 0 and 0 is the result of 0 and 0. This line is when y is equal to 0 In this place , if I bring the value of b into it and change it to 10 and execute it again , you can see that the straight line is going up , so we can know that the value of b is to control the up or down of the straight line. If it is changed to 40, it will go up . If it is -30, execute it. You can see it and go down . Now let’s try to change the value of w . It was originally 0. Let ’s change it to 10 and see. This line has become slanted and slanted up . If it is changed to 20, it will be more slanted . If I change it to a negative number -10, you can see that it will fall down and slope downward. If it is positive, it will slope upward. Negative number If it’s a downward slope, I’ll try -5 on this side , and I can see that it’s a line like this. We can’t see this side because it’s beyond the border . Then let’s make this picture dynamic , so that we don’t have to. I have been manually adjusting the values of w and b. If I want to add some interactive components in colab , we can use the ipywidgets tool. Here I introduce the interact in it . Then I can do this and use it. Write a function in the first parameter Name Then what we want to dynamically adjust are the values of w and b , which are the two parameters to be passed in by this function. Here I can set the value of w . I hope its range is between -100 and 100 and then Its spacing is 1, and then set the value of b . Suppose I want it to be -100, and the spacing between 100 and 1 is the same . Let’s execute it directly. You can see that there will be more interactive components here . The default value is w and b is all 0 and I can make adjustments. When I add w to 8, the line will look like this. Then add b to the top and it will slowly move up and then add more. Everyone can play around and adjust different ones. Let’s see what the straight line will look like with different values of w and b. Well, I’ll leave it to you to play with. After seeing what kind of straight line will be produced by different w and b , then our question will become what is the best way? What about the straight line suitable for these data? Let ’s define it and give each line a score. Let’s first look at a set of relatively simple data . If I also want to use a straight line to represent these data , suppose today I am Using this straight line , this straight line is the result of w equal to 0 and b equal to 0 , then we should give this straight line a score for how well it fits these data , or you can say, how well this straight line fits these data Let’s give a score. Then how do we give this line a score? It ’s very simple. We just need to calculate the distance between these real data and this line . Because if the data is closer to this line If they match, the distance between these data and this line will be smaller , so we only need to calculate the distance between each line and these real data , and then find the smallest one among them, and that line will be the most suitable for these. The straight line of the data Let’s do an actual operation directly . In this example, there are a total of three data points , which are respectively at the positions of 1, 1, 2, 2, and 3, 3. We want to use this straight line to represent these data . This straight line is w Equal to 0 b is equal to 0 and then we want to give a score for the suitability of this straight line with these data. How to score ? We can use the sum of the distances between these data and this straight line Use it as the basis for scoring. If this point is 1-0, the distance between it and the line is 1. If this point is 2-0, the distance is 2. Here, 3-0 is 3. So our formula will be written like this, you may see Here, besides 1-0, I also squared it . The reason for doing squares here is because it is convenient for us to calculate , because you may have negative numbers in the future , so you can solve the problem of negative numbers by directly square. Solved So in this example, the square of 1 minus 0, the square of 2 minus 0, the square of 3 minus 0, the sum of the final distance , it should be said that the sum of the square of the distance will be 14. Let's look at another line . Suppose we want Use this line to represent these data . This line is the result of w equal to 2 and b equal to 0. We also calculate their distance . The point here is also 1, so here is 1 minus 2, 1 minus 2 Then here is the square of 2 minus 4, 2 minus 4 , 3 minus the square of 6. You can see that there will be a negative number here . If we don’t have a square, there will be a negative number here , so after the same calculation here Do the sum , the same is 14. Then look at the next line. You can see that this line is quite consistent . It is the result of w equal to 1 and b equal to 0. Then calculate the distance , and you can find that each of them has the same value , so the calculated distance will be If the three lines are 0 , we will say that the line where w is equal to 1 and b is equal to 0 is the most suitable straight line for these data , and then the scores or distances generated according to different w or b The sum of squares , we can write it as a function . This function is called cost function in Chinese, it is a cost function. In our example , the cost function can be written like this. It takes the real data to subtract the predicted value and then squares it , that is, we To calculate the real data, the square of the distance from that straight line will be the cost , which is what we just said about the score . Let’s go back to the original example . We want to find a straight line that best fits the data , so we need to help the straight line evaluate A score , that is, to calculate the cost here , let’s assume that all the straight lines now have their b equal to 0 , let’s see how much the cost will be corresponding to different w, we will bring in one by one , first bring in one w is equal to -10 or so, and its cost will be here , then bring in the second and third , bring in all the way, bring in and bring in, finally we bring in many, many points , we will find that the cost function is actually the cost function , it will It is a parabola like this. This is the case where b is equal to 0. If b is not equal to 0, what will the cost corresponding to w and b look like? The value of w here is the value of b , and the z-axis is the value of cost . You can see that its graph will look like this. The red point is the point with the lowest cost . Let’s look at two other angles. From these two angles, we can You can see that the place with the red dot, as I just said, is the place with the lowest cost . We can see that the place where w is equal to 10 or so , and where b is equal to 20 or so, will produce a point with the lowest cost . This is our goal. Well, our goal is to find the most suitable straight line . To find a most suitable straight line is to find the distance between the straight line generated by the most suitable w and b, the most suitable w and b, and all the real data. The sum of the squares will be the smallest , that is , the place where the cost is the smallest here. After we understand it, then we will directly implement the cost function . Here, I opened a new colab file to implement the cost function. At the beginning, we need to read the data first. The method of reading the data is the same as before , so I directly copy and paste it for execution. After the data is read, then we will implement the cost function and open a grid. Our cost function is Take the real data and subtract the square of the predicted value, so here we can write it like this , first calculate the predicted value, I call it y pred , it will be equal to w, multiply x and add b, then w here I don’t know anything about b and b yet , so let’s assume that w is equal to 10 and then b is equal to 0. After calculating the predicted value, we can take the real data. The real data is y minus the predicted value y Pred , and then we need to square it . So I put it in brackets and square it. This side is called cost . Let’s display the cost directly. You can see that there are a total of 33 values displayed . These 33 values are the real data minus the prediction. Value and then do the squared result , which is our squared distance. If we want to sum up the squared values of these distances , just write .sum() after it to display it . You can see that this is the summed up The result is the sum of squared distances . If the value here is too large, we usually average it . We have a total of 33 records, so we will divide by 33 here , but it is better to write it like this . To calculate the length of x, the length of x is 33, so dividing this by 33 is the average of the squared distance. Next, let’s write the cost calculation here as a function , so that we can bring in different w and The value of b is good to open another grid . Here I call it compute cost , and then it needs to pass in our data x and y and the predicted line, its w and b values. The calculation method is the same as here . Paste it directly. We need to do the predicted value first. The predicted value is w multiplied by x plus b , then calculate the cost and then add it up and take the average. I will make the final cost equal to the sum and then take the average result. Well, let’s pass this cost back, let’s try to see what kind of results will be produced by adding different w and b, let’s call here to see that xy is the real information here, and w we just brought 10 and b 0, w is the same as b, let me try to execute 10 and 0, he said that the compute cost is not define, oh, it has not been executed yet, so it needs to be executed first and then executed here, and the value returned is 602 points , which is the same as before. The input is 10 and execute it again. You can see that the cost this time is 227 points . Then you can import it yourself to see how much the cost will be generated by different w and b . Then let’s try it when the fixed b is equal to 0. In this case , let w be between -100 and 100 , and then let's see what its cost will be. Here , I will first define a cost, let it be a list , and then store the cost value of w between -100 and 100. OK What about here, we can use a for loop to do calculations . I let w make it range from -100 to 100. Here I want to write 101 , and it will generate a similar one from -100 to 100. the list Then we have to use the function on the cost calculation side to do it directly . Use xy and fix b equal to 0. Let it always bring in the value of w. This side is called cost and then append it to our costs. In the list, I finally displayed it to see how to execute it . Here, the total number of w equals.. Wow, there are a total of 201 values from w equal to -100 to 100. I will turn off its cost first and then Let's draw the cost corresponding to different w into a picture to see. Here we introduce our drawing tool matplotlib and the pyplot under it. I call it PLT. Then we can use scatter to put each data point into it. Draw it. We draw each w and the corresponding cost . The w on our side is between -100 and 100. The cost corresponding to it is displayed and executed. You can see that it is between -100 and 100. There will be a total of 201 points. The 201 points will be so densely packed when drawn. Besides drawing like this, we can also directly connect them into a line or a parabola. Here we use plot to pass in w and Corresponding to the cost and then execute this parabola to make a connection. Here I will help him add a title and a label . This one is the cost function when b is equal to 0 and then w is between -100~100 . Then set its xlabel xlabel is The value of w and then execute the ylabel cost to see that the title and the labels of the x-axis and y-axis are all up . Then we also take the value of b into account and let the value of b also range from -100 to 100. Let’s see what the cost will be How much? Here I introduce a very useful tool for matrix operations, numpy . I call it NP. Now both w and b are between -100 and 100. We can use arange under NP . The usage of this arange is basically the same as above. The range is almost the same. I want to create a matrix between -100 and 100, and the interval is 1. We can write it like this. Here, I call it ws because there are many w and b are the same. I call it bs Then I create a two-dimensional matrix. Here I can use NP point zeros to create a matrix with all 0s in it. The first dimension is 201 and the second is also 201 because the value of w here is total. There are 201 b values and 201. This matrix is to store the costs corresponding to different w and different b , so I call it costs and then you can do calculations. I first use a for loop to run through all The value in ws and then use a for loop to run through all the values in bs , and then we can calculate their cost. If we want to calculate the cost, we have already written the function above, just take it down and use it directly copy here Come down , well , xy is the real data , and then the cost will be calculated by passing in wb here, and we can store it in the costs matrix. Here , we define another i, which is equal to 0 at first , and then define j here. If it is equal to 0 , then the values of i and j here will be equal to this cost , then let j add 1 here, and finally let i add 1 here, after the calculation, all w and all b will be added The cost corresponding to the combination is stored, stored in this cost , and displayed to see the execution. He said that numpy has no attribute and wrote an extra r here to execute it again. He needs a little time to wait for him . It looks like this A two-dimensional matrix . Each value in it represents a corresponding cost calculated by w and b. Next, let’s draw the cost calculated by considering w and b at the same time. Take a look Because we need to consider the values of w and b at the same time , we will draw a 3D graph. To create a 3D graph, we can write plt.axes and set its projection=3D. I call it AX and we will display it first. Come out and execute it . You can see that a 3D image is created , but there is nothing in it now . We can see that the edges of this image are a bit gray. I don’t want it to be gray . I can also make a setting . I can Use ax.xaxis.set_pane_color to set the color . I want it to be white . I can write an rgb value here. For white , let it be 0 0 0 for easy execution. He said that there is no attribute and it is reversed here , so you can see it When it comes to the x-axis side, it will become white. Then I will use it on the y-axis and z-axis side to copy it directly . Just change it to y and z to execute it again , and it will be white. It feels much more comfortable. Next , let’s draw the cost corresponding to w and b as a surface graph . Here we can use the plot_surface under it to compare the w value we just created with the value of b , that is, ws and bs These two matrices are passed in , and the last is the cost we stored. Pay special attention here . In fact , we don’t just pass in the two one-dimensional matrices of WS and BS . What we want is the two one-dimensional matrices. A two-dimensional grid is generated . If we want to generate this two-dimensional grid , we can use the mesh grid under it in numpy, and here we pay special attention to the first one. We first pass it into BS and then pass it into WS. Then it will be It will be made into a two-dimensional grid and sent back to us . I will call it b grid w grid. If you want to know more about what this two-dimensional grid is and how it works , you can take a look at this URL for his explanation. It is very detailed and I will not go into details here . If you are interested, I will put the URL below and you can study it yourself . Here we will change it to pass in w grid and b grid , so it will be no problem. We can execute it and you can see this surface The picture is displayed. Let’s add some titles and labels to this picture . Here, we can write ax.set_title. Here, pay special attention to the fact that we need to add an additional set in front of it, which is different from the previous one. In the front , just write title directly . If we want to add a set title, we will let it be the cost generated by wb, that is, the corresponding cost will be used in Chinese. If it is used in Chinese, we need to add Chinese fonts . So I will go back to the previous file. Inside, the Chinese language is added in this place . I still need to download it first, so I will paste it here . This is to download it, execute it first, and then add it . This is to add it , so I will paste it here. First download and then add the font. After the title is set here , then we set the labels of the xy and z axes . Here , we also need to add set xlabel . The label of the x axis is w , and then set it.. here I directly Use copy to set its y-axis label, the y-axis label is b , and then the z-axis, and the z-axis is cost Ok, let’s display it again, you can see that this is w and then b, and their corresponding cost is the z axis . If we want to rotate this picture, it is also possible. If we want to rotate, we can do it here Set a view init, which can pass in two parameters. The first parameter is the rotation angle of up and down , and the second parameter is the rotation angle of left and right. Suppose I write 45 for the first one and -120 for the second one. Execute it again , and you can see now As shown in the picture, it turns like this: this side is w, this side is b, and this is the cost . Well, you can also play with the angle of rotation. Then I think the color of this curved surface is not very good-looking, and I want to make it One more modification, you can set another parameter here called cmap , I set it to spectral r, let’s execute it and see, you can see that the color is much better now , as for what other colors can be set here, I will leave it to you Go research, I think this color is good , and then we can set another opacity value , the opacity is called Alpha Alpha , let it be equal to 0.7 and execute it, you can see that now it has a more transparent feeling, it looks like It’s more comfortable. Then, if we want to make this picture look better , I can add a border to it. I can write a plot wireframe and pass in the first three parameters . Then I set the color of the border to black , so let’s execute it. You can see that this border is added now , but it is a bit too dark . We can also set its transparency Alpha assuming that I make it 0.1. Then execute it. You can see that it is very comfortable and beautiful. Next, let’s put Find out the point with the lowest cost. To find out the point with the lowest cost , we can use the min under it in numpy to find our entire cost. I will print it out to see what the lowest cost is . The lowest cost It is more than 32.69 . If we want to know the w and b corresponding to the cost of more than 32.69, we can do it like this. We can use np.where to find its location and find the lowest cost among all the costs . Where is the index of the position , because this cost is a two-dimensional matrix , so it will return two values, that is, two indexes. I call it w index and b index . Then we can put these two values Print it out and execute it. You can see that the index it finds is at 109 and 129. We need to find out the w value corresponding to this index from all w matrices . The part of b is the same . OK , let’s execute it again. Once, you can see the corresponding value. The corresponding w value is 9 and b is 29. That is to say, when w is equal to 9 and b is equal to 29, there will be the smallest cost. Here we can write like this. When w is equal to this value and When b is equal to this value, there will be the minimum cost . Then the minimum cost can be obtained from the two-dimensional matrix of costs. The value to be obtained is the value corresponding to the two indexes . Let’s execute it again and it will be Speaking of which, when w is equal to 9 and b is equal to 29, there will be the minimum cost , and the minimum cost is more than 36.69. Finally , we can also draw the point of the minimum cost. If we want to draw it, we can Use the first value of scatter to pass in the value of w, then the value of b, and then the z-axis, which is the value of our cost. It is easy to execute. The point where you can see the smallest cost is here, so I can put it Change the color , set its color equal to red , and then I want to make it bigger , set its SS to be the size , and let it be 40, so I can execute it again. You can see that it is much more comfortable . Finally, I want to make this To make the picture bigger , I can go to the top to set it . I can write plt point figure and then set its figsize. He can set two values. These two values represent the width and height of the figure respectively . Suppose I start with 5 and 5 are fine. Let’s see what it looks like. It’s still a little small. I’ll make it a little bigger. 7 and 7 will make it look more comfortable . Well, the other parameters here are the rotation angle. The size of the picture Or color, etc. Everyone can make adjustments by themselves . This is the implementation of our cost function. After reading the cost function , our question will become how to find the best w and b efficiently. The best w and b correspond to the point with the lowest cost. It is not difficult to see from our last implementation that we use a brute force method. I exhaustively enumerate all the values of w from -100 to 100 and then go to Look at its cost and find the lowest point from it. The w and b considered here are the same. I exhaustively enumerate all the combinations of w and b from -100 to 100 , and then find out the corresponding cost. Its lowest point , but this is not a good way . We must efficiently find out the best w and b. To efficiently find out w and b, we can use a method called gradient descent, which is gradient descent in Chinese . Everyone Don’t think too hard about it . In fact , gradient descent is to change the parameters according to the slope . In our example, the parameters are w and b, so the values of w and b are changed according to the slope . Let’s directly look at the gradient descent. How does it work ? Let’s take this as an example . Let’s assume that b is equal to 0 and only consider w. How to find an optimal w that can minimize the cost? First, we need to set an initial value of w . Then The initial w value can be set randomly. Let’s say I set it here , which is roughly equal to -75. Then we can calculate the tangent slope at this point through differentiation. We can calculate the tangent slope at this point through differentiation . Everyone here Pay special attention when we use gradient descent . In fact, we don’t know the blue line. We don’t know the blue parabola. This parabola is obtained through our exhaustive violence . So you don’t I don’t know that the lowest point is here . Let’s re- describe the problem now. It can be like this. Today you are blindfolded and thrown into a place where you can only go forward or backward . What is your goal? You have to go to the lowest point , but fortunately, you can use some method to calculate the steepness of the front and rear of your current position, and then you can use this steepness to find the way to go down , back The same as the original example, at this point , you can calculate the slope of the tangent line through differentiation. The slope of the tangent line is equivalent to the degree of steepness . Then we can use this slope To find out the way to go down , let me briefly explain how the slope is calculated. First of all, we need to set what the cost function looks like . In our example, the cost function looks like this. We use real data. To subtract the predicted value and then square it . In mathematics , it is to subtract ypred from y and then square it. This ypred can be expressed as w times x plus b because we want to represent the data as a straight line. So here is w multiplied by x plus b and in this example b is equal to 0 , so we can omit it directly and then we just need to differentiate it with respect to w to get the slope of the tangent line . After differentiation President, I will not show you the detailed differentiation process. If you are interested, you can study it yourself . In fact, it doesn’t matter if you don’t know what differentiation is at all , because there are many tools that can help us do calculations automatically. After the differentiation, if we want to know what the tangent slope will be when w is equal to -75, then we will bring in -75 . The x and y here are our real data , and we can calculate the tangent slope by bringing in the same. How much is it ? After we know the slope, it is equivalent to knowing the steepness here. After knowing the steepness here , we can go down. How to go down? We can subtract the slope from w and multiply by A learning rate , what is this learning rate, I will explain to you later . Let’s look at the previous w to subtract the slope. In this example, now w is about -75 , and the tangent slope is obviously a negative number , so we let w to subtract a negative number , that is to say, it will add a value , so after adding a value, it will move forward , then we repeat this action all the time, and then calculate the tangent slope at this point , then we bring the number with You can find the slope when you come in. After you find the slope, bring it to the following formula . Now, where w is about -60 , you can obviously see that the slope is still negative. Let w subtract a negative number , that is, add a Value, so it will move forward . We keep repeating this action . Calculate the slope update w Calculate the slope update w Let’s take a look Calculate the slope update w Calculate the slope update And then keep repeating until we are almost close to the lowest point Or at the lowest point, the slope of the tangent line here will be quite close to 0. OK , the slope of the tangent line here will be very close to 0. After it is very close to 0, we let w subtract a value close to 0 , which is equivalent to w being If there is no update , then we will find the lowest point . This is probably the operation process of gradient descent . Next, let’s take a look at what the learning rate is. Let’s go back to the previous example . At the beginning, we first find an initial w, and then we can calculate its tangent slope . After calculating the tangent slope, if we want to go down , we can subtract the slope from the value of w and multiply it by a Learning rate The learning rate is that you need to set a value . You have to decide how much the learning rate is. Let’s take a look here, multiply the slope by the learning rate . If your learning rate is larger, the side will be larger. Well , it may be the larger the positive value or the larger the negative value. Then we subtract a larger value from w, and the change of w will be greater , that is to say, its pace will be larger . Conversely, if your learning rate is higher If it is small, it will be relatively small after multiplying here , and then we take w to subtract a relatively small value, and the change of w will be relatively small , that is, the pace will be relatively small . Well, let’s see what happens with different learning rates. The result of the appearance, let’s first look at the learning rate. If the learning rate is high , then his steps will be relatively large. OK, it’s relatively large. Well , you may have a question when you see this , is whether his steps are gradually getting smaller. That’s right , he is gradually increasing. Get smaller , even if our learning rate remains the same , his steps will gradually become smaller. Why ? Because the slope here will also change . The slope here is relatively large , and the closer to the lowest point The slope will be smaller , so his steps will also be smaller. Let’s see what it would look like if the learning rate is smaller . Then his steps will be smaller. Let ’s compare the two pictures together. Clearly, the one on the left has a high learning rate, and the one on the right has a small learning rate . You can see that the strides on the left are relatively large, while those on the right are relatively small . Speaking of this, you may say that we must have set the learning rate to a high value. Ah, because it is set to be too large, the speed of going down will be faster, so that we can reach the lowest point faster . This is a very good question, but our learning rate may also be too large. What will it look like if it is too large ? Take a look, you came here in the first step, and then in the second step, you didn’t go to the lowest point , you just stepped over to the opposite side , then let’s look at the third step. In the third step, he stepped back and you will In this way, you keep stepping over and over and over and over again, and you will never reach the bottom point because your steps are too big and there is no way to reach the bottom point at all. This is the problem of too large learning rate . Let’s see if What will happen if the learning rate is too small? Every step you take is very small, very small, so small that you can’t go to the lowest point even if you go forever. So you can’t make the learning rate too large or too small . To find A most moderate value , how do we find the most moderate value? We can find it through continuous experiments and tests. Well , this is the learning rate . Then let’s go back to the original gradient descent. The examples just now are only considering w In this case, if we have to consider even b now , how will the gradient descent work? It is basically the same . First of all, you need to set a random initial value of w and b . Here you see this The picture is also obtained after exhaustive enumeration , so you don’t know that the lowest point is here , so we directly apply it to the metaphor . Today, you were taken to a place inexplicably and then blindfolded . This place is a bit The terrain is similar to a canyon , so your goal is to find the lowest point of this canyon , but this time is different from last time, you can not only walk back and forth, you can also walk left and right , you can also know your front and back through some method The steepness and the steepness of the left and right , then you can find the way down through the steepness. The steepness on this side is the same as before, which means the slope . So here is to calculate the slope in the w direction , and here is to calculate the b The slope of the direction To calculate the slope , we can also differentiate the cost function. In this example, the cost function is long. It is to subtract the predicted value from the real data and then square it. That is, subtract y pred from y and then square it. What about y pred here? We can also decompose it into w multiplied by x plus b, and then differentiate w to get the slope in the direction of w . If we differentiate b, then You can get the slope in the b direction. After the differentiation, the result will be like this . The slope in the w direction is as long as this. The slope in the b direction is as long as this. If we want to know what the slope is at this point , then we can just bring the value into it. Now , bring in the w here, bring in the b here , and then x and y are also the data we have , then we can calculate the slope in the direction of w and the slope in the direction of b , and then we can update it The values of w and b means that we can go down . To update w, we subtract the slope in the direction of w from w and multiply it by the learning rate . To update b, we subtract the slope in the direction of b from b and multiply it by Learning rate In this way, we can go down . After reaching this point, we will recalculate the slope in the direction of w and the slope in the direction of b . After the calculation, we can update it, and then we can slowly move forward. Walking to the lowest point , we are walking blindfolded . How can we judge whether we are at the lowest point? Similarly, when you get closer to the lowest point , the slope of the w direction and the b direction will be smaller , and then you If w and b are subtracted by a small value, it means that there is no change . We can use this to judge where we are now. For the learning rate, you have to set a value yourself . This value cannot be changed. Too big and not too small. If your learning rate is set too high , his steps will be very large, so big that there may never be a way to reach the lowest point . On the contrary, if the learning rate is too small, your steps will be very small. It's so small that you may not reach the lowest point even if you go to the wildest place. This is our gradient descent. Its operation process . Then we will directly try it out . Okay , then we will implement the gradient descent. First of all, we must read it first. The action of getting data and reading data is the same as before , so I directly use the copied one. Then we just said that gradient descent is to calculate the slope and then update the parameters. To calculate the slope in the direction of w , we can differentiate the cost function with respect to w. To calculate the slope in the b direction , we can differentiate the cost function with respect to b . Let's do the calculation. Let 's differentiate the cost function with respect to w . The result is 2 times x and then multiplying w times x plus b to subtract y is the result of differentiating the cost function with respect to w . If we differentiate the cost function with respect to b , the difference is that one less x is multiplied here, so the slope in the direction of w is called w gradient, and the direction of b Well, I’ll call it b gradient Now if I want to know what is the slope of the w direction and the b direction when w is equal to 10 and b is also equal to 10, then I will first set w to 10 and b to 10 , let’s go first Looking at the good execution in the w direction, we can see that it has generated a total of 33 values because we have a total of 33 data. Every data you bring in will generate a slope . Here we will average it. If we want to average, we can First add it up , and then divide it to see how many records it has . Here, how many records are there? I will use n to represent it. Let ’s calculate the length of x , and then we can divide it by n. Let’s execute it again. This is the average result, which is -118 points. Then let’s look at the b direction again. We have 33 data, so it will calculate 33 values . Here we do the same , that is, add it up and divide it by n Execute again , and you can see that the average result is -27.46 . In fact, if we want to calculate the average here, we can also directly write it like this . We write some mean, which also calculates the average. There is no need to calculate the length of x separately. You can see that the result of the execution is the same. Change the direction of w here, so they are all the same. For convenience, I will calculate the slope or you can say the action of calculating the gradient and write it as a function. Here I will Call it a compute gradient , we need to pass in x and y, which are our data , and then we can do calculations with the value of wb , and we will return the calculated results to the w direction and the b direction , then I will open another grid to try it out Suppose what I want to know now is when w is equal to 20 and then b is equal to 10 , what is the calculated slope ? This side needs to be executed first and then execute it. The w direction is 537 points , and the b direction is more than 70 points . Calculate w and After the slope in the b direction, then we can update w and b , because our update method is to take w and b to subtract its slope and multiply it by a learning rate . We first assume that the initial w is 0 and the initial b is also 0, here you can set it randomly, I set it to 0, then we can calculate the slope when w is equal to 0 and b is equal to 0, then we can update w according to this slope and The value of b To calculate the slope, use the function just written. We bring in wb, and it will send back to us the slope in the direction of w and b . Then we can update the value of w and b according to the slope. For w, we can subtract w from the slope in the direction of w , and then multiply it by a learning rate. After my tests and experiments , I think 0.001 is a good learning rate. Well, if you want to update b , the same thing is to subtract b from b in the direction. The slope is then multiplied by this learning rate . I will make w equal to the result after the update , and then b is also equal to the result after the update . Let’s display it and execute it. You can see that after the update, w has changed from 0 to more than 0.87 , and then b has changed from 0 to It’s more than 0.14 , then let’s see if wb has changed from 0,0 to this . Is it really reducing the cost , that is, is it really going down? If you want to calculate the cost, you have written it before and copied it directly. The compute cost is directly copied , and we can use it directly here . Let’s see if the cost has really dropped after wb originally changed from 0,0 to this new value. I will print it out and put it It is printed out and executed. You can see that it was originally 6,040 and then changed to 5,286 , so the real cost is decreasing , that is, we are really going down , so let’s pause for a while. Let’s take a look at the place where the gradient was just calculated. How is it written? Find the compute gradient here . Whether it is to calculate the gradient in the w direction or the b direction, you should have found that it has a multiplication by 2. In fact, this multiplication by 2 can be omitted. Here we can omit it. It ’s gone. I’ll run it again after omitting it. Why can it be omitted? Let’s look down here . Do we take w to subtract the slope in the direction of w and multiply it by a learning rate , then b to subtract the slope in the direction of b and multiply it by a learning rate . What we just did in the direction of w and b The slope is multiplied by 2, which is equivalent to multiplying by 2 later. In fact, this multiplication by 2 is unnecessary because it will indirectly affect the size of the step . The size of the step is controlled by the learning rate. That’s good , so multiply this by 2. In fact, we don’t need to write it. We can just write the double here. Let’s just multiply the learning rate by 2 and calculate again. We can see that the result is the same , so we The above multiplication by 2 is unnecessary, so just omit it. I delete the multiplication by 2 and execute it again . This time, it is 6040 and then becomes 5656. This is the result of only one update. We only updated the result of w and b once. Then let's try to see what it looks like after updating 10 times. I will use a for loop to repeat this 10 times and let it run 10 times. Then let's record the values of wb and cost . The cost here is equivalent to recording it. Let’s use an f string to write it. At the beginning, I will first write how many times it is now, that is, the i-th update of the iteration . Then we will record the current cost. If the cost is this value , then the value of w and b will also be written. Record it and let’s execute it to see what it will look like . The cost is this value at the 0th time, then w and b let’s see if the cost is really decreasing . It is really decreasing. 5656 to 3161 I let him have an interval , and execute it again , which is much more comfortable , but he has a lot of decimal points, so it will look a bit untidy . If we want him to display only two decimal places , we can do this. Write it on the back: .2f will only display 2 digits after the decimal point. If you want 3 digits, it will be .3f and so on. Then I will only display 2 digits of w and b and execute it again so that it looks neat. It’s a lot, the cost is decreasing, and then the value of w is updated all the way, and the value of b is updated all the way. Let’s try it out. If you give it 20 times and then execute it, you can see that the cost is also decreasing all the way . When it reaches 20 times, only There are 1,705 left , but now he seems to be untidy again. The reason is that he occupies two grids by the 10th time . If we want to make the number of grids occupied by him the same, here we can Write on the back of him: Then I want him to occupy a few squares. Suppose I want him to occupy 5 squares, then I will write: 5 and then execute it. No matter what the number is, it will occupy 5 squares , which is much neater . In addition to calculating the cost In addition to the values of w and b, I will also record the slope of w and the direction of b . Here, it will also have a lot of decimal points , so I will let it only display 2 digits . Execute it again so that everything will be fine. It ’s recorded. Where we see the cost , we can see that it is still obviously declining. So I asked him to run a few more times to see how far he can drop . After running 100 times, it still seems to be declining. But we You can see that the right side seems to be becoming irregular again, and this side is becoming irregular again. The reason is that our numbers still have sizes, maybe two or three digits , so they will still become irregular. Here, if we really want to make it neat, we can use scientific notation . Here, I can write .2e and change it to .2e, then it will display two digits . Others are presented in scientific notation. What Is it a scientific symbol? Let’s see if we execute it . You can see that this is a scientific symbol . It writes 5.66e+03 , which means 5.66 multiplied by 10 to the third power, which is equivalent to 5,660 . If what you see here is 5.66 e-03 is 5.66 multiplied by 10 to the power of -3. We use scientific symbols to represent it , which is much neater. We see that the cost is still falling, so I let him run a little more here . Let him run 100 times , then I will let him run 1,000 times and then execute it. Let’s take a look . It seems that it is still falling at 42, and then it is still falling at 41. Let’s let him run it a little more . I let him run it once. 10,000 times, it is too much to print it 10,000 times, so I asked him to print it only once every 1,000 times. Here , I judged that if it is divisible by 1,000, I would print these. Information , execute it again , then it will only be printed once every 1,000 times. Well , we can see that this side seems to be a little messy. The reason is that it has an extra minus sign . If this problem needs to be solved, we can add one in front of it. Blank If you add a blank, it will give up one more bit to represent the symbol . To indicate this symbol , let each side give up one more bit and execute it again. This problem will be solved . Let’s take a look at the place where the cost is still declining. After 10,000 times, it seems to be still declining. He has multiplied 3.51 times 10 to 3.39 . Let’s let him run another 20,000 times to see. Here, let him display more decimal places . I asked him to run .4e and execute it again, so that he would display 4 digits , and it continued to decline, but the decline became very small . We just let him run it 20,000 times, and we can see that the decline is slow It's getting smaller and smaller. Let 's take a look at the slope next to it. We can see that the slope in the direction of w and the slope in the direction of b are also very, very small , almost close to 0. Then I will directly write this gradient descent process as a function , so that It is convenient to use later. I will call the name of the function gradient descent . There are many things to be passed in. The x and y are our data, the initial w and the initial b, and then the learning rate and us. The cost function used to judge the quality and the gradient function used to calculate the slope are finally the total number of times you have to run run iter and how many times you want to print out the data. I call him p iter, and I let him preset the value of p iter. 1,000 , that is, 1,000 times, it will be printed once , so here we change it to p iter, and for 20,000, we change it to run iter and the initial w is w init , so I make w equal to the initial b of w init. B init then calculates the function of the cost. You need to change the compute cost to this function , and then the compute gradient to this Then here, by the way, I store the cost in the process and the values of w and b. For cost, I call it c hist and make it a list. For w, I call it w hist. b is also b hist . The three lists are used to store the cost and the value of w and b of each time we ran so many times. Here I store the wb and cost after each update . So w hist.append Store w in and then b is also the cost , store it in and then finally I can do the return action . For the return, I just return the final w and b and all the w processes in our process All the w, b, and cost are all returned , so let ’s implement it . He said that there are some problems. Create another grid here. Before using it , we need to set the initial w and b . Suppose my initial w and b are equal to 0, and then the learning rate is 0.001 . I can also use science here. How to write the symbol I write 1.0e and then -3 , that is 1.0 multiplied by 10 to the power of -3, which means 0.001 . What about the cost function? Here we need to pass in the compute cost , which is the gradient function that we use to calculate the cost. What we wrote is the compute gradient. Here we can pass in the function as a parameter . Then it is run iter. Run iter . I can’t move it, I can move it, I don’t need to set it, okay, let’s execute it and see , it will return these 5 values , so I’ll also write it, this w is the last w, I call it w final b is also the last For b, I call it b final , and the stored wb and cost are good for us to execute. Why is it only printed once? Let’s see why it is only printed once . Oh, I accidentally wrote it into the for loop in the place of return So it should be executed again outside , so it should be no problem. 1000. Let’s see that the cost continues to decrease , and the slope is also decreasing. The final w can be seen to be more than 9.17 , and b is about 27 . I'll also print it out to see the final w, the final w and b values , let's take a look, the final wb is more than 9.14 and then 27.88 . The same here, I let it not have so many decimal points, and let it display two digits That’s good , so I wrote that the final wb of .2f is 9.14 and 27.89, then we can use this final value to make predictions. You should not have forgotten the problem we want to solve. Let me review it for you. Suppose today you are a The boss of a start-up company , you want to hire your first employee , but you don’t know how much to pay him, so you go to the market to collect some relevant information about this position , that is, his seniority and his corresponding salary . Now Well, you want to use a straight line to represent these data, and after the representation , you can use this straight line to predict how much salary this employee should be given . Now that we have found this straight line , we can make predictions. Action, suppose the employee who applied for the job today tells you that he has 3 and a half years of work experience , which means that his seniority is 3.5 years , then we can help him calculate how much salary he should give him. Here , his If the seniority is 3.5, we can use the found w final to multiply it by 3.5, and then add the value of b, which is b final, because we use a straight line to represent these data. A straight line can be written as w multiplied by x Adding b, now our x is 3.5, and the unit of this is k. He said that the seniority is 3.5, and he can give him about 59.88 k . There are too many decimal points, so I just give him 2 digits, er, one. That’s ok, one decimal point is good to execute again , then we need to give him 59.9 k. If we predict it, we can probably give him 59.9 k. There are more blanks here . Suppose another employee comes and he tells you that his seniority is 5.9 years. Then we Just make a prediction. Here I will type two more words to predict. The predicted salary is easy to execute . If he tells you that his seniority is 5.9, then the predicted salary is probably 59.9k. If not, I will change it to 5.9. Execute , the predicted salary is about 81.8 In this way, our problem is solved. We can predict salary in this way . Next, I will draw some data into a graph to see . For example, I can draw it into a graph to see the cost here. If we want to draw a picture here , we can use the drawing tool matplotlib and then use numpy. Let’s draw the process of the 20,000 update cost decline. We can use plt.plot to draw it as a line . The value of x here we use np.arange 20,000 times, so it is 0~20000 and the value of y is the historical data of our cost. The updated data is stored here, so we can display it to see Optimistic about the implementation, you can see that he looks like this , then I will add some tags and titles for him, and set his title. The title here is a total of 20,000 updates and its cost. So I will write iteration vs cost here and then Then add an x label for him. The x-axis is the number of updates. It is iteration, and the y-axis is cost. Execute it again , so that the subtitle and labels are all up . From this picture, we can find that the front of him has dropped very quickly. But later, it will be slower. If we want to look at the previous paragraph in detail , suppose I only want to see 100 0~100 , then I will change it to 0~100 here , and then I will write: 100 also It is the first 100 data, so it can be executed again. This is the descending process of its first 100 updates. If you want to see other intervals, you can also set it yourself. It seems that there is one less a iteration , so you can execute it. Finally, we can also use w Follow the update process of b. The 20,000 update process draws it with a picture . Let’s first go to the previous cost function and copy the 3D picture. The 3D picture is here . We copy the action of drawing a 3D picture over there . At that time, we set w and b to be between -100 and 100 , so the calculation of w and b from -100 to 100 should also be copied here . Here, w and b are from -100 to 100 and its cost Here I will open another grid to do calculations I will let him calculate first , then we will use Chinese fonts here , so we have to do the action of downloading fonts, the same way I went to the previous cost function , found the place where the fonts were downloaded, copied them, and then opened a grid for him to download , okay here Let's wait for him and let him finish the calculation, installation and drawing . OK, the drawing is finished. I 'll adjust its angle . It was originally 0 degrees and 0 degrees. I made it 20 degrees and -65 degrees , so I can execute it again . The angle is good, the red one. The point is the lowest point. Then we draw the update process of w and b with a line. If we want to draw a line, we can use the plot to pass in the w hist b hist and the hist of the cost that we saved , that is For the c hist side, let me run it first and see that he draws the line , but I don’t know where the initial point is , so I draw the initial point as well . If I want to draw a point, I use scatter Here I directly use the copied one . If we want to draw the initial point, it is its 0th value b hist is also the 0th value cost is also the 0th value. Here I make the color green and execute it again, and you can see the green. The point is our initial position , and then it updates and updates all the way to this side , basically reaching the lowest point. Ok, it’s updated all the way, so I think the color is a little bit of an eyesore . The color of the surface is a bit of an eyesore , so I changed the color I don’t think it’s necessary to cancel the border , so execute it again and it looks better . I set its opacity to the first point and let it be 0.3 and execute it again . It looks much more comfortable. It ’s a Such a curved surface. Then our green point is the initial point . It is updated along the way. Basically, it is about to reach the lowest point , because the red point is the lowest point. Well , I can also play around here . If I am making a gradient When descending, I set a different value. Here I copy a copy to the following and copy it here . If my initial point here is set at -100 and -100 , let’s try it and see what the result will be like. Well, after running 20,000 times, we will also get the final w and b and the stored wb and cost for each time. We will execute the drawing here again to see what it will look like . You can see this time. The initial point is here , and then he goes all the way down, down, down, and down... and then turns around like this, which is also close to the lowest point . Then I can try it if I didn’t let him update it here. Many times , I only asked him to update 1,000 times for easy execution . I only asked him to update it 1,000 times. Let me see what it looks like. He walked down this time , only came here and didn’t move forward because we didn’t update enough times If there are many, you can also adjust other things . For example, I can increase the learning rate a little bit. I set it to be 1.0*10 negative quadratic, which is 0.01 . Then I will try again here . He still has reached the lowest point , then you may find that the initial point we set is not -100 -100. According to you, the initial point should be here . Why is it here? The reason is because in fact, we have stored it here. It is the result of the first update, so he ran here after the first update -100, -100 is the initial place here, he ran here for the first update , and the storage here is from the first update When the last result is reached, I will try again . If I increase the learning rate a little more , let’s say I change it to 5.9 for easy implementation . Let’s see what it will look like. You can see that his stride this time is very large. He has become like this. Back and forth, but it seems to have reached the lowest point, so what if we make it bigger? Here , I change it to -1 of 1.0*10 and execute it. It’s easy to execute. You can see that something terrible happened . Ours The surface in the picture is gone. Why ? The reason is that it has exceeded the range too much. When we drew the surface, we set the values of w and b to be between -100 and 100. Now it has exceeded too much. It will cause the surface to disappear. When our learning rate is set too high , it is possible that each update is not closer to the lowest point , but farther away from the lowest point. In our example, every update is farther away from the lowest point. The farther the lowest point is , so it will lead to such a situation that the entire surface disappears. The example just now is like this. If you set the learning rate too high, it may go farther and farther away from the lowest point . One step to this, the second step to this , and the third step to the point where he doesn’t know where he is going. Here , everyone can play and see by themselves , set different initial values of wb , and set different learning rates and different settings. This is the implementation of our gradient descent . After completing the simple linear regression, I will briefly summarize the process of machine learning. In our example , the first step is to prepare the data first . According to this The distribution of the data , we think we can use a straight line to represent it , so we use a straight line to represent the data, and then we need to find the straight line that best represents the data, that is , the line that is most suitable for the data . What kind of straight line is the most suitable for these data? We always have to give him a standard for judging, so we set it up. As long as the sum of the squared distances between these data points and this straight line is smaller , then we will call this straight line. The more suitable these data are , on the contrary , the larger the size, the less suitable it is. Well , after we have a scoring method, we can never exhaustively enumerate all the straight lines and then score them all and find out from them. The best one , it looks too inefficient . We must find the most suitable straight line in an efficient way. Here , we use the gradient descent method . In fact, the whole process is probably machine learning. The process is the same when you apply it to other examples . First of all, you need to prepare the data first , and then you need to help him set a model based on your data . In this example, the model we set is a straight line . After setting the model, it contains There may be some parameters that you need to adjust. In this example, you need to adjust w and b . How to adjust is called good , and how to adjust is called bad. We always have to give him a standard for judging, so we need to set a cost function . For this example, we set it like this . Then it is impossible for us to exhaustively enumerate all parameter combinations and then calculate their cost and then use it . It is too inefficient to find the best one . We must find the best parameters in an efficient way. In an efficient way , that is, we need to set an optimizer . The optimizer is translated into Chinese. Optimizer In this example , the optimizer used is gradient descent , which is probably a simple machine learning process . No matter what other examples it uses , it is actually the same . You must first prepare the data and then set up a model. Then set a cost function and finally set an optimizer OK. The second model I want to introduce to you is called multiple linear regression. It is multiple linear regression in English. It is actually similar to the simple linear regression we introduced before , but it can be compared. There are many features , so let’s take a look at what problem we want to solve today. In the previous example of simple linear regression , we used seniority to predict salary, but think about it carefully. We only want to predict salary based on seniority , right ? It’s weird, we should consider more factors , so now we have collected more complete information. In addition to seniority , we also collected his education background and where he works. So now we want to use seniority, education background and work To predict his salary, that is to say, our features have more education and employment . If we also want to use a linear model to represent these data , then we can use multiple linear regression. Multiple linear regression uses mathematics. It can be written as y is equal to W1 times X1 plus W2 times X2 plus W3 times X3... you can multiply all the way depending on how many characteristics you have, and finally add a b to that In our example, there are three features , which are seniority, education and place of work , so the mathematical formula will be long when written out. What we want to predict is the monthly salary. The monthly salary will be equal to W1 multiplied by the seniority. Our first feature plus W2 Multiply by the second characteristic of education, add W3 and multiply by the third characteristic to work , and finally add a b. Then our goal is to find a combination of W1, W2, W3 and b so that he can best express This information is what our multiple linear regression needs to do . Before we start to find the most suitable w and b, I think everyone should have discovered a problem . In our formula, we multiply W2 by education and then take W3 is multiplied by the place of work , but we can see that the two characteristics of education and work place are both text. How to make the text into a number? This is not good , so we must first combine these two features. Do some processing and convert them from text to numbers before we can do the calculation here. Let’s deal with the feature of education first . There are three possible values for the feature of education, which are below high school, university and master’s degree . From this feature, we can see It can be concluded that it actually has a relationship between high and low. If there is a relationship between high and low, can we use numbers 0, 1, and 2 to represent these three situations? 0 is used to represent the smallest high school and 1 represents a university. 2 is a master’s degree or above. If a feature has a size relationship and a high-low relationship , we can use this method to replace these words. This replacement method has a name, it is called label encoding, and we replace the education just now. After the fall, it will become like this. Come back and take a look. We use 2 for master’s degree and above, and 1 for university, and 0 for high school and below, so here it will become 1 2 0 and so on . Then we will first Let’s implement this label encoding. First, read the data in. The reading action is the same as before , so I directly use the copied one. However , in the part of the URL , because our data this time has two more features. So there are some changes. We are on the side of salary data because it is the second edition, so we write 2 at the back and read it. We will display it to see OK. Our data looks like this. There are three characteristics in total , namely , seniority, education and In the place of work , we want to deal with the feature of education first . We want to convert it from text to number first. Well , we can do the conversion in this way . Let’s get the data. Let’s take a look at the feature of education . It’s called education level and we’ll get it. This feature , let’s display it first to see what he looks like. We want his university to be 1 , master’s and above to be 2, and high school and below to be 0. Here we can do it like this, write a map on the back and put it in it. Write a dictionary. We want to correspond to high school and below, let it correspond to 0 , then university, let it correspond to 1, and finally, master and above, let it correspond to 2. We write it like this, it will help us do the conversion, and put this feature into it The value inside is converted like this. After the conversion, I will change the value of this feature . After the change, we can display the entire data and see that it will now become the corresponding value. It’s worth it . The first piece of information here is a university , the second is above a master’s degree, and the third is below a high school , so it will become 1 2 0 , and then the same conversion will be done below . In this way, our label encoding is completed. After dealing with the feature of academic qualifications, let ’s deal with the feature of place of work . There are three possible values for this feature , which are city a, city b, and city c . Here, you may wonder whether we can use the same method . We use The label encoding uses 0 1 2 to represent the cities a, b, and c, but if you think about it carefully, do you think this is okay? We don’t know whether there is a high-low relationship or a size relationship between the cities abc . If we If it is represented by 0 1 2 , whoever is 0 and who is 1 and who is 2? If we want to convert this feature with no size relationship, we can directly convert it into multiple features , like this . We originally There is only one characteristic of a city , so I changed it to three characteristics of city a, city b, and city c. The first piece of information here is that he originally worked in city a, so we gave him the attribute of city a. 1 Then the two attributes of city b and city c are 0 , so look at the second data. The second data originally worked in city c. So we give it the attribute of city c as 1 and others give it 0 and so on. This method is called one hot encoding as long as we want to convert a feature that does not have a relationship between size and height from text If it is converted into a number, then we can use one hot encoding , which will change from one feature to multiple features. See how many values this feature originally had , and how many possible values it has, then it will become several features. Take our example Say its possible values are cities a, b, and c , then it will become three features, city a, city b, and city c . After converting it like this, in fact, we can also use one of these three features as Delete why ? The reason is that among these three features , in fact, we only need to know the value of two of them to deduce the value of the third feature . Suppose we delete the feature of city c now. Let ’s see if there is any way Deduce city c through cities A and B. It looks like this now . If city a is 1 and b is 0, then city c must be 0, because only one of these three features will be 1, so let’s look at the second If city a is 0 and city b is 0 , then city c must be 1, because it is not city a, not city b, it must be city c , and so on . Therefore, we have a way to deduce the third through two of the characteristics. The value of a feature . At this time, we can delete one of the features . Let’s take a simpler example . Suppose you now have a feature called gender . There are only two possible values , either male or male. Girls , obviously, this feature does not have a relationship between high and low, and there is no relationship between size, so we can use one hot encoding to turn it into two features that are divided into boys and girls. But it is either a boy or a girl , so we can derive it from one of the features. In this way, we don’t need to divide it into two features, because too many features will make our calculations more complicated, so we can delete one of the features. Okay, so go back. In the original example, among the three characteristics of cities a, b, and c , we only need to know the values of two of them to derive the third one. In this way, we can delete one of them . Here we choose to The feature of city c is deleted . Let me remind everyone that not all the features that can be derived should be deleted , because even if some features can be derived , they have special meanings or they can speed up our calculations. Efficiency , then we won’t delete it . But in the example of one hot encoding, we can delete one of the features after conversion . Well, then, we will directly implement one hot encoding . Now our data It looks like this, we want to convert the feature of city , from text to numbers, so for this feature , we can use one hot encoding to convert it. Here we can use the preprocessing under sklearn and then Well, let's use the one hot encoder sklearn suite in it. It provides a lot of things we will use when doing machine learning , such as the one hot encoder we introduced now , which can help us quickly convert This one hot encoder is a category , so let’s create it first , let’s create a converter first , or you can say create an encoder first , here I call it the one hot encoder, after creating it, we can first let This encoder allows this converter to read our feature , the feature of the city, so here we can write some fit and let him see the feature of city , but here it is important to note that it only accepts a binary input It is a one-dimensional matrix , so we can’t write it like this because it is a one-dimensional matrix. If we want to make it two-dimensional, we need to add a pair of square brackets and let him use two pairs of square brackets. It is a two-dimensional matrix , let him read all the values of this feature , and then we can transform it. We can use the transform under it to do the transformation . It also passes in the feature we want to transform , and here is the same A two-dimensional matrix is required , so two pairs of square brackets are required . Here I will call the converted result city encoded, and then we will display it directly. After execution , you can see that the converted result looks like this. Why does it look like this? The reason is because of this one After the hot encoder is converted, it is preset that what it will send back to us is a sparse matrix . OK, it is a sparse matrix. It doesn’t matter if you don’t know what a sparse matrix is. What we want to see here is that it will send us back a The complete matrix, we can write toarray later for the complete matrix, so it will return the complete matrix to us and execute it again . You can see that this is the result we want. It will put a total of 3 in a feature The possible cities a, b, and c can be turned into 3 features OK and 3 values . Then we can replace the result of the conversion with the original feature and replace the original feature of the city . We can do this Our original data is data. Let’s execute it once OK. The original data looks like this. If we want to add 3 more features for him , that is, if we add 3 more columns, OK is 3 columns . If we want to add 3 more columns and 3 more features for him , we can write it like this. Write two pairs of square brackets . I want to help him add a cityA, then a cityB, and finally a cityC, and then we Its value can be specified as the result after the conversion we just made , and then we can display it to see. You can see that writing it like this will add 3 more columns and 3 more features cityA BC and then we can put The feature of city is deleted because it has been converted into these 3 features. Then, among these three features, we can delete one of them. I delete cityC , then we can do it like this . We can write data.drop Then write the features we want to delete in the first parameter. The features we want to delete are represented by a list . We want to delete the two features of city and cityC. Then write the second parameter that we want to delete. Where is the axis of the deleted thing? The axis we want to delete is a column. City and cityC are two columns. If the thing we want to delete is a column , then we need to specify the axis axis as 1. If we If the thing you want to delete is a line , then here we need to specify the axis as 0 Okay, so here we write 1, and then we will display the deleted result to have a look. Execute , you can see that the two columns of city and cityC are gone now, so we will put the work place This feature is also processed . After converting the text , we usually divide the data into a training set and a test set when training the model. Training the model is to find the most suitable parameters . In our example , it is to find the most suitable parameters. Appropriate w and b Usually when we train the model, we don’t use all the data , we only use part of it, what to do with the other part , and the other part is for testing , because you think, if we Use all the data for training , and then find out the best set of w and b , but how do we verify the effect of this set of w and b ? Do we want to test it ? If we want to test it We will test it on unfamiliar data. We will not use the training data for testing , because the training data machine has already seen it, which means that he already knows the answer . If he already knows the answer, we If we still use this data for testing, it won’t be accurate, so in order to have unfamiliar data for testing , we usually divide the data into training sets and test sets, like this , assuming we have 10 data . Usually, the training set will account for about 70% to 80%. In this example, we account for 80% , so 8 data will be the training set , and then 20% of the data , that is, 2 data will be the test set. The training set is to take To find the best w and b, the test set is after we find the best w and b and use it for testing to see how it works . After understanding what training sets and test sets are, we will directly implement them. OK Now our data looks like this. First of all , let’s separate x from y . Everyone should remember that the model we want to use is multiple linear regression . Multiple linear regression is written as a mathematical formula , which is y=W1*X1+W2 *X2 is multiplied all the way down to see how many characteristics you have. Finally , add a b . In our current example, there are 4 characteristics , namely seniority, education and job location . The y we want to predict will be salary , so we First separate x and y. Here, x is equal to the four characteristics we need to obtain it . The first is seniority, the second is education , then cityA and cityB , and then y will be equal to our salary. Here I display x and y to see that x is these 4 features , and then y will be the salary. Then we can divide it into a test set and a training set . Here we can also use sklearn, which is very useful. The tool uses the model selection under it , and then we introduce the train test split under it, and then we can use it to help us divide it into a test set and a training set. In the first parameter, we pass in our x, which is the feature, and then the second Enter y in the first parameter, and then we can specify the size of the test set. Here we can write the test size . I want it to be 2. Here, I can make it equal to 0.2 and write it automatically. Take 2 components of our data as the test set and the other 8 components as the training set . If you want the test set to account for 30% , you can write 0.3 and so on . If you write it like this, it will return 4 The values for us are x for training, x for testing , and y for training and y for testing, so I will call the first one x train and the second one x test and then y train and y test will return these 4 values to us , then I will display it for a look. I will display x train, which is the x we used for training. You can see it and take out these data as a training set . Let's take a look at its length . Let's take a look at its length . Its length is 28. Let's take a look at the original length of x. The original length is 36. Let's take 20% as the test set, which is 80% As a training set, let’s take 36 and multiply it by 0.8 to see what it is. It is 28.8 . So, if it is 28.8, it will automatically eliminate the point 8. Take 28 data as a training set . Let’s take a look at the length of the test set. How much can you see in total? x has 36 pieces of data , and after we divide it , 28 pieces are used as training sets and 8 pieces are used as test sets . The same is true for y here, and we display x train Come out and take a look here, and you will find one thing here , that is, the x train seems to change every time we execute it. Let’s take a look at the first data. It is 5.1 0 and then 1 0. We execute it again and it changes to 6.9 2 1 0 Execute it again and change to 7.8 2 0 0 Why did it change? The reason is because of the splitting process. It will split randomly for us by default , so the result of each split will be different. If you want him to split If the result is fixed, here we can also set another parameter called random state, and then you can specify a number for it. Each number will correspond to different segmentation situations . Suppose the number I gave him is 87 , so we can execute it again. Seeing that the result of its division this time looks like this, the first stroke is 4.6 1 1 0. Now, when I fix the random state equal to 87, I execute it again , and you can see that it will not change. But if I change this number to It makes changes. If I change it to 86 and execute it again, it will change again . If I continue to execute it without changing 86, it will not change. So if you want to fix the split result here , you can give He specified a number, here I will specify 87 for him because I want the result of its segmentation to be fixed , so that it will be convenient for me to do a demonstration later . Okay, then I will also display it in the x test to see how long it is like this. 8 pieces of data , then y train will also display it OK, there are these, then y test is done, then we have successfully divided the test set from the training set . Finally, for the convenience of subsequent calculations, I will first put x train and x test convert it into numpy format, now they are both in pandas format , so it will be very beautiful after execution, OK, there will be such a grid , if I convert it into numpy format, it will look like this, I can write after it to numpy , it will become a matrix, and it will be ugly if it becomes such a matrix , but it will make my subsequent calculations more convenient , so here I will convert it first . The x test is the same, and I will also convert it It can make my follow-up calculations more convenient . After converting the text into numbers and dividing them into training sets and test sets, we can return to the model. The model we want to use is multiple linear regression , so it can be written like this , but our features have changed from 3 to 4. The place of work has become two features of cityA and cityB , so we have to rewrite it Now that it has become like this, our goal now is to find a combination of W1 2 3 4 and b, so that the monthly salary predicted here can be closer to the real data . Well, let’s implement this part first . Let me set it up first. The value of w and b , at the beginning, I set w randomly, it has 1 2 3 4, so I use a one-dimensional matrix to represent it . There are 4 values in this one-dimensional matrix , so here I first introduce numpy and call it np Next, I create w and let it be a matrix with 4 values. Suppose I let it be 1 2 3 4. This 1 2 3 4 means our W1 2 3 4 and then we need to set a value of b In the same way , I set it randomly and let it be 0. After setting w and b, we want W1 to multiply the first feature , W2, the second W3, the third W4, and the fourth . Now because We are in the training phase , so the feature we want to multiply is in the x train . I first display the x train , and it looks like this . It has a total of 4 columns and each column represents a feature , so what we want is this 1 is multiplied by the first column, then 2 is multiplied by the second column, 3 is multiplied by the third column, and 4 is multiplied by the fourth column. If we want to multiply like this , we can directly write it like this. We directly multiply x train by w , then it will be It will automatically multiply 1 by the first column, 2 by the second column, and so on. Let’s execute this. This is the result of the multiplication. The first column here is W1 multiplied by X1 , the second column is W2 multiplied by X2, and so on. Then what we want is for them to add W1 multiplied by X1 plus W2 multiplied by X2 and then add 3 plus 4 , that is, we want to add each row here , we want to compare each row If you add it, you can do it like this. First put the calculation result here in brackets , and then use some sum. If I only write this way here, it will sum up every value here . But what we want is every Well, what we want is to sum up in a row. Then I can set the axis equal to 1 and equal to 1 to do the sum in the horizontal direction. If you want to do the sum in a straight line, you can also set it to be equal to 0 Okay, let's try to execute it . This is the result of the sum . Each value here is the sum of each line just now , which is the result of multiplying W1 by X1 plus W2 by X2 plus 3 plus 4. Here After all the additions are done, we need to add a b at the end , so if I add a b here, if I add b here, it will add b to every value in it , but now we set b to 0, it seems to be It doesn’t come out , so I set it to 1 , let’s execute it again, then add it like this, every value here will add 1 , so every value calculated here is our predicted salary, I call it y pred Next, we want to find out the most suitable combination of w and b. To find out the most suitable combination of w and b , we must first define what is called the most suitable , that is, we need to set a standard for judging. That is, we are going to set a cost function. The cost function here can be set the same as the previous simple linear regression . Because we also want the predicted monthly salary to be as close as possible to the real data. So I set it as real. The reason for subtracting the predicted value from the data and then square it is because there may be negative numbers in the subtraction here. For the convenience of calculation, we directly square it so that there will be no negative numbers . So what is our current goal? It is to subtract the predicted value from the real data and then square it . The smaller the value, the better , that is, the smaller the cost here, the better . Then we will directly implement the cost function . Our cost function is set to subtract the real data. Drop the predicted value and then square it. The predicted value has been done in the previous step . The y pred here is the predicted value. Let me display it first . This is our predicted value. What about the real data because we are now In the training phase , the real data is y train , so we use y train to subtract y pred. I also display y train first to see if it is OK. There are a total of these sums. We use y train to subtract y pred , which is After subtracting each sum, we will square it for easy execution. You can see each value here . It is the result of subtracting the predicted salary from the real salary and then square the result . We hope The smaller the value here, the better. If you want the value here to be as small as possible , we can calculate its sum or calculate its average. Here I will calculate its average. I hope its average should be as small as possible . Calculate it You can enclose it in brackets and write “mean” at the end for easy execution. You can see that this is the average result . Here I will directly write the cost calculation process as a function , which can facilitate us to bring in different w and b. I call it compute cost. It also passes in x and y, which are our real data , and then passes in the values of w and b . In it, the value of y pred is calculated first, which is our predicted value , but here is x In train, change it to x to calculate the predicted value, and then we can calculate the cost. The calculation of cost is here, and the words here are the same. In y train, I changed it to y, the real data, subtracted the predicted value, squared it, and then took the average. This is If the cost is good, the last thing is to return the cost and try it directly. Here we will use it directly. The compute cost here is to pass in x train and y train because we are in the training phase now. Then w and b are the same as I pass in first. Enter these two values , first pass in these two values , let’s execute and see, and you can see that the calculation result is the same as above , if I set b here to 0, then set this to 0 2 2 4 Well, execute it again. The cost calculated by the combination of w and b in this way is even higher than the one just now. The one just now is 1,772 . How about this ? Let’s define the evaluation standard, which is the cost function. After setting the evaluation The standard is after the cost function, and then we need to use an efficient way to find a set of w and b, which can make the cost as low as possible. The efficient way is to set an optimizer, which is the optimizer . In our example, we can also use the gradient descent method . You should remember that it changes the parameters according to the slope . However, when we used the simple linear regression before, the parameters were only two w and b . Now our parameters are Becomes 5 W1234 and b Let's take a look at how to update the parameters before. If we want to update w , we need to subtract the slope in the direction of w * learning rate from w. To update b, we need to subtract the slope in the direction of b * learning rate from b . In fact, the parameters are updated. The method is the same, but we have changed from two parameters to five parameters . Now we have five parameters W1234 and b. To update these five parameters, we need to subtract the slope of their direction*learning rate individually . How to calculate the slope in each direction ? In fact, it is the same as before . We only need to differentiate the Cost function to get the slope in each direction . Our Cost function is to subtract the predicted value from the real data and then square it to write it in a mathematical formula. The sub is to subtract y pred from y and then square it. We can spread out the y pred to become W1*X1+W2*X2... multiply to 4 and add b at the end if we want to know W1 The slope of the direction is to differentiate it with respect to W1, or to be precise, to make a partial differential with respect to W1 . If what you want to know is the slope in the direction of W2, then you need to make a partial differential with respect to W2 , and so on. W3, W4, and b are both It’s the same. It’s okay if you don’t know what differential is at all here , because there are many tools that can help us to do differential automatically, so the slope in the direction of W1 looks like this after calculation, and you can see a very long string , but in fact, we can I found that this string of W1*X1+W2*X2 plus 3 plus 4 and then adding b is actually y pred, so I simplified it and it will look like this , it is 2*X1 and then multiply Use the result of y pred-y and then use the same method to calculate the slope in the direction of W2. It will grow like this . We can find that in fact, he just replaced X1 with X2, and everything else is the same . From this we can see that the slope in the direction of W3 is the If the side is replaced with X3, then the slope in the direction of W4 is replaced with X4, and the slope in the b direction is calculated to be longer. Here , there is no need to multiply an x . Here we can see whether it is in the w direction or in the b direction. In the previous place, there is a multiplication by 2. I don’t know if you still remember. In fact, we have said that this multiplication by 2 can be omitted , because when we update the parameters , we will multiply it by another learning Rate, so this is multiplied by 2. In fact, we can leave it to the learning rate to multiply. Here, we can omit it all and become like this , so that we know how to calculate the slope. After we return to the original place , the slopes in all directions are now. We will forget it , but it will be multiplied by a learning rate later . How to set the learning rate? Just like before, it is through testing and experimentation . You can’t set it too large , because it may directly cross the lowest point . Or it is farther and farther away from the lowest point , so you can’t set it too small , because it may reach the lowest point forever. After setting the learning rate and calculating the slope , we only need to keep updating the parameters. Let it approach the lowest point step by step , and then we can find the most suitable combination of w and b we want . Then we will directly implement the gradient descent. First, we will calculate the slope in each direction because the slope in the b direction is relatively simple. So I calculate it first, I call it b gradient , it is y pred minus y, now it is the training phase, so our y is y train, this y pred has been calculated before , so I just copy it , but we Now it is the training stage , so I will change the x here to x train , and I will display the calculated results to see , because our training set has 28 records , so there will be 28 values here , so let’s do it An average , put it in brackets and then use the point mean to calculate the average . The average is -46.94 . After calculating the slope in the b direction, let’s calculate the slope in the w1 direction . I call it w1 gradient, which is y pred minus Drop the y train and then multiply it by x1 . This x1 is actually our first feature . We are in the training phase now, so our feature is in the x train . I will display the x train first. Seeing that it shows a total of four columns , these four columns are our four features . Here, x1 is the first feature, which is the first column here . If we want to get the first column here, we can write it like this. Write a pair of square brackets and then:, 0 means that we want all the values in the first dimension : means all of them, and the comma 0 means the values in the second dimension. We only need the most The first one, we can see that this matrix is a two-dimensional matrix , we want all the values in the first dimension , that is, we want all the values in the first square brackets here . We all need the first value of the second dimension. The second dimension is every row here. We only need the first value , which will be the first column here . We execute it, and we can see that it will take out all the values in the first column , which is our x1, so here I can replace it , replace it , and then we will display the calculation results to see. It will calculate a total of 28 values , because we now have 28 data in the training set , so let’s take its average , enclose it in brackets and then take the average and execute it , so that the slope in the direction of W1 is calculated 295 -295.8 and then the calculation method of W2W3W4 They are all the same , so here I directly use the copied one . Here we change it to 2, 3, 4 , and then we want to get the second column . Here is 1, then 2, 3 , and we will display the results of the execution. Come out and see W1, W2 OK execution, you can see that this is the slope of W1 W234 in their direction , here I use a loop to write it, it will be more convenient , so I recreate a w gradient , which stores the slope of W1234 direction, here I will First create a matrix with all 0s in it , let it have 4 values , so I can display it first, and I will let it be 0 first if there are 4 values in it , but it seems not good to directly write 4 here. The meaning of 4 is How many features do we have? How many ws do we have because we have several features ? If we want to know how many features we have, can we directly get x train? If we want to know how many columns x train has , we can do this here. Let's get its shape and it will tell you that it has a total of 28 rows and 4 columns , that is, it has 28 values in the first dimension and then There are 4 values in the second dimension . The value of the second dimension is what we want . We want to get the value of the second dimension. Here , I can get 4 with square brackets 1. So this It will be better if I change it to this Then I use a loop to ask him to calculate the slope here. He will run it four times in total , so I will paste it here directly , and the i-th slope of w gradient will be the calculation method here. But here I will change it to i , so it should be no problem. I will also display the result after the calculation . You can see the slope W1234 in the four directions is these four values. Well , what about this ? Calculate the slopes of the w and b directions. This is the w direction and this is the b direction . Next, I write the process of calculating the slope as a function so that it can be used later . I call it compute gradient . Pass in x and y, which are our real data , and then there are the values of w and b . Here we first calculate y pred . Then y pred has already been calculated. I will paste it directly to the calculation of y pred. After finishing, I create a blank matrix , create a matrix with all 0s in it , and then change this side to x and then use a for loop to calculate the slope of each w, here also changed to x and here changed to x Then after the calculation here, we still need to calculate the slope in the b direction . The slope in the b direction is also pasted here . It is also changed to y here. After the calculation, we will return the result to w gradient and b OK to execute . Then we Just use it directly and see here . The x to be passed in is now the training phase, so x and y are both x train y train w and b. I pass in the value here , so let’s execute it and see. The result is exactly the same as above, w and b are the same. Let’s make a change. Let’s say I change b to 1 and then change this to 1 2 2 4 and execute it again . Then the calculated slope will be different. It’s easy to calculate. After the slope, then we try to update the parameters. To update w, we need to subtract the slope in the direction of w from w , and then multiply it by a learning rate . It is the same when updating b . Just change this side to b and then the learning rate here. Let me set it casually. Suppose I let it be 0.001 as the initial value of w and b. I also let it be like this. The slope in the direction of w and b. We will directly call this function to do the calculation, and it will send the slope back to us. In the direction of w and b, here I will display the updated results of w and b to see if there is any difference. You can see that it was originally 1 2 2 4 and then it becomes 1.2 more than 2.0 more than 2.0 more than 4.0 and then b There are also changes. Let's see if its cost has really decreased after such an update , that is, has it really gone down? Here, I will directly use the function here to calculate the cost. Calculate it before it is updated , and then I will print out the result . After the update, I will display it again to see if it is really smaller . You can see that it is really smaller. It was originally It is more than 1,800 and then becomes 1,675. After confirming that it has become smaller, we need to repeat the actions here to let him keep updating the values of w and b. Then we write it as a function called gradient descent In fact, we have already done this function in the previous simple linear regression. The writing method is basically the same , so I directly copy it, so we can execute it directly . After execution, we can call it directly. We have called before , so I copied it directly , but there are a few places to change here . The initial values of w and b here, I changed it to this learning rate, I set it to 0.001, so This side can also be written like this, I just let him run it 10,000 times first , well, what about this side , because we have now been divided into a test set and a training set , so here we are now in the training phase, I write x train and y train after The places are basically the same and we don’t need to change it . The names of compute gradient and compute cost are the same. Well, I will execute it and you can see the errors it finds . The reason for the error is because of the error here. We use the numpy matrix format for w and w gradient. Here , we use the numpy matrix format for w and w gradient. If we use the numpy matrix format , there is no way to write directly: .2e to put here Format conversion , so let me delete it first and see if we execute it again , so there should be no problem . You can see that there is no problem. You can see that the cost is decreasing . Then the value of w The value of b and b are both being updated here , but if we don’t format it like this , it will look ugly and uneven . Here are the slopes in the direction of w and the slope in the direction of b because w has W1234, so there are 4 values here , and there are W1234, so there are 4 values , but now it is ugly to execute. If we want to make it more beautiful , we can directly set the format of the numpy matrix that it prints out . By the way, we can set np.set_printoptions to set its formatter. If it is a dictionary , then I want the format of the floating-point number to look like this. The colon is blank. The format of 2e is written at the end . How about setting it like this? It is equivalent to writing in this way, it is equivalent to writing in this way, it will print out every value in this matrix in this format , then let’s execute it again here , and you can see here It has become much more beautiful. Let’s take a look at the place where the cost has been decreasing. There is no problem . After running 10,000 times, we can find that the cost seems to be still decreasing. Here I will test to see if I let Its learning rate is a bit higher , well, it seems that it is still declining , and the decline is obviously faster than before, so there should be no problem with this learning rate . Slowly, it seems that the speed of decline is getting slower and slower. Here , I will first set it like this and then let it It’s good for him to run 10,000 times . Here, everyone can go and play for themselves . Set different initial w, different initial b, different learning rate, and different times of running. What kind of results will there be? Here I will put it at the end. Find out w and b and display it to see. You can see that the w and b it finally found are these 5 values . Here I want to verify whether these 5 values are good or not. We can use it in On the test set, we take the w we finally found and multiply it by x test, which is the test set. This multiplication is W1*X1 W2*X2, and after multiplying 3 and 4, we need to add them together, so I put It is enclosed in brackets to add it. I want it to add in the direction that the axis is equal to 1. At the end, we need to add another b plus the b we finally found. The result calculated here is our prediction on the test set. I call it y pred and we can compare the difference between the prediction on the test set and the real value . Here I display it as a Pandas dataframe The format will be better. It will have a grid . I let him have two columns. The first column is our prediction result y pred on the test set , and the second column is the real data y test on the test set. Let’s see if there is any difference. How many executions? If there is an error here , type an extra u and execute again . It is still possible to execute again if there is wrong data. You can see that the column on the left is the result we predicted on the test set, and the column on the right is the real data on the test set . We predict It is more than 4.0 and the real 43.8 does not seem to be much different. 67.7 72.7 seems to be worse. 61.6 60 OK. The difference feels okay. Well, then we successfully implemented the gradient descent and found the final w and b . Use the final set of w and b on the test set. The result is like this . Finally, if we want to know more accurately whether this kind of prediction is good or not , we can also calculate it besides directly looking at their errors. Because the value of cost is our criterion for judging good or bad, so here we can calculate its cost , then we directly use compute cost. Now we are the test set , so x is x test, y is y test , then w and b are We finally found the w final and b final, let’s calculate and see, its cost is more than 18.1 , then let’s take a look and compare the cost during the previous training . When we are training, the final cost is reduced to 2.52*10. It is more than 25.2, which seems to be no problem , because our cost on the test set is smaller than that on the training set, which means that its performance seems to be better than ours on the training set , so after the cost calculation and the above crossover After the comparison, if the error is acceptable to you , we can apply this model to the real situation. Suppose someone actually came to interview today. He told you that his seniority is 5.3 years, and his degree is above a master’s degree. After the interview in city A , I think I want to admit this person, but I don’t know how much salary I should give him . At this time, we can use the trained model to predict how much salary we can give him . First, we must first Do some processing on the data , because there are texts here, we need to convert it into numbers first , how to convert the data on the test set, and how to convert the data in the real situation , I will directly convert here. Seniority 5.3 is no problem Use it directly. After master’s degree or above, we use label encoding to convert it into 2 and then work. We use one hot encoding to turn it into 3 features , and then delete one of them, so it will become 2 features . So here is city A The third feature is 1, and the fourth feature is 0. Here I use a matrix to represent it. I call it x real , which is equal to np.array. If you have feature scaling here, it is the same in real situations. We also need to do feature scaling. How about feature scaling? Let me look up and see where our feature scaling is done. Here is how our test set is scaled . Then we will do scaling in the real situation. OK, I will directly copy ours . The x test is scaled in this way , so the x real is also processed in the same way . Let’s display the zoomed results to see. He said that there is a problem . Here he said that only two-dimensional arrays are accepted , not accepted. It is not wrong to enter one-dimensional. I forgot to transform. It only accepts two-dimensional matrices and does not accept one-dimensional ones. So here I add a pair of square brackets and it will become two-dimensional . It can be executed again. Seeing that this is the result after zooming, then we can apply it to the model . Is this how our model is calculated? I just copied it here. What we want to apply here is x real, so here Change it , then let’s display the last one as y real, and you can see the prediction of the final model . It tells us that we can give this person a salary of 6.55*10 , which is 65.5k . If a second person comes for an interview today and he tells you that his work experience is 7.2 years and he has a degree below high school, then the place of work is in city B. If we want to predict how much salary we can give this person I ’ll just add this piece of information and write another one here directly . It’s 7.2 and below high school is 0 for us. After the conversion, it’s 0. City b is 0, 1 OK after the conversion. This is the second person We do the same prediction execution, and you can see the prediction given by the model , that is, you can give this person a salary of 23.4k . This is how to apply the model to the real situation . We have already done the gradient descent in the previous step. In fact, In our example, we can accelerate the gradient descent. We only need to use a small technique called feature scaling in English , which can allow us to achieve the effect of acceleration . Well, let’s take a look first. The current example has four features, namely seniority , education and work places cityA and cityB , and then we use multiple linear regression to predict his monthly salary , so it will be written like this : W1*X1+W2*X2 plus 3 plus 4, and finally add X1234 on the side of b. For our four characteristics , we can find that their distributions are different from the values of these four characteristics. For the first characteristic, its distribution range is between 1 and 10 , and the second characteristic is 0 to 2. The third and fourth features are either 0 or 1, so the range of their distribution is like this. We can clearly see that the distribution range of the feature x1 is larger than the other three features , so we take it Going back to the original formula, it will look like this. Our w1 is multiplied by x1 , so it is multiplied by a relatively large value , and W234 is multiplied by a relatively small value. If so , let our gradient descent be slower , that is, make our gradient descent slower. Why ? Because you can see here that W1 is multiplied by a relatively large value , and W234 is multiplied by a relatively small value . In other words, this W1 only needs to be multiplied A slight change will greatly affect the calculated result here , because it is multiplied by a relatively large value, it will greatly affect the calculated result here , and indirectly affect the calculated cost That is to say, as long as there is a slight change in w1, the calculated cost will also change a lot So if we use W1 and W2 to compare their corresponding costs here, it will probably look like this . Here I use a contour map for comparison. The center point is the place with the lowest cost, and the cost is higher as you go to the outer circle . Then the x-axis is W1 y The axis is W2. We can see that this contour map is long and narrow. Why ? Because W1 here is multiplied by a relatively large value , so as long as there is a slight change in W1, it will greatly affect the cost here. We said that the lowest point is the place of the red dot , and the further you go to the outer circle, the greater the cost. Here you can see that as long as there is a small change in W1 , the change in cost will be great . On the contrary , the change in W2 is not so good. Affecting the value of cost Let's take a look at what will happen if we do gradient descent in this case. Suppose our initial point is here , the initial w1 2 is here, we update the gradient descent parameter , it is very likely to happen like this It can be found that it oscillates here and there. Why? Because we said that as long as there is a slight change in w1, it will have a great impact on the cost. When we are doing gradient descent to update W1, it is very likely that it will happen accidentally. The update is overdone, so if you accidentally update it here , you will end up here , and if you accidentally update it , it will cause such a back and forth oscillation, and this will make us reach the lowest point very quickly. Slow , that is , our gradient descent will become very slow , so how do we solve this problem? It is very simple . The reason for this problem is that the size and range of the features are different. Our first feature has a relatively large range and then The others are relatively small , so this problem is caused. If we want to solve it, we can change the scope of the features to the same . I will change the large here to small. Let’s see that the four features are all in the same range. Afterwards, our contour map will look like this . If we are doing gradient descent now, it will be very smooth , and we will go directly to the lowest point . So , we only need to scale the size of each feature to the same range. There are many ways to speed up our gradient descent to do feature scaling. Here I will introduce a very classic and commonly used one called standardization in English. Its method is to subtract our feature from the average of this feature and then divide it by The standard deviation of this feature, in terms of our seniority feature , is to subtract the average of seniority from each data of seniority and then divide it by the standard deviation of seniority. It doesn’t matter if you don’t know what standard deviation is, because there are many tools. It can help us to do calculations automatically , so if we standardize each feature, it will look like this. You can see that their size ranges will be very close . You may have doubts about the three behind us . Aren’t the features inherently small? Why do we still need to do feature scaling? There’s nothing wrong with them ; Let's implement feature scaling directly . If we want to do feature scaling, we do feature scaling after dividing the test set and training set. We divide the test set and training set here, so we do feature scaling after they are divided. The way we want to do feature scaling here is standardization. If we want to use standardization, we can directly import the prepossessing under it in sklearn and then introduce the standard scaler . We can use it like this because it is a category, so I will create it first. It is called scaler. After the creation, we will let him read the feature data of the training set . Note that he can only see the feature data of the training set here , but not the feature data of the test set , because we only have the data of the test set. It can only be used when doing tests. Here we can write some fit and pass it in x train . After passing it in , it will calculate the average, standard deviation, etc. of the features in it. Well , after it is calculated , we can Let’s convert it. To do the conversion, we can write some transform and pass it in the same way as x train , so that it will do the conversion . Let’s replace the original result with the result of the conversion. Well , let’s display it to see . This is The result after conversion , here I also convert the x test by the way , here we just do the conversion first and don’t use it , so that it will be more convenient when we do the test later, and here we just replace the converted result directly Pay special attention here . We will not calculate its standard deviation or average based on the data of the test set, and then do the conversion. We will directly use the result of the training set fitting , which is calculated by the training set here. The resulting standard deviation and average will be directly used on the test set for conversion , so I will display it directly to have a look. This is the result of the conversion of the test set , and then we will compare it directly . After feature scaling, we will do it again. What is the difference between gradient descent and gradient descent without feature scaling ? The gradient descent we made before is here . This is the result. I will directly copy a copy and execute it again here. All the parameters here are the same . The only difference? It is because our x train has undergone feature scaling , so I will execute it again under the same conditions to see if it is really faster. You can see that its cost will reach 2.5* when it is updated about 1,000 times 10 times, and then the following 2,000 to 3,000 updates , basically the cost has not moved, so we can guess that it has almost reached the lowest point. Let ’s compare the previous results . It has been updated and updated to about 4,000 times. It has only reached the first power of 2.5*10 , so we can clearly see that after feature scaling, we are indeed doing gradient descent, and its descending speed is relatively fast . Next, let me introduce to you that we are very good at classification problems. A model used is called logistic regression. English is logistic regression . Let’s first understand the problem we want to solve today . Suppose we know whether a person has diabetes or not . It may be related to his age, weight, blood sugar and gender, so we will collect it. After collecting some information, each piece of information here represents a person ’s age, weight, blood sugar and gender, and whether he has diabetes. If 1 means he has diabetes, 0 means he has no diabetes . So we just want to base on these 4 Features To predict whether this person has diabetes , there are only two possibilities for the output value of whether he has diabetes, either 1 or 0 , that is, he has diabetes or he does not have diabetes , which is much different from our previous two examples. In our first two examples, when doing simple linear regression and multiple linear regression, The prediction is the monthly salary, so the output of the monthly salary may be infinitely diverse . Unlike here, there may only be two types of output. Either you have diabetes or you don’t . The possibility of this output is limited. We will say that it is a classification problem. It’s a little vaccination on the classification side. These materials are randomly generated by me , so if you have a medical background, don’t take it too seriously, and you may also say , isn’t it true that if your blood sugar exceeds a certain level, you will have diabetes? Let’s treat it like this first. I don’t know , so let’s take a look at it with a graph . This is an example of linear regression before. We want to use seniority to predict salary . In fact, the characteristics can not only be seniority, but also work place, education, etc. Here It is for the convenience of presentation, so I only show the seniority . We said that we can use a line to represent these data . Let’s go back to our example today. We want to use a person’s blood sugar to predict whether he has diabetes . The characteristics of this side can also be Many of them can have the weight, age, gender, etc. just mentioned . Here, I will only draw a feature of blood sugar for the convenience of the graph. We can see that the distribution of the data here is much different from that on the left. Because here Its output value has only two possibilities , either 1 or 0 , that is, with diabetes or without diabetes , so it will look like this . At this time, you still think that we need to use a line to represent these data, which seems wrong, right ? What should we do? In fact, we don’t need to make too many adjustments . Just bend the line a little bit to look like this. Does it seem like it can represent these data? The value of this curved line can only be between 0 and 1 . It is very consistent with our data. Because our data is either 1 or 0 , the question now becomes how do we bend it. It is very simple . We only need to use a function called sigmoid function in Chinese. It can be called The s-type function can achieve the bending effect . Taking our previous linear regression example, our mathematical formula can be written as y=w*x+b or if you have many features, we can write y=W1*X1+W2* Add X2 all the way to see how many features you have. If we want to bend such a linear model , we just need to bring it into the sigmoid function. The sigmoid function looks like this , it is a power of 1/1+e It sounds like a tongue twister, so we just need to take this linear model and add a minus sign to bring it to the power to make a bend. Here you may have a question , what is the e in the English letter? What does it mean? This e is just a constant representing 2.7182818.... Just like our mathematical pi , it is just a constant . The position of the square and adding a minus sign can achieve the bending effect . Before bringing it in, I will first change the y here and change it to z. OK, change it to z. Then after bringing it in Our new predicted value y will become like this . Bring this string into it to see how many features you have. Here is how many times you multiply . This is our logistic regression model . Don’t look at him like this and feel very Scary. In fact, there are many tools that can help us do calculations automatically . Back to the problem we want to solve. Today we want to use age, weight, blood sugar and gender to predict whether we have diabetes. So our characteristics have these 4 characteristics That is to say, we want to find the best combination of W1234 and b , bring it into the Sigmoid function here and bend it , which can best represent our data , and then we will directly implement it. First of all, we are the same First read the data in, the action of reading is the same as before, so I just copied it directly , but the URL of the data here is different, so I changed it , and the URL is the same, you can find it in the course file OK , let's execute it, and we can see that this is our data. There are 0-399 in total , so there are 400 data . After reading in, we first process the data. There is text here . We need to convert the text into Here I convert boys to 0 and girls to 1 , so let’s get its gender, which is the characteristic of gender , and then I’ll convert it . We can use map to convert it , so boys correspond to 1 and girls correspond When it reaches 0 , let’s display it after converting it. You can see that the conversion is complete . Then we divide the data into training set and test set. The division method is the same as before, so I just use it directly. The copy is OK here, just copy it here, but the x and y here need to be changed. Our characteristic is age and then weight... the part of y is whether there is diabetes or not. After this is done, I will display it to see Look , display the x train and x test to see OK. This is the result of the division . Then we also display the y part to see the y train and y test. OK, there is no problem. Here we also use 0.2 for the test set, which is 2. Well, we have a total of 400 records, so the 2 results will be 80 records for the test set and 320 records for the training set. Then because we have 4 features , their distribution ranges are different . If we want to let him in the future If the gradient descent runs faster, then we need to do feature scaling . The feature scaling part is the same as before, so I will directly use the copied OK feature scaling here and copy it directly . Here, we will also use the scaling results. Display it, and you can see that this is the result after zooming . I also display the x test. OK, there is no problem. After the data processing is completed , we can bring it into the model. First, I set the values of w and b casually. Then here, I first introduce numpy into it. If w, I will make it an array with 4 values in it , because we have 4 features . OK, there are 4 features. W1234, I will let it be 1234 and then set another If b b, I will let it be 1 , then our model is like this, first let w multiply by x , we are now in the training stage, so I multiply by x train to display the result , this is the result after the multiplication , after the multiplication We want to add it, it is the same as before , so we use sum and let it be the part of the axis 1 to add and then execute , well, this is the result of the addition , after the addition, we need to add b to it Then this string will be the y we predicted in the multiple linear regression before , and now we are going to use logistic regression , so I call this z , and then we want to bring this z into the sigmoid function I will first do the sigmoid function, I call it sigmoid, and then it passes in a z calculation method is to divide by 1 and add by 1 A certain power of an English letter e , so you can write np.exp on the side of the English letter e , and how many powers does he want? It is written in the parentheses , and what we want is the -z power , so just put it It returns the denominator here , and the denominator needs to be enclosed in parentheses . Then I will import the numpy and write it here OK to execute. Then we will use it directly . Let’s take this z in and have a look . After conversion, you can see it. After this simoid function , each value in it will be between 0 and 1. So now we have successfully transformed this linear model . This is originally a multiple linear regression. We have bent this linear model. The next thing is the same, we just want to find the best combination of w and b so that it can best represent our data
