PyTorch Tutorial 08 - Logistic Regression

hi everybody welcome back to a new PI torch tutorial this time we implement logistic regression if you've watched the previous tutorials then this should be very easy now once again we implement our typical PI touch pipeline with those three steps so first we set up our model we define the input and output size and the forward pass then we create the loss and the optimizer functions and then we do the actual training loop with the forward pass the backward pass and the weight updates the code here should be very similar to the code in the last tutorial where we implemented linear regression we only have to make slight adjustments for the model and the loss function so we add one more layer to our model and we select a different loss function from pi towards built-in functions so let's start first of all that it let's import some things that we need so we import torch of course and we import torch dot n n s and n so the neural network module then we import numpy s and P to make some data transformations then from SK learn we import datasets to load a binary classification dataset then from SK learn dot pre-processing we want to import standard skaila because we want to scale our features and then from SK learn dot model selection we import train test split because we want to have a separation of training and testing data and now let's do our three steps so first we want to set up the model then we want to set up the loss and the optimizer and then in the third step we do the actual training loop and as a step zero we want to prepare the data so let's do this so let's load the pressed concert data set from SK learn so we can say BC equals data sets dot load breast cancer this is a binary classification problem where we can predict cancer based on the input features so let's say x and y equals BC dot data and BC dot target and then we want to say well first of all it's gotten get the number of samples and the number of features by saying this is X dot shape so let's print this first so print the number of samples and the number of features to see how our dataset looks like and we see we have 569 samples and 30 different features so a lot of features here and now let's continue and let's split our data when we say X strain and X test and next test and y train and Y test equals here we can use the Train test but function where we put in x and y and we want to be the test size to be 20% so this is point two and let's also give this a random state equals let's say 1 2 3 4 and there should be a small s and now let's convert or first of all now we want to scale our features scale them here we set up a standard scale ax SC equals standards gala which will make our features to have zero mean and unit variance this is always recommended to do and when we want to deal with a logistic regression so now we scale our data so we say X train equals s C dot fit transform and then as an input we put in X train and then we want to do the same thing with our test data so we say X test equals SC dot here we only transform it and here we put in X test now we scaled our data now we want to convert it to torch tens us so let's say X train equals torch dot and then here we can use the function from numpy and then we put in X train and cost this to a float32 data type so we say X train dot s type numpy dot float32 because right now this is of type double and then we would run into some errors later so let's cast this and convert this to eight tens oh and now let's do this with all the other array so let's say X test equals this and our y train and also our Y test tens on my test and now as a last thing to prepare our data is to reshape our Y tends us so Y train equals y train dot view this is a built-in function from PI torch that will reshape our tenza with the given size so it gets the size why train that shape zero and one so right now our Y has only one row and we want to make it a column vector so we want to put each value in one row with only one column so this will do exactly this and also for our Y test so Y test equals this Y test and now we are think we are done with our data preparing so now let's set up our model and here our model is a linear combination of weights and a bias and then in the logistic regression case we apply a sigmoid function at the end so let's do this and for this we want to write our own class so let's call this model or we can also call this logistic regression just tick regression and this must be derived from n + dot module and then this will get a in it which has self and then it gets the number of input features and here first we call the super init so let's say super logistic regression and self dot in it and then here we define our layer so we only have one layer self dot linear equals and here we can use the built-in layer n n dot linear and this gets the input size so and input features and the output size is just one so we only want to have one value one class label at the end and then we also have to implement the forward pass here which has self and the data and our forward pass is first we apply the linear layer and then the sigmoid function so here we say Y predicted equals Torche dot sigmoid so this is also a built-in function that we can use and here we apply our self that linear layer so linear layer with our data X and then we return our y predicted so this is our model and now let's create this so model equals logistic regression of size and here we put in the number of features that we have so now our layer is of size thirty by one and no sorry thirty input features and one output feature and now we have our model and now we can continue with the loss and the optimizer so for a loss the loss function now is different than in the linear regression case so here we say criterion equals and n dot BCE loss so the binary cross-entropy loss here and our optimizer is the same so this can be this is Torche dot optim dot s GD for a stochastic gradient descent and this gets some parameters that we want to optimize so here we just say model dot parameters and it also needs a learning rate so let's say our learning rate equals point zero one and then here we say L R equals learning rate so now this is step two and now step three so let's define some number of epochs equals let's say 100 iterations and now we do our training loop so now we do for epoch in range num a pox and then first we do the forward pass forward pass and the lost calculation then we do the backward pass and then we do the updates so let's say Y predict it equals here we call our model and as theta it gets extreme and then we say loss equals criterion and this will get a the y predicted and the actual y training so the training samples or training labels and now we do the backward pass and calculate the gradients and again we simply have to call loss that dots backward and PI torch will do all the calculations for us and now we update our weights so here we simply have to say optimizer dot step and again PI torch will do all the update calculations for us and then we don't or we must not forget to empty our gradients again so one to zero the gradients because the backward function here will always add up all the radiance into the dot grat attribute so let's empty them again before the next iteration and we simply must say optimizer dot 0 grat and then let's also print some information if a POC plus 1 modulo 10 equals equals 0 so every tenth step we want to print some information let's use an F string here let's say epoch and here we can use epoch plus 1 and then we also want to see the loss so the loss equals loss dot item and that's format this to say to only print for decimal values and yeah so now we are done this is our logistic regression implementation and now let's evaluate our model so the evaluation should not be part of our computational graph where we want to track the history so we want to say with torch dot no grat and then do our evaluation here so here I want to get the accuracy so let's get all the predicted classes from our test samples so let's say this is model and here we put in X test and then let's convert this to class label so 0 or 1 so remember the sigmoid function here will return a value between 0 and 1 and if this is larger than 0.5 we say this is class 1 and otherwise its class 0 so let's say Y predicted classes equals y pretty that dot round so here we can use a built-in function again and this will do exactly this and yeah so if we do don't use this statement then this would be part of the computational graph and it would track the gradient calculations for us so here we don't want this we don't need this because we are done so that's why we use this with statement here and now let's calculate the accuracy by saying AK equals y predicted classes and here we can call the equal function equals y test and then the sum so we want to sum them up for every prediction that is correct it will add plus one and then we divide this by the number of samples of test samples so Y tests dot shape 0 this will return the number of test samples and then let's print our accuracy print accuracy equals ax dot point for F also only for decimal values and now let's run this and hope that everything is correct and Standards gala has no attribute transfrom so here I have a typo transform now let's run this again trance form one more try and now we are done and we have a accuracy of point 89 so it's okay it's good but it's not perfect so you might want to play around with for example the number of iterations and where do we have it the number of epochs or the learning rate for example or also maybe try out a different optimizer here but basically that's how we implement logistic regression I hope you liked it if you liked it please subscribe to the channel and see you next time bye

Channel: Patrick Loeber

Views: 62,221

Keywords: Python, Machine Learning, ML, PyTorch, Deep Learning, DL, Python DL Tutorial, PyTorch Tutorial, PyTorch Course, Neural Net, Logistic Regression, sigmoid

Id: OGpQxIkR4ao

Length: 18min 22sec (1102 seconds)

Published: Mon Dec 30 2019