Create & Deploy A Deep Learning App - PyTorch Model Deployment With Flask & Heroku

hey guys welcome to a new pie torch tutorial today i show you how we can deploy our pie charge model with flask and heroku we create a simple flask app with a rest api that returns the result as json data and then we will deploy to heroku and as an example our pi touch app should do digit classification so at the end i show you how i draw my own digits and then predict it with our life running app so let's start so first of all let's create a new directory so let's call this pytorch flask to tutorial then let's go into this directory and then we want to create a new virtual environment so we say python 3 minus m v and vn and this will create a new virtual environment and by the way this command might be slightly different on windows so for this you can have a look at this installation guide from the official flash site i will put the link in the description here you find all these instructions so now we have our virtual environment and now we want to activate it by saying dot v and fin slash activate and again this is different on windows and now we install this things we need so we say pip install flask and then we install the packages for pytorch so we say pip install torch and torch vision now this will take a couple of seconds all right so now we have this and now i create another directory and call this app and then i open up my editor so here i'm using visual studio code and let me make this larger for you and now we see we have two directories so this is the one with our virtual environment and this is the new empty app directory now inside the app directory let's create a file main dot pi and here we will write our flask app so we say from flask import flask then we create a new or by the way yes i say yes i want to use the virtual end in here um so this one and now i create an app by saying app equals flask and then double underscore name this is just the convention how we do it and then we only want to create one function so one api endpoint and we call this predict and here we do our prediction later and for now we also have to decorate it by saying app dot route and then the path to the url so here we say slash predict and we want to say the methods the allowed methods is only a post method so here we create a list and say post now this is the api route that we need and here what we want to do is we want to do four things so we want to load an image we want to or or get the image from our requests then we want to transform the image to a tensor then we want to make our prediction and then we want to return json data so the result as a json data so for this we also need to import requests from flask and we also import jsonify so for now let's do a simple return statement and return a dummy json data so here we put in a dictionary so as a key let's say results and then as a value let's use one so now we have this and now let's open the terminal inside here and let's go into the app directory and now in order to start flask we have to set two environment variables so i say export flask app equals the file name main dot pi and export flask f equals development this will give us hot reloading during the development process and by the way again on windows this is not export so here you have to say set and then this part and now we can say flask run and this will start our flask app on localhost port 5000 so now we have a live running app with this endpoint so now let's test this let's create a new directory and let's call this test and inside here let's create a file test dot pi and for this um we use the requests module so we say import requests and i think i have to install this too so let's quit the server for a second by using ctrl c and then say pip install requests so now we have this so now let's restart our flask app again and now here we want to send one post request to our um to this url so we say requests.post and then the url path so we say http colon slash slash and then we can say local host port 5000 so this is the same as we are seeing here and then we also have to say slash predict like we specified here slash predict and then what we want to um send so for now we don't want to send anything um so let's say prints the response dot text this will print the actual json data that we are returning here so now if i go to the other terminal and let's go inside the test folder and run python test dot pi so i have a typo here localhost 5000 let's do it again let me clear this and now let's run python test and then we see this is working so we get the dictionary that we specified here so our app is already working and now let's do these four steps and um for this we create another file so in the same app directory let's create a file and let's call this torch utils dot pi and here um we do our imports from the torch library so we say import torch we import torch.nnsnn [Music] then we also say import torch vision dot transforms as trans forms and we say from pill import image so we will need this to convert our image to the tensor and now before we go on let's have a look at the code that we are going to use so for this i have this in my pytorch repository from my pytorch beginner course and i will also put this link in the description so here we are going to use the tutorial number 13 our basic feed forward neural net to do digit classification and so let me open up this in a second editor so here this is the exact same code that you can find and download on github and so now basically we want to run this file and then save the model so what we have to do down here at the very end we want to say torch. and then we want to save the model dot state dict and then the file name so here we say let's call this mnist underscore ffn.pth and then we want to do one more thing so here um for our data set right now i only have one transform so transforms dot to tenzoa and now let's use another transform here so let's say transform equals transforms dot com pose because we use more than one and here as a list we first of all do our to tensor transform and then we also want to normalize it so i say transforms dot normalize and then here um i put in the mean and the standard deviation and this has to go in as a tuple so the mean is 0.1307 and then a comma because this should be a tuple and then as a standard deviation let's copy and paste this and here we use 0.3081 so these are just the global means and the standard deviation from the m this data set so this is given here so just put in the same numbers here and now we use this transform here and so you don't have to do this but i just want to demonstrate that later in our interference we want to apply the same two transforms so now we use this and now we want to run the code and then save it at the end so here i'm using a conda environment so the same as i used in the uh in the tutorial in the beginner course tutorial so i say conda activate sorry conda activate pi torch and then let's say python and this file is number 13 feedforward.pi and then run it and this will take a couple of seconds all right so now it's done and you could see that our during our training the loss decreased and at the end we have this accuracy so it's pretty good and also in the same folder appeared this file here so we want to copy this and then paste it into this directory here so now um we can close this and now we can start implementing the pipeline here so here we want to do three things so we want to load our model we want to have a function to transform an image to a tensor with the same transformations that we've just seen and then we want to have a function to predict to do the prediction so now let's go back to the um original code and from there we copy a few things so we copy this part we want to have the same model class so let's paste it up here and we need these parameters and we remove the two device calls so for now or in this tutorial we are only using the cpu so now we need the input size the hidden size and the number of classes and we also get this from here so we have these hyper parameters so let's copy them over here then we create our model and then we want to load the state stick so by the way you can also um find this in my github repo how we save and load a model so this is tutorial number 17 and here we can grab this part so this is the recommended way by the way when we only save the state stick so then we create our model from the class then we load the state dict and we also have to set it to evaluation mode so this is very important and we also need the path so the path equals and then this is the file name here so it's in the same directory so we say mnist underscore ffn.pth and by the way if you trained on the gpu then you must be careful so but you also find how you load it from a gpu to the cpu then so you have to specify the map location and this is our cpu device so now we have this so now we have our model and the next thing we want to do is define a function to transform transform the image and this will get image as raw bytes as an input and then here we want to create a transform by saying transform equals and then we grabbed the same thing as we are having here so we want to have the same transformation steps as in our training but we also want to or need to have some more here so for now this could also be a rgb image as an input but we only want to have it as a gray scale like um here when we are loading this mnist data set and this is in grayscale format and it's also in exactly this size so each image has 28 by 28 pixels so we have to use these same values too so we apply two more transforms so we say transforms dot and then we can use grayscale gray scale and then here we can say the num of output channels equals one because we only have one to have one output channel and we also want to re resize it so we say transforms dot resize and then as a tuple the size 28 by 28 so now we have this so let me format this a little bit nicer so now this is our transform and then we want to apply it so first of all we want to create an image a pillow image from the image bytes and we can say image dot open and then io dot bytes io and then our image bytes and for this we need to import sorry import io so to handle input out output streams and then we want to apply our transform and then can return it so we say return transform and then our image and we also say unsqueey unsqueeze zero so this will return a new tensor and inserts a dimension of one at this position because during training we have the number of batches on one axis and now we only right now we only have one image so we have to add this axis and only have one sample in our batch so now we have this and now one more method is missing so now we have to write a function to do the predictions so we say define get prediction and this will get a image tensor as input and now for this we also can have a look at the original code and down here we do the evaluation and here we use the same code so let's copy and paste this and here so we resize our image to be in this shape then we can remove this part because we don't use gpu support then we perform the forward pass and then we get the prediction by calling torch.max and then we can return the predicted class or predicted index and now this is all we need in this file so now this is all the code for our inference and now we can go back to our main file and then import this so we say from torch utils we import um let's call it was called transform image and get prediction so now in our predict method we want to implement this pipeline and here we want to be very careful because we don't want to have a error here so then our server crashes so we do some error checking so first of all we check if our request dot method equals equals post and only then we do the code then we get the file so we later we pass a file to this post request and then we get this file by saying file equals request dot files dot get and then we use the key file later so now first we check if file is not or we say if file is none or file dot file name equals equals an empty string then we return uh jsonify and here we return a dictionary where we put in a arrow as key and as a value we say no file so this is the first thing we check then we also check if this is an allowed extension so for example we only support p and g and jpeg files so we say if not allowed file and then here we passed the file.file name then we also return a error so let's say return chase sonify and then let's say format not support teds and for this we create a little helper function so let's say define allowed file and this will get a file name and then our file name could be for example some name and then dots p and g our ending and now we want to define some allowed extensions so let's say allowed extensions equals and this is a set with only unique values so we support png jpeg and jpeg with an e so and now here we return dot in file name so our file name should be this format and also now we want to check the ending so we want to split our files so we say file name dot r split so from the right side and we want to split at the dots and only have maximum one split and then we take the second part so this part so this is index one then we convert it to lower and then we check if this is in our allowed extensions so now this will return true if our file name looks for example like this so now down here we can go on and so now what we want to do is we want to load the image bytes then do the transformation and our prediction and this part and for now i wrap this in a try except block so this is not very good error handling but for now this should be okay so if we have a exception in this part then we say error and let's say error during [Music] error during prediction and then here we say our image bytes equals file dot read then we say our tensor equals transform image and put in the image bytes then we get the prediction prediction equals get prediction with our tensor and then we want to return the result as a json file so we do let's say our data equals a dictionary with the prediction key and here we use prediction prediction dot item because we cannot put a tensor in this but our tensor only has one element so we get this by saying prediction.item and in case of the mnist data set so this for example this has index zero and then this is equivalent to the number zero that we have drawn so basically we also want to return the actual class name but in this case this is the same so here we just do for a demonstration we return the same thing as a string so string prediction dot item so here for example if we would use the cipher 10 data set then we would have to do a a mapping so for example index zero is a car or a ship or whatever so this is what you want to put in here so now we have the data and now we only have to return this as a json object again and then we should be done so i think this is all we need so now um let's see so our server should have reloaded automatically because we set the environment to development so now let's go to our test file and here we want to put in the files to our post request so we say files equals and remember this is a dictionary with the key file and now we need a file so for this let's go to google and let's search for mnist image and let's go to images and for example where is it let's grab this image so let's click on save image as and then i will save it as 7 and on my desktop and then i will let me quickly grab this and copy it into this folder so now we see we have this image in the folder so now we want to open this here so here we say open and then the file name 7 dot png and we want to open it in read binary mode so now we have this so now again let's go to our test file and now we should get a result with seven if everything is working correctly so now let's do python test dot pi and cross fingers and of course it's not working and if we go and have a look at this terminal then we get a syntax error invalid syntax so let's go back and let's have a look so here i missed a comma i guess so let's save this and now our server crashed so we have to say flask run again and that's exactly why we want to be very careful in this method and do some error handling here so now let's test this again so let's clear this and now let's say oh sorry this was the wrong one so here i want to say python test dot pi and then we get error during prediction so it's still not working so for now um let's comment the try block out and let's see which arrow we are having so now you could see why this is not the best error handling because we don't see what went wrong so again our server restarted so let's say python tester pi again and then name transforms is not defined so again i'm having a typo here transform forms sorry about that so now so now let's save this and let's run it again tensor has no op attribute on squeeze so again a typo on squeeze and again python test.pi images image tensor and now python tester pi and now it's finally working so now we see that our flask gap is running and our pi touch model is doing the correct thing so now i can add our try accept block again so let's in then this part and do our accept block again so now we can remove this and now let's test this with our own images so i'm using paintbrush here and then i specify a new image with width and height is 100 pixels and now for example we want to be careful we want to have the background in black so i fill the background with black and now i'm using a white pencil and then for example let's do a three here and let's save this as three dot p and g and then let me copy this inside the folder here and now let's use this one so let's use three dots png and let's go back and let's clear this and run it again and we get the prediction is three so yeah this is pretty nice so now we have this so now the only thing left to do is deploy it so for this we have to do a couple more things so the first one is to use an actual web server that is suitable for production and for this we use g unicorn so i say pip install g unicorn then in our base folder we create a new file and let's call this wsgi dot pi and move this to the base folder and here we say from app dot main import app and now we want to create a heroku app from this so for this of course you need to have a heroku account and the heroku command line face installed so if you haven't then go to the official website heroku.com and there you find the instructions i already did this so now i have to log in so i say heroku login minus i to do it in the terminal then i have to put in my credentials and my password and now it says i'm logged in so i can clear this again and now we want to create a new app by saying heroku create and then the name so let's call this flask pie torch tutorial this will create a new heroku app and in order to test this we can go to the base directory again and we need to do one more thing we need to update our path names the same way as we are doing it here so now app is our base package so this folder here so in the main file we have to say from app dot torch utils and inside this we have to modify our path so this is now app dot mnist ffn.pth sorry here it's app slash this one and now we can test this by saying heroku local so this will start how we first need to have the prop file so let's create a new proc file sorry sorry i want this in the top directory again so let's say let's call this proc file this is what we need for heroku and here we need to specify we want to have one web app and we want to use g unicorn with our new we at wsgi file as starting point and our app so this is the syntax now again we can test this by saying heroku local and now this is working so this is starting our t unicorn server on the same port so now not our flask app is running but our heroku app is running locally and again so the server didn't change so we can again test this by saying python test.pi so this is still working with our new correct path names and now we can stop this and now i also want to create a runtime dot txt and inside this we specify our python version so now this is python 3.8.3 then we also let's clear this we also need our requirements and since we use pip we can say pip freeze and then greater and then require er men's requirements.txt this will write all of our requirements that we installed inside this package inside this txt file but we have to modify this because right now we are using the normal torch and torch vision and this takes a lot of space and this is too large for heroku so we have to use the cpu only version here and for this we can go to the official pytorch website and then have a look at the installation guide so here we use pip as our package manager and we don't want cuda support and so on mac i actually use this command but on heroku we have linux so we need this command so we need to have plus cpu appended to our torch and touch vision so we say plus cpu here and plus cpu here and we also need this command and we can write it at the top of our requirements file so here so now this will install only the cpu version on heroku and now in order to push this directory to heroku we need a get repo so we say git init then i want to add a git ignore so i create a new file git ignore and for this let's go or let's google let's say python git ignore and i think this will lead us to github and then here we can copy all of this and put it in our get ignore and at the top i also put in the test folder so i want to ignore this folder too so let me quickly check if i have everything that i want uh in the repo now so this looks good so now we have all of this and now we want to add this as a remote repo for heroku so we say heroku get colon remote minus a and then our app name so i think we called it flask minus pi torch minus tutorial this will add this as a remote repo and now we add all of our files so we say get at dot and then git commit minus m and let's say initial commit and and now we can push it so we say get push heroku master this will push all to heroku and then start building and deploying the app so now this takes a couple of seconds all right so this was successful and now it said deployed it to this url so here at this path we now have our live running app so now let's go back to the test.pi and instead of the localhost we use this url and then slash predict and then let's go back to our test file and let's run python test again and now we get the result so yeah so this is working so now let's do one more test so again let's use my paint program and as a background we have black and now let's draw for example a 8 like this and let's save it as 8 dot p n g and then let's move it into this folder and here let's use this eight dot p and g and again save it and clear this and run it again and we see we get the prediction is eight so this is working and we have a live running app deployed to heroku and yeah i hope you enjoyed this tutorial if you like this please subscribe to the channel and leave me a like and see you next time bye

Channel: Patrick Loeber

Views: 46,989

Keywords: Python, PyTorch, Deep Learning, DL, Flask, Heroku, Model deployment, PyTorch Deploy, Machine Learning, ML Web App

Id: bA7-DEtYCNM

Length: 41min 52sec (2512 seconds)

Published: Wed Aug 05 2020