How To Deploy ML Models With Google Cloud Run

welcome everyone today i show you how you can deploy your machine learning app to google cloud run for this we build a small web app and after deployment we get one endpoint url that we can call and that executes our model predictions in this example we use flask and tensorflow but the approach basically is the same for any other web framework and other machine learning framework so let's get started so here i've listed all the steps we need to do and this is actually pretty straightforward so of course we need to write our app then we need to set up our google cloud project we also need the google cloud sdk and we need to write some files the docker file the requirements and docker ignore if we want and then we need to execute two commands to build and deploy our google cloud service so this is all we need to do and let's go over all the steps one by one so let's write our app first and for this i prepared some example code in another directory that i call test so the deep learning code should not be the center of this tutorial i will go over this only very quickly but if you're interested and want to learn more about this then i have free courses here on my channel where you can learn both tensorflow and pytorch and i can link them here so this is the code from one of my tensorflow course videos where we build a simple neural network and do digit classification on the mnist data set so we load the data set we transform the data and bring it in the correct shape and format then we set up our model and compile it and train it and now this is the new part here so now of course we want to save our model and then later call it and load it in our app so we save the model and then here to quickly test this we evaluate this and we also load the model again and evaluate the loaded model to see that this is the very same so let's run this code all right so training is done and we can see that the results are the same so this worked and we also have one new file in here so this is what we need in our app now the next thing we need to prepare is how we can apply this to our own data so here we use the built in mnist data set but now we want to send our own images to our app so for example here i prepared an example image that we can use so we need to load this and so of course we need to load our model and then we need to load a image from a file so we can simply open this and then in order to load this we use the pillow library so we can call image dot open and then open this as bytes io and we also want to convert it so this will convert it to a grayscale format which is what we want and now the next step is also important so we need or we want to apply the same transformations that we used during training so we convert our pillow image to numpy data then we divide it by 255 and normalize it to be between zero and one so this is the very same that we did here in our training step then we need to bring it in the correct shape so we want to have it in this shape one for only one sample then we have x and y and then one for one color or gray scale channel for example if you have an rgb image then this would be three and we also want to have it in the correct shape so we want to have it in 28 by 28 pixels which is what the mnist data set here is so these are the steps to transform and load the image and now to apply the predictions we call the model and this will give us the raw numbers because when we define the model here we didn't apply an activation function at the end so we have to do this now so we use these soft marks to get the actual probabilities and now to get the label we take the highest probability so we call numpy arc max and then print the label so let's run this code for this image with the three and see if this works all right and we see we get a three so this works as well all right so now we need to put this code into our app so for this the first thing we want to do is we want to move or copy this file in our root folder so this is what we need and now let's create a new file and let's call this main dot pi and now let's implement our actual web app okay so i already put the import statements in here and the same code that we just saw so we load the model and then i created two helper functions to transform the image and bring it in the correct format and one helper function to do the predictions and now to implement our app we use flask so we say from flask we import flask and we also want the request and we want chase sonify and then as always with a flask app we create a app so we say app.flask and the convention is to give it the underscore name and then we only want to implement one function so one endpoint so we call this index and we also want to decorate this so we say at app dot route and then we give it the url endpoint so here only a forward slash and then you can also specify the methods that you want so for example here we can allow both a get and a post request so get and post and we use this get request only for a health check but we do the actual prediction when we get a post request with the data we want so here let's first check if request dot method equals equals a post method and then we want to um implement the actual code to do the prediction and otherwise we simply return a okay for the health check so this is if we get a get request and now our request should contain the image as a file so there are several ways how you can send over the data so in this example i used the files so you will see this later when we test out the request so we say file equals requests.files.get and we also call this file again so the first thing we can do is to do some error checking so we can say if not or let's say if file is not none or file dot file name equals equals an empty string then we want to return an error and we want to return this as a json format so we say return chase sonify and then put in a little dictionary here with the key error and as a value we just use another string and say no file and otherwise we can continue and we have a file but now let's also wrap this in a try except block to do some more error catching so let's say try and then accept and we want to catch the error as e and if we have an error then again we return a json data with error and here as a string we put in our exception and of course this is not error this is exception s e so now here we want to load the image so let's go back and have a look at what we did in this example code so let's grab this command and copy and paste it in here so we want to call file.read and this is the same name that we used here so it's already working so we say file.read and then we get this as a bytes object and then we call the image function and now we have it as a pillow image now let's apply the transformations so we say tensor equals and then we use this helper function transform image which which gets our pillow image and then let's do the predictions so let's say prediction equals and then again we have a helper function that we call predict and it gets the tensor and then we have the prediction and again we need to put this into json format so let's create a small dictionary with the key let's call this prediction and then here we want to put in the prediction but in this example we need to be careful so this prediction that we get back from this file is a numpy data type so this is in numpy in 64 and this doesn't work so we need to convert this to a normal integer in order to build a json object out of this so now we have this and then we simply again return jsonify our dictionary data and this is basically all that we need and now in order to test this we can say if name equals equals underscore main and then we can as always with flask we can say app.run and here we can also say debug equals true and run this and we can also leave this code in here because later when we deploy this then this will not execute this file but instead we use g unicorn to start the server so let's run our app alright so our app is running at our local host port 5000 so again i prepared some a code to test this so there are different ways to test this of course you can do it with curl or you can call this from your javascript front-end application so whatever you have so in this example i also do a small python code and we use the requests module for this so we want to send a post requests and we want to send it to this endpoint where our app is running and as i said there are different ways how you can send the files so or the image so you can send this as a data parameter here but since in our app here we accessed the files and then the file key we want to do the same thing here so we use the files parameter and then the file key and then here we need to open our image in read binary mode so this is what we want and then we print the response in json data so this should be a 8 in this example so let's quickly have a look at this image so let's try out this code and see if this works and we get an error no files so let's go back if file is of course we need to say if our file is none so let's save this and since we are in debug mode this should already reload the server so let's run the code again and now we get the prediction eight so this works so let's change this to three and run this code and see if this works so yeah our app is running so now we need to deploy it so now we are done with step one so now we need to set up our google cloud project and all of these steps are actually pretty fast so let's do this so the first thing you need to do is you of course need a google cloud account and then log in and then create a new project so here i you can click on the this and new project and then here i already did one so i called this deploy tutorial then you need to activate two apis so you need to activate the cloud run api and also the cloud build api so this one here so now we are done with step two so now we need the google cloud sdk and for this i can send you over to this link so i can put this in the description so we need to install this so for this you want to select your platform and download the correct package so in my case i downloaded it for mac os and after you downloaded it you want to extract this and then run the install command and after this you want to run the init command and this will ask you to or this will open up a browser and ask you to put in your password and log in and also select a project so i already did this and i selected this deploy tutorial so we have step three and now we need to create the docker file and the requirements txt and we also want to have a docker ignore so in the root folder let's create a new file and we call this simply docker file then let's also right away create a requirements.txt and we also want to create a dot and then dr ignore so this is similar like a git ignore and we'll ignore the files that we put in here okay so let's start by writing the docker files if you don't know how docker works then i will link another tutorial here on my channel where i explain all the steps how you can get started with docker so in order to use a docker file i again have a link here from the official documentation and we use the very same file as in this installation guide so let us copy and paste this over here so we use a simple python image and then we copy everything into our container except the files that we will put in the docker ignore in a second then we want to run the installation and this is what we want to change so in here we want to install this from our requirements txt files so we say minus r requirements dot txt and then we leave this execution command so this will use a g unicorn server so this is a production server that is better than the built in flask development server and we leave the default arguments and you also want to be careful so this is how your file is called so in our case we used main.pi and this is how your app is called so the app that we built that we created here so this is already correct and yeah so this is all that we need for the docker file and now of course we need to fill the requirements txt so we used flask we need to unicorn then we want tens of flow and we want numpy and we also want pillow so these five packages should be all that we need and now let's also fill the docker ignore so in here we put the files that we don't want to copy so we don't need the docker file we also don't need the readme that i created then some python cache related files then this is from our ide from pycharm and we also want to ignore the whole test folder so now we are done with step four now we only need to build and deploy this and for this we need these two commands and we need the project id so if we go back to our console and click on here and here we find the id so let's copy and paste this so we want to put this in here and also in here and after the slash so this is how our function is called so here we call this index so we want to use the same name here so slash index and slash index so let's first grab this command and copy it and now in our terminal you want to be in the same folder where you have the docker file otherwise it won't work so paste the first command in here oh sorry now i've overwritten my copy statement so paste this in here and execute this now this will containerize your application and put it on google cloud and also execute this pip install command that we put in our docker file all right so building was a success so let's grab the second command to deploy the actual service so copy and paste this one and this will ask you for the server service name so for example i called the function index but we can give it another um function or another name so let's call this get prediction so it doesn't allow the underscore so let's rerun this and let's call this gets prediction without the underscore and then it will ask us for the server or for the region so in this case i use the 10 for europe central so let's put in 10 and now that it can work for anyone we also want to allow unauthenticated invocations so i say yes here but you might change this if you actually deploy this and make this public all right deployment was successful so now we get this service url so we want to copy this and test this in a second but before we do let's go back to our console and then here we can go to cloud run and here we should see our new service get prediction and this is green so everything should work fine so now in our test script now we want to use the service url so let's copy and paste this in here and now let's run this code and now see if this works and prediction works so our cloud service is running so let's test this for the other file that we have and run this and this is running as well so yeah pretty cool now we have a machine learning model deployed to the cloud and yeah i hope you really enjoyed this tutorial and if so then please hit the like button and consider subscribing to the channel and then i hope to see you in the next video bye you

Channel: Patrick Loeber

Views: 45,603

Keywords: Python, TensorFlow, Cloud Run, Google Cloud Run, Google Cloud, Machine Learning, ML Deployment, PyTorch, Flask, Flask ML, Flask Machine Learning, TensorFlow Deployment, TensorFlow Deploy, ML Deploy, Python Cloud Run

Id: vieoHqt7pxo

Length: 20min 10sec (1210 seconds)

Published: Sun May 30 2021