Build an AI app with FastAPI and Docker - Coding Tutorial with Tips

hi everyone and welcome back on my channel in today's video we built a machine learning app with a model from hugging face and then we built a fast API app around this and dockerize the whole thing and for this I have a very special guest here with me today Sebastian Ramirez Sebastian is the creator of fast API typer and other very cool open source tools so I'm super excited that you're here today Sebastian welcome thank you thank you very much for having me uh very cool let's see how it works let's see how it goes yeah let's see so Sebastian will give me some help along the way and by the way the folks from Docker were kind enough to provide some swag so if you're watching this video live then there will be a giveaway during the video and for this you have to monitor the chat and the only thing I already prepared is the model code so let me share my screen so the model is a vision and language Transformer that can process both images and text and here for example you see you can send an image to it and then ask a question for example here what's the animal doing and to get this running you need to install the Transformers library and pytorch and then here you can grab this um example code so I already did this and pasted it here in Visual Studio code and then as I said you need to install the Transformers library and pytorch um so I also already have a virtual environment with this um here and install this already so now we can for example run the code so let me quickly press this by running Python and then I call the file model starter.pi and as you can see we get a predicted answer with the response and now before we turn this into a fast API app um I want to explore this in an interactive session so this is a very cool tip I got from Sebastian so maybe Sebastian you can talk a little bit about more about this interactive session so yes of course I I think this is one of the coolest features of this editor under you know modern editors uh is that uh I guess some of you have probably worked with Jupiter notebooks and there's a way to have the equivalent of a Jupiter notebook inside of Visual Studio code and that this is what is called an interactive session and what it actually does is that the editor communicates with the Jupiter kernel with the internal parts of Jupiter so that it can show like all the interactivity and all the interactive window and everything inside of the same editor and it's actually quite simple the way you you use it is that you go to the line that you want to select for example like I don't know import requests or from Transformers import this then you just select that line and then you uh trigger like the multi-command thing that in my case command shift p or in a in Windows it will be Ctrl shift p and that shows you like all the commands that are available and then you can say search and like start texting they start typing run an interactive window and that will show you the the available commands that are for that and then you can see that there's Jupiter run selection line in interactive window and if you select that line and then that if you just hit enter Then in that line that is gonna normally it's gonna ask you hey this needs this additional component from Jupiter called ipy kernel should we install that and then you just click yes and then it will install it if we have it already installed then it's just gonna connect directly and it's gonna start the jupyter kernel locally in that same virtual environment and then it's gonna run that line that we selected the cool thing is that now you have on the left side the final code that you're gonna use that can be you know like the production code and on the right side you have an interactive window where you can explore that same code without having to you know switch back and forth and copy paste from Jupiter kernels or and things like that so now we have the Transformers components in this interactive window and we will be able to you know like just trigger out the completion or like explore that right there the cool thing with this is that you can select lines of code or like even sections even pieces of code and then run those sessions in the interactive window and that allows you to explore a bunch of things you know just interactively and manually maybe let me try this again so now I will select some more files some more lines and now I say command shift p and then run selection in interactive window and enter and now yeah now it will send everything over to the notebook and one of the one of the coolest things is that it can give you it has a lot of the interactivity that you will have with Jupiter notebooks so for example if you have an object that is an image like the variable image here and then you execute that then it's just going to show the actual image there in the in the interactive window so you can explore these things and it can it can help a lot yeah you know interactively exploring what is the code actually doing this is super cool so yeah like I said like Sebastian said now you have access to all the variables and can explore them and yeah I think this is super handy to explore your objects while you write the code um for example let me type encoding and I think now if I yeah I will also get Auto completions so yeah we can we could explore this some more and yeah so very very cool tip I got from Sebastian um so like yeah let's also for example let's select the rest of the codes and I think you said you have a short shortcut right yeah I have a shirt to configure that is shift enter but I think by default is not configured uh but like that way whenever I'm I'm writing code I think you could actually just like you know like it actually says that it's there shift enter I don't know if it works did it work seems like it worked yeah yeah maybe that's the default now because at some point it was not the default and I had to configure it to be the default but that means that when you have the shortcut enabled you can just shift enter select a piece of code and then shift enter the same shift enter that you will do if you were in Jupiter notebooks and then that just runs that selection in the side it's very efficient is the the iteration is very quick it's very convenient yeah I think so too cool yeah so now for example you have the model outputs and often it's not very clear how you get to the actual item you want from this so then here for example you could explore the outputs and then see that you need the logic Logics fields and yeah so but yeah this is just one tip we also wanted to show you in this video but now moving on so now I think um the next step and when we want to um put this into a fast API app is I think we should um we should turn this into a function right yep yeah it will make sense yeah it will probably it will probably make sense to have the model loaded right there at the top of the of the file of the not loading it inside of a function because otherwise we could end up loading the model for each request so I think like most of it is just like that but then yeah like a function that does like the rest of the logic of getting the inference from the model very good point all right so let me create a new function maybe down here and let's call this for example model pipeline and now as an input I think we want to pass in the text as an input and what you use are type annotations here right away I I will use type annotations just because if I use type annotations there I get up to completion inside of the function yeah data will be able to know that hey text is a strange so it will be able to give me like this or two images so I end up just using type annotations pretty much everywhere just to get that that feature Okay Okay cool so yeah let's send in the text as a string and then also the image as an input and I think this can be a pillow image and then we basically need to do the rest of the code in here um and then in the end we need to yeah let's just return this part I guess this is the actual final label yeah so now we have the model function so I think now we should we can turn this into a fast API app right yeah maybe in line 11 that there's an image that is being open maybe uh maybe we can comment that out so it's not being yeah an image is not being so we actually don't need all of this anymore also yeah but fully correct we also don't need requests now and we just need to make sure that we we get the image somewhere else so we want to do this with a file upload later yeah um definitely all right so now let me create a new file um do you have by the way like any best practices how you set this up would you do this would you put this into a new directory or just I will normally just just create a new file right there and then move it to another directory whenever it actually grows I think you know like I think it's even more important than to have a very predefined structure that is like applies to everything I think it's more important to be able to refactor quickly according to whatever is the the things that need to be done so yeah that's a very good point okay okay yeah so let's keep it simple in the beginning um so main.pi and now for example we could um import this right away so from model starter oh let's rename this um that's um let's rename this to maybe just model Pi or yeah let's just do it model pi and then we can say from model we want to import the model pipeline right yeah and now for fast API so I think I already installed the requirements but yeah just for completion I think we need to do pip install fast API right yep that's correct and also probably also uvicon and we'll probably be it's probably better if you put people inside of quotes install ubicorn and then uh like ubicorn inside of quotes and then you can standard yes that's so it has like all the all the all the standard features including the thing that gives it a massive performance nice okay so now we have this installed so now we say from fast API import fast API and then um yeah yeah enough like you you actually know like everything I don't know everybody's and she'll just like say just to respect it but like yeah yeah so actually yeah that is an instance of the fast API class um so yeah so what I often um always do is just go to the to your home page because I think the documentation here is excellent and then just simply followed the uh getting starting getting started um part so here yeah we again have the PIP install fast API pip install unicorn standard and then I I just grab the getting started code basically so let's do the same here um yeah all right so now we have a Home Route that simply returns hello world and then there is another um route so maybe you can quickly explain what this will do yep so what we are doing we are creating an instance of the class first API and then we use the add symbol at app.get so this add whatever for those that might not know this is called a decorator and what it says what it does is that it tells fast API hey do your thing with the thing that you have below so whatever is it that you do with this do it with the thing that you have below so we are telling here fast API in the first line in the line 10 we are telling fast API Hey whenever someone sends a request an HTTP request that is using the operation get or the method to get like an HTTP get and it goes to the root route to just the slash then this is the function that I want you to use to handle that request the the function that is called read root and then this function is just returning a dictionary then first API is going to make sure that it converts that dictionary to a Json payload and it's just gonna return that then in line 15 what we are doing is that we are saying hey fast API whenever someone whenever someone sends an HTTP get request that goes to this path and the path is slash items slash and then we have you can see these curly braces and then inside of the color braces we have Item ID so Item ID is going to be a path parameter so this is going to be variable this is going to change whenever someone sends a request if someone goes to slash items slash Five then this is a 5 is going to be the path parameter and then what fast API is going to do is that it's gonna extract that five or seven or ten you know like whatever is the item id it's gonna extract it from the path in the URL and then it's gonna put it in the parameter inside of the function read item in the in the parameter with the same name in the parameter item ID and now because we have the type annotation saying like hey this item id should be an integer first API is going to make sure that what we receive is an integer so it's gonna do data conversion it's going to extract that piece of you know it's just a long string a URL is just a string but fast API is going to extract that string containing the number it's going to convert it to a natural integer so that our code inside of this function is going to receive an actual integer and if what we send in the path is not an integer then the the client is going to receive a response saying like hey this data is invalid because this thing is not an integer so we get a data validation and data serialization just from these type annotations this is uh this is not something that comes with python this is something that comes with fast API with python just by using the type annotations you get out of completion and inline errors and you get like you know like the certainty that the code is correct so that's like totally worth it in my point of view just to have a type annotations just for that but then when you are using fast API or whatever these tools then you get like these extra benefits of like data validation serialization and like also automatic documentation I think we'll probably see the automatic documentation in edit yeah this is this is what we are doing here yeah very cool and one more thing I noticed so you the the getting started code has the normal function definition but I think you can also do it async right so I could also say amazing so the thing is that first underneath fast API is amazing framework so it does like everything async underneath but you don't have to use async and when you don't use async functions then fast API that's also like a way to talk us API hey whatever is the code that I'm running here it could be stuff that is blocking so it might not be compatible with async kind of weight like you know a compatible with using a weight internally so if we run this if we run this inside of an async function it could end up blocking the event Loop and the grading performance but fast API What fast API does is that it does the let's say the smart thing and it runs these functions that are non-assync it runs them on a thread worker on a thread pool so that you can get like the best performance on like the best practices by default without any extra effort when you are doing machine learning in most of the cases you will not want to use async and actually like you know like almost most of the scenarios you don't want to use async and just want to use normal irregular functions and then whenever you have like a path like a like a region of code that is a hot path is something that is used a lot and then you need to squeeze and optimize performance for that specific region then it makes sense to optimize that and to use async inside of that but apart from that most of the cases you end up just like better objects using regular functions and then facility does the smart thing it's just that you have the convenience of like okay if I need it I will be able to just use these other tricks hey but like you don't have to start doing like that all that complexity right away from the from the start yeah right yeah I think that was a very good explanation because I know that many beginners are not sure should they use the normal function or async right away to optimize everything but yeah so let's keep it simple and have the normal um function definition without async and now we could um run this already and test this right so for this we have to run uh uvicorn then it's main colon app dash dash reload let me see if that works main column app is the equivalent as you if you say from Main import app so whatever goes before the column is from blah blah blah and whatever goes after the column is the import so we are telling you because hey go to this file called name and import the object that is called app which is the fast API that's that's what it means yeah and it's working so as you see it's returning hello world at the Home Route so now um we can get rid of this actually so now what we need for our machine learning model is a we want a app dot post right yep with the image I guess yeah so let's call the route slash ask for example and then we Define a ask function and here we also want to send the parameters right so we need the text and this has to be a string and now we also need the image but for the image we need to upload it so I think a fast API has a what's it called upload file yep yeah so the image is a upload file um now we can use this inside so now we need to the first thing we need to do is to turn this into the actual content right and I think for this I need your help a little bit so what what do you have to say to get the actual so the thing is this upload file by default is like compatible with async and all the stuff but we first we want to have a so by default like the default methods that are releasing can serve but internally that actually uses a standard python file so if we do to image dot file then that object that attribute file that is the actual file so then we can interact with this as as before so we can say file dot read and then that will give us the divides all right so now this is the let's call this content and now we need to um also turn this into a pillow image so we can say from pill import image and for that we also need to import IO so now we can say the image equals a image and here we say IO Dot um what's it called bytes IO and here we can put in the content and now we can apply now maybe instead of bites Io if because bytes are yo what what it does is simulate an uh python file maybe instead of simulating the Python file we can pass directly image.file directly to this image IO although you know like maybe it's good just to have it like this just to have like the all the the additional options in case someone needed to send a the code in a different way but yeah I don't know I think it's probably possible to just like pass directly image.file I don't know yeah okay so let's let's try that so you think here I can say um image.file yep without without reading the content there all right so let's try it like this and now we can send this to the um function so let's call this for example results equals the model Pipeline and here you can see we get the suggestions so it needs the text first which has to be a string and then the image um so let's put in the image as well and then um what would you return here so would you simply return this as a dictionary or do you have like any preference we can like we can just start with returning the result directly and just just to the book and see like if it works and then if it works then we can also like try to see if we need to return like you know like additional metadata or something like that we can just start with that okay so let's run this and since I started this with dash dash reload it should already work so now I just have to go to the this page um yeah so now here I have the localhost and now we can I think we can go to slash Docs yeah nice and now here you see all the end points and here you see this is a post endpoint that now needs a text and the image and then we can click on try it out and as a text so let's ask um how many or let's select the image first so I have an example image um if I find it let's select this one with the cats and then we can ask how many cats are in there then we can click on execute and we get a server error so let's see what module objective object is not colorable okay maybe the thing about the image that file didn't work I don't know if it's oh wait you must yeah so I think it's image dot open and maybe the second one will also work so but let's try this one let's just give the one that is known to work all right so you don't even have to you don't even have to reload choose file cats and there and no it's working so many quotes right the application changing quotes so he's returning a string not a number that's funny yeah I think that's because I only returned the last part index to label so it it Returns the label here which is a yeah which is a string so that's fine so yeah that's actually doing the day right oh yeah of course like it's returning like uh that that's another question like is the model is returning some text and the text just happened to contain the number two yeah nice that makes sense all right so now we have the code working now would you would you put this into a dictionary for example or would you just keep it as a string yeah maybe it will make sense to put it in an additionally in case later you want to add additional information uh yeah so maybe it could be like returning uh yeah the conditioner is saying like yeah I think like that that works yeah okay but you know like this is one of those things that depends a lot on the application and the conventions of the team and like it's it's one of those things that is good not to be super strict about them to say like oh like the convention or like best practices or this or that is just like yeah well what is practical what actually works on solves the problem uh correctly that's that's the main thing to do I don't think yeah good point Okay cool so now we have a fast API that calls our model pipe a fast API app that calls our model pipeline so now I think as a last step step we only have the docker Rising part missing right nice yeah I think so so let me stop the server with Ctrl C and there is a a a kind of new Docker command that you can now use which is Docker in it so with one of the latest versions it should be available so let me try to run Docker in it and what this will do it will um generate a Docker file for you and also a Docker compose file and a Docker ignore file so you have some best practices all already in the beginning and it makes it a lot easier to get started so for example here we can now select python um here it already suggests this is suitable for a python server application which is what we have so let's select Python and then you can as a Next Step select the python version so I just want to yeah let's keep the default 3.10 as python version then the port and then the command so now this is something we have to write so I think here we have to write the same command that we used before if I'm not wrong unicorn main colon app dash yeah we could while we developed this we could do testers reload and now we also have to yeah there you go go ahead go ahead and now we also have to specify the port I think dash dash Port 8000 so the same thing yeah I think we also have to specify the host because by default uvicorn will only listen on localhost but localhost inside of a locker container will mean that only whatever is inside the docker container can talk to itself and we need to be able to talk to the docker container from outside the docker container so it has to be Dash host uh cl.0 which means like listen on all uh all the IPS very good point all right okay cool and now let's hit enter again and as you see it now generated our Docker file then a compose file where you can simply run Docker compose up later and a Docker ignore file where for example it ignores a lot of um a lot of folders already so for example here let's also ignore the VF file without the vnf directory without the DOT and now we need to do in this case we need to do a few adjustments so first of all let's go over the docker file very quickly so here it suggests to select a the python 3.10 image and then the slim version in this case do you have like any favorite images you usually I'm using I normally use the default python version without slim a a mainly because it simplifies like all the development and installing the dependencies and all the stuff a later at some point whenever you have like the code running and like everything is working and like you you like you know you already have like the server working on maybe you even have users already maybe at that point it could make sense to try to optimize things and then in that case I will do like you know the install on a normal standard Docker image and then probably copy the installed things inside of a Docker slim image but for installation because sometimes it requires like you know like things to to build and compile stuff uh it's just like I think it's just like simple number just to have this like this what I definitely don't recommend is using like a Alpine of or one of these alternative images because they uh they are super lean if you don't install anything on them but if you need to install Python and all the packages and all the stuff then they become like quite big and which is like you know equivalent to having like python slim or something like that and in the Alpine images you also have to install all the build tools and then you have to compile custom for that image which means that whatever was like the cost savings that you were making for using Alpine then like you're expanding that on on like the computation of always installing and compiling everything from from scratch so whenever you need to optimize and squeeze things then like Python's limp if you don't have those squeeze things and like you need to build a product and like move fast then probably just like not slim at first at least okay okay cool yeah this is very helpful context so yeah like you said let's use the normal python image and then um so here it um sets up a working directory then in this case it also selects a or creates a it calls it non-privileged user so this is kind of a best practice to have a new user that doesn't have all the permissions and then here it says pip install requirements so this is one thing we have to do inside Docker of course then copy our code and then in the end we expose the port and then we run the command that we just typed in so in this case we need to save all the requirements in the requirements txt file so I already did this so it's yeah fast API UV corn then pillow and torch and Transformers and the python multi-part is necessary for the file upload I think yes that's correct um all right and now um in this case we also need to have the rust compiler for the Transformers so I prepared the additional M command that we have here so let me copy and paste this in here so we also need to get a so this basically installs the rust compiler and sets it as environment variable and now to keep it simple let's actually remove the additional user because um hugging face needs to save and download and save the model and if we use this then we don't have right permissions so we could basically set um specific permissions but to keep it simple let's simply remove this and yeah so this is now the whole Docker file and now here it already suggests the command so now we can type Docker compose and then up and then minus minus build so the first time it will then build the image and this is because we also already have the docker compose file generated for us so this is a very simple file in the beginning it doesn't have a lot of info in it so it simply says that it starts in the current the docker file must be in the current directory and then it defines this part here so let's run the docker compose command and see if this already Works alright so the docker build process finished and as you can see we now again have our app running so let's go again to our endpoint and now we have the app that is running from inside the docker container so let's again test our endpoint try it out and let's select a different file for example let's select our tiger and then ask what is the tiger doing and execute and again we get our response so now we have this as a dictionary with the answer settings so our tiger is sitting so yeah this worked and now um again so now we dockerized our whole application simply with the docker init command that makes it super simple and in this case we had to do a few slide modifications um and then we can run Docker compose up to have the container up and running and yeah so that's it nice that that was that was much more straightforward than I thought we were going to have like more issues and problems along the way as always happens when like building something like but yeah that's awesome yeah that's true but yeah your your help and your input was super helpful you you also gave a lot of cool tips along the way so yeah I really enjoyed the session I hope you you did too yes thank you thank you very much for for inviting me okay yeah thank you

Channel: Patrick Loeber

Views: 17,847

Keywords: Python

Id: iqrS7Q174Ac

Length: 35min 17sec (2117 seconds)

Published: Thu Aug 31 2023