How to serve any machine learning or deep learning model using FastAPI

hello everyone and welcome to my youtube channel in today's video i'm going to show you how you can use fast api to build an api to serve machine learning and deep learning models so we have we have trained a lot of deep learning models in the past if you're not aware of how we trained bird for sentiment classification i would totally recommend you to go back and take a look at that but i will show you some of the code on how we train the model but today we are just going to use the already trained model and we are going to use fast api to serve it so what is fast api fast api is a framework which is much better than flash at least i find it much better than flask and i've been i've replaced flask with fast api it's it's high performance is easy to learn faster code and ready for production as it says here so yeah it's it's quite good and you will start loving it once you start using it so here i am in this folder called fast api and here i have main dot pi so it's it's the same model that we have trained in the past on imdb data set for sentiment classification and that's what we are going to use today so we have the dataset class and yeah i would like to mention that i'm using tase which is a very simple python wrapper on pytorch so you can use anything you want so we have the word data set and that returns us input ids attention mask token type ids and targets and then we have the model so i'm using tase.model i'm inheriting from there but you can enter it from nn.module if you're using purepytarch but then there's a few things that you would need to change and i have the forward function i have some i'm tracking some metrics like accuracy i have a bce with logic loss and i have uh adam optimizer i have get linear schedules pretty much default stuff so it's a binary classification problem and i'm outputting only uh one um output and i will just take the sigmoid of it and i will threshold it but for bc with largest loss i don't need to do anything so it's just raw output so this is also available in days so if you go and see these examples you will find it let me just open it for you um not day but days okay so here it is and um here i'll go to examples text classification and binary classification so here you can find all the code it might be a little bit different but the idea stays the same so now we have the model and we have trained it for some epochs not a lot and we already have that in here model.bin so what we can start now is we can just copy paste some of the stuff from here and then remove some of the stuff so let me take the model and data set and all the imports and maybe i will create a new file called app.pi or api.pi api.buy and i paste everything here so now um there are a few things that we will just remove so our dataset class stays like this i know it has targets but we can always supply fake targets for test data in the model class i we will just remove the stuff we don't need we don't need the training steps scheduler blah blah blah we know that it's num classes is one so it's always going to be one because it's test set uh so we have that we don't need the optimizer we don't need the scheduler we don't need loss we don't need to monitor any metrics because we don't have any targets and we just need a forward function and forward function should have the same uh if you're using these then it should have the same arguments here as you have in here in this datasets dataset class so i'm going to keep targets there and then i can remove all this and yeah this is my code that i will use for uh the api so now if i go to fast api they provide you with a starter code and it's very simple so let's start from there so let's start copy pasting some stuff so i'll take these three lines and i will put them here so i got typing for uh types type suggestions and i have this app so it's just initializing fast api i don't need pandas metrics or adam w okay so we got all this and now um what i'm going to do so it's it's like like you do in flask it's very similar but much better so now what we're going to do is i'm just going to take this part and paste it here so in flask you do jsonify here you don't need to do that and uh you got this you can just do app dot get for a get request or app.post for post request or something like that okay so now we have this slash endpoint and i can just say slash predict okay or maybe just let's just keep like this and to run this we will use uv icon so i'll go to fast api one and uv icon and i run it on zero zero zero zero or you can use localhost and twelve port twelve thousand so let's see how it looks like uh let me open port 12 000 here um okay it's not able to connect but that shouldn't happen oh yeah okay i have an error going to import uh module app so it's the other way around api and then app so api is a file and app is the app okay so now we should be able to access it so yeah you see uh let's show you the raw data and you have hello world so it's working one one more good thing is you can just open slash docs to the url and it will give you swagger api where you can just go and try it out so you can click on try it out and i don't have any parameters here so i can just execute it and then get a response very cool right um so you can try everything here so you can also get the curl request if you want uh okay now what i'm going to do here is first of all i'm going to reduce the size a little bit and create a new endpoint and this is also going to be a get request and i call it predict slash predict so slash predict you can also do slash break something i will show you on that part and here i'm going to change the name to fetch predictions now we have our model here but we have not initialized our model so let's initialize it model is text model and since i'm using phase i can just do model dot load model dot pin and it loads my model and i can also specify the device here if i want to but by default it's cooler so i will use gpu for this and then i have the fetch predict and maybe i will write another function called predict so i have the predict function and the prick function takes what it takes a string or maybe let's just write it here so i have the fetch predictions and fetch predictions should return me a positive and a negative label so positive will be zero let's say and negative is one minus zero so one minus prediction let's say and this is also prediction so now my fetch predictions should take us text which is a string we actually don't need the typing optional here i'm just going to remove that import should not import stuff you don't need and we have text string here and now what we're going to do is we're going to take the text and put it in a list of values so if you look at the training code here we have the train data set so i'm just going to take this and i'm going to copy paste it here and instead of training data that it's just a data set which takes a list or an array and a list or an array of targets so here i don't have any targets i can just do list with minus one that's my fake targets and here i can do text so in a list so now uh since i'm using these i can just do pred is list so it returns so like model dot predict will return a generator so i just convert it to a list and it predicts on a data set so data sets model the brick data set um and it should return me the predictions so let me see print red so let's say prediction is zero just so that this doesn't fail and if i i'll go here now and let me refresh it and see what what we have so we have a slash bricked endpoint now um fetch predictions so let's try it out click on try it out and input some text or maybe input something like awesome and execute it so let's see what we get we don't get anything it's it is executing okay so we got uh positive and negative and we go to uh here and we see we have a list of arrays numpy arrays so one more thing that we must do here if we are doing uh we are predicting only for one sample so let's make batch size one this is going to be much faster and uh let me save it again so since we used minus minus reload when we started uv icon it should reload automatically so let me show you again so uh this is the command uv icon api app host port and minus one is reload so whenever you make any change in the code it will reload uh the endpoint on its own so now you see it took quite a while to make the prediction last time so let's see how much time it takes now and it's much faster so um this is your request url so i'm just going to copy paste it and open it in a new tab and you see the raw data positive zero negative one and that's what we have so uh what i'm going to do is um since we since since we have a list of um arrays so this is our first element and then second and then third this should give us a prediction and if i convert all this to float i should have a prediction value and i can remove this i can take this here put it here and that's my prediction so pretty simple in like three four lines of code you have your pricked end point and now i'm going to save it and here it should restart automatically and it has and i'm going to refresh this and see what happens so i'm getting three point two two and minus two point two two something like that doesn't make sense so we will do one more cool thing i will just add torch dot sigmoid here in this model and save it again and let's try again one more time and see what happens so now positive is 96 and negative is uh 0.03 i will i will add one more thing here sentence and this is going to be text the text that we input so that you will also see the text that i'm inputting because you are not able to see the um bar so this is an awesome video and uh it gives me 98 and and this is i don't like this video let's see let's try i don't like this video and it gives me a negative sentiment more than 0.77 77 so um what i can do is i can also add another argument here or if i have to convert it to a post request i can do that and um using dot post so for for that i will show you like very quickly what you can do is you know you can use pydantic from pidentic import base model and you can create a model so like i will create new class um so this is your api model and it has like uh i will call it sentiment predict okay and it inherits from base model and here uh i can input some things like text so text will be string and maybe a threshold now i can say okay threshold can be and float sorry and now i'm going to use this as an input to the model so i will just say sp sentiment predict and here it will become sp dot text and i will say if prediction greater than sp dot threshold then um sentiment equal to positive and i can just say here sentiment by default is negative and i can add and here i can add just so you can see here i can add sentiment sorry not here [Music] okay okay looking pretty cool now uh so let's see text is not available so this should be sp dot text and i will change my end point to post and point now post request so you have to send post request so let's go back to swagger and let's refresh it so now this is boost slash correct and we try it out it takes a string and a threshold so let's say i set the threshold at six this is so cool i love it and i click on execute and it will give me the curl request and it gives me the response body so positive negative detected sentiment is positive and here is your sentence so now you can go and you can use it in production if you want so i can you can just you can increase the number of workers so minus minus workers to four let's say to four and now you have four workers running so it can handle multiple requests in simultaneously so yeah that's all for today's video uh this is how you can create an api for any kind of machine learning or deep learning model very easily and this is the most basic version of it and you can do a lot of cool stuff i've not even shown you one percent of fast api but there's a lot to learn with fast api and i hope you will like it and i hope you like this video too so yeah if you do like it do click on the like button and do subscribe if you haven't subscribed yet and do tell your friends about it and thank you very much see you bye

Channel: Abhishek Thakur

Views: 10,904

Rating: 4.9448819 out of 5

Keywords: machine learning, deep learning, artificial intelligence, kaggle, abhishek thakur, fastapi, fastapi machine learning, fastapi deep learning, using fastapi for bert, sentiment classification using bert, serving bert model using fastapi, fastapi serve machine learning models, fastapi serve deep learning models, deep learning using fastapi, fastapi get and post requests, how to create api for deep learning models using fastapi, how can i create fastapi api, api using fastapi

Id: Mw9etoRz0Ic

Length: 18min 10sec (1090 seconds)

Published: Sun Jan 24 2021