Talks # 8: Sebastián Ramírez; Build a machine learning API from scratch with FastAPI

hello everyone and welcome to talks number eight well today's guest needs no introduction but i'm going to introduce him anyways he's sebastian he built fast api and typer and is currently working at explosion ai so without wasting time i i would hand it over to sebastian now so welcome sebastian thank you very much thank you for the invitation thank you for having me and thanks everyone for attending and so we are going to see uh this talk about serving machine learning easily with fast api fast api is a python framework for building web apis with high performance it's easy to learn it's fast code and it's ready for production and this time we are because i have given this talk you know a couple of times before uh in the machine learning meetup from berlin which is the the place where we go to meet this time we're doing something new which is we're gonna do a live code version of the whole uh talk so the first tip is a never do live code versions because they must normally go wrong so you're gonna see me fail uh dramatically let's see how that goes so first who am i i'm sebastian ramirez i'm a developer at the explosion in berlin germany i'm actually from colombia that's why the accent you can find me on github linkedin twitter i'm working for explosion explosion are the creators of spacey which is a natural language processing package so for doing things like machine learning over text and prodigy the package for doing data annotation for a machine learning data sets using active learning and think the deep learning framework using ideas from functional programming and python types hey i created fast api and typer and we're going to talk about fast api so a little bit of class api it has around 18 000 gift of stars it has been growing about 1 000 stars per month so people have been liking it it's currently used by microsoft over netflix and a bunch of other people and the performance is in the top rank for python frameworks so that's quite nice but now why would would it be useful for you machine learning and data science people like what are the benefits of farsis api or why would you want to learn about it so we're going to talk about fast api for machine learning people i'm going to assume that you know machine learning i'm not going to teach you machine learning itself and like you can learn that and you probably know that a lot better than me uh but but i am gonna assume that you know the basics of web api web or api development just like the basics of how these things work like http and json nothing else and we're going to focus on these things of building an api for machine learning a little bit about fast api before we dive into the code class api is based on standards like open api json schema or oauth2 all too is this thing that systems like facebook twitter or github use to authenticate people so when you click on an application and say authenticate with something that's all done with oauth2 and fast api has a lot of integrations for these things these are all open standards and fast api is built around them so that allows a lot of interoperability that for example provides automatic api documentation like this one that we see on the side and we're gonna see an example of this in a bit uh this is just like about the standards that we are we are not gonna see that very obviously in the code so that's why i mention it here so when you build an application with fast api is built on top of open standards for free and you get uh an application that is based on those standards now because this is the version of the talk where we don't show a lot of slides slides and may means a as dinos said once talk is cheap show me the code we're gonna jump right away into the code so let me go here to the code you should be seeing my screen i have here a python project we don't have anything here we just have some requirements i'm installing fast api which is the thing that we're going to learn spacey which is the machine learning package created by explosion this is open source and free and everything so we're going to use it as the demonstration we're going to use uv corner to run the server that will execute the application that we are going to build with fast api i also have some development requirements here just to to link the code and to format and i'm using this package that is installed automatically when i try to run a jupiter things inside of my visual studio code editor we're going to see how this works in a bit i have a virtual environment here vm this is a standard python that you should be using we are not going to go deep into how this works and how this is created this could be like for another session or something for now let's just run jump right away into the code we are going to create a file ml.pi and we're gonna import spacing import space because i have a this package this package over here ipad kernel this is used internally by jupiter because i have that one i can run this command that is wrong line run selection line in python interactive window i'm gonna run this so this is going to start an interactive window equivalent to a jupyter notebook where i can run specific lines in my code so i can keep the code in my python files but explore the data and the packages and everything live in a jupiter notebook so i'm importing spacey and now i have spacey here i can use this just as i will use in jupiter and i can see that i have the module there cool now i'm gonna load a machine learning model that is like a pre-trained machine learning model so i'm gonna call an object nlp which is going to be equals to spacey.load and the name of the a model that i'm going to use is a second and i forget always the name and core web exam so this is the pre-trained model that is the small version of the model train on this data set for the english language i'm gonna load this and run this on the a oops i didn't run it i'm gonna run this here on jupiter now i have this nlp object here nlp i can check it and indeed this is a spacey english model now i can use this a machine learning model to do some machine learning over text so i'm going to call this as if it was a function and then i'm going to pass a string inside of it i'm going to take it from here we are going to use these examples in a little bit but for now i'm just going to take it here and i'm going to assign that to a variable doc so you can see that doc is a result of running my model over this text if i run this with shift enter it will be executed here on the side so i have now this dock object if i do for example type of dock to check what is this thing that i have here i can see that this is a duct object from space now with this dock i can extract the predictions generated from this model so for example i can for each one of the entities in doc dot ends i can print print the text of the entity and the label of the input i can select this section of code and i'm gonna run it and it's gonna will run as if it was a cell right here and i can see that the model is detecting apple as an organization which is this apple and is detecting uk which is this uk as a geopolitical entity so this is like a location or a country something like that and one billion is detected as money which is this one nice so this is a machine learning model that can analyze text it's just a very basic model pre-trained model we're not going to do anything fancy with this but you get the idea that we have some type of machine learning model and then we have some type of input that we are going to pass through the machine learning mode and then the material model is going to generate some type of output or prediction you will have an input that could be an image or a video or an audio or anything that you are doing machine learning over and your model will somehow read that input and generate some output so this is the basic idea uh you could have like any imaginary model that you are using pytorch tensorflow theaters whatever you're using it will work the same we're gonna use this as an example we don't need this a jupiter interactive window anymore so i'm gonna close it and we are gonna reuse this in a little in a second but we don't really need this document with this specific pre-made text and this part so i'm just gonna remove this part we're gonna save it and now we're gonna create another file now this is just the basic example that we're going to use for our machine learning model now let's get into fast api so i'm going to create a file main.pi and i'm going to import fast api let me see that i'm not skipping anything that i was going to show you yeah perfect so from fast api import fast api the clouds api and i'm going to create an object app which is going to be an instance of this path api class and now i'm going to use the add symbol the decorator of app dot get and then i'm going to pass the string slash and then i'm going to write the function below that define with a name let's see what this is all about so app is the application that we are creating which is an instance of this fast api class and then we can use this app with a decorator if you have ever used flask this is very similar to flask if not don't worry this at symbol what it does is that it's going to somehow use modify or register or do something to the function that is below so what this is doing is that this is registering this function that is below in the app and it's going to register it to handle requests that go to the url slash so the root a path the root request and it's going to do that when the http operation is get or the http method is get so because this whole thing combines a path and an operation i'm going to call it a path operation now what we are going to do here is that we are going to return a simple dictionary that is going to have a message and we just hello world very simple this thing that we have here is already a valid api this is already a valid fast api application now let's actually go and run it i'm going to open a terminal and i have unicorn in my virtual environment let's see if i have the correct cubicle the one from my virtual environment that i have here yeah that's right so i'm going to run uvicorn and i'm going to tell you record that i want to i want it to run main i'm going to pass base main this part refers to this file main which is this file that we are editing here it refers specifically to the module to the python module and this app here that is yeah i can select the f this app here refers to this object inside of the module so this is the equivalent of from main and then import this column is the equivalent of import and then app and ubicon is going to import main from name is going to import up and it's going to execute it as a function so we execute it like this so this is going to execute and run our fastapi application and it's going to run it on a unicorn on this url now let's go and check how that looks like we are over here so if we reload this page now we see actually let me hide the file here and move this to the side so now we have the browser here we are going to the location where ubicorn is running and we can see that we receive a json message that contains the dictionary that we sent so we don't have to send some type of special object or some instance of some specific class we can just return a dictionary and fast api will take care of converting it to json so this is actually json this is validation i have a plugin to show it formatted for me but it will look raw like this just like plain json and json well we can read it anywhere else and we can interact with it very easily from code and from other places so nice this all works but now this is the most interesting part if we go to slash dogs then we get this nice user interface that documents our api automatically so we can see that we have this a path operation as i was calling it that goes to slash to the to the root url and there is a get http and method or http operation and well we don't have any parameters we don't have anything else but we can actually try it out if we click try it out and we execute it this is gonna go ahead and call our server you can actually see the log here that this is holding our server that is running live and we are getting the response this is the response that we have from here so we can interact with our api while we are developing live from the browser which is automatically documented and you can see that this hello world that goes to this url is the same that we receive if we just open that same url on a new window nice now let's add another path operation with a little bit of more stuff we're going to say app dot get and then we're going to receive a an article and this article is going to have an id article id and now we say analyze article we are declaring that we are going to have another panel variation that goes to slash article slash and this is going to be a path a parameter that's why this is in curly braces and you can remember that is curly braces the syntax that you have to use because it's the same one that you will use for f strings or formatted strings and this article id is going to be a path parameter so we want to receive it in the function that is going to handle it so article id will go here but we want this article id to be an integer so we are going to use standard python type annotations like this one to say article 80 is going to be an integer notice if you haven't seen this type annotations before this is different from the default value so this will be a default value this is a type annotation so this part is what we are using to tell a python and our editor and fast api that this is an integer let's remove the default value because this is a required parameter that should be taken from the path and now we're just going to return return article 80 is this article id that we are receiving now uh if we reload the browser window we won't see any change because the server has not been reloaded the server is running still the old application so we have to stop the server and we're gonna start it again but now i'm gonna run it with reload with the reload option with this cli option so when i run it that way then it's going to reload automatically when i change the code here that that's not how you will run it for production but that's how you can run it for development so that you can like iterate very quickly now if i reload the browser i'm gonna get now that we have the main uh path operation that will appear on the new one that is again an http get operation or http get method and goes to slash article and we have the article id we have the name analyze article which is the same name from the function and if we check the documentation internally we can see that now we have this article id which is a path parameter that should be an integer is required and is extracted from the path here so if we click and try it out and we try to execute it we can pass here an article of five let's say we execute it and we receive the response saying that the article id that we sent was five this response comes from here and this is how the url looks like so we are saying slash article slash and here goes the path parameter now if you notice this five goes in the url so this is natively a string but fast api is not only using it to document what is the power parameter it's not only documented that this is an article id that is required and then it should be an integer it's also converting the data so that we actually receive an integer in our code so we could say for example previous id and say article id minus one so we can do arithmetic with this thing because possibly we'll make sure that we actually receive an integer this is not going to be a string with the number 5 inside this is going to be an integer and now if i try to execute this again because the server is reloading automatically if i execute this again now i get the new response article id the five that i'm passing and previous id four so yeah we are being able to do a standard arithmetic so we are getting automatic documentation from this type annotations okay which is what we see here that is the technique that is an integer and fast api is also doing data conversion so that we get the actual data type that we want to receive now we can also add another parameter here let's say q which is a string and by default is none we save that and if we load here the user interface for the documentation we can see that now we have the new parameter queue which is a string and is taken from the query first api detects that this queue is not part of the path parameters because it's not in curly braces like this one and uh so fast api knows that it has to read from the query and because it is by default none then it is not required you can see that article id is required q is not required if we try it out actually let's return q here it's going to be we executed because by default q is none then in json the equivalent in json for none is null so we receive q is not if we say hello world here in the query parameter then we receive the query parameter hello world and this is how the url will look like so this query parameter the query parameters are these ones that go after the question mark which and are like a key value pairs so query is this queue where a parameter is equal to this string hello world and if i wanted to make this query parameter required i just remove the default value and then now fast api is going to know that this is a required a parameter cool but let's not make it required we're going to make it by default no because a query parameter are required for a parameter it's very weird now let's start doing some basic machine learning so we have this ml module that we created here so from ml we're gonna import nlp which is the model machine learning mode that we're gonna use and now we are just gonna uh we're just gonna read the we're just gonna count how many entities we have there we're going to say count is equal to the actually first we have to create a document from each specific query we're going to do machine learning over the query or the stuff that we pass in the query just it doesn't make that much sense it's like a contrived example but we can see how we can use uh the parameters from the url so we're gonna parse with nlp the query parameter but now this query parameter will be none so we need to make sure that q exists so we're going to do if q to make sure that it actually exists then we parse a u with our machine learning all we generate the document and now we're gonna say that doc that ends which extracts all the entities and we can count them we're gonna count the number of entities we have and we assigned it to a variable count and now we are just going to return count here and it's going to be just this variable but now if we see here the it is telling us that count is possibly inbound so this could actually not have a value because we are assigning it inside of the if so we need to have a default value for discount so count by default is zero if there are no entities and now we can just try this again article 5 with no query we executed oh wait we have to reload this because i made it not be required again so article 5 no query executed and now we see that we received the article id the query parameter is no and count is zero because we didn't pass anything if we pass some string like i eat pizza this doesn't have any entity in it so the count is zero but now if we open and if we pass this uh text that we have before apple bytes you can start the 41 billion we execute that now it's going to be able to detect those entities and we already get some type of prediction of something at least it's showing us that we have three entities detected in this text for apple uk and one billion cool so we we're getting somewhere now we actually don't want to have to pass like all the data in a query parameter so we are not going to use this query parameter we're going to extract the data from somewhere else we want to be able to receive json data with more complex stuff so let's actually remove this for now we're gonna we want to reciprocation data we don't want we don't need this article id actually because we are just going to receive new data we can just remove this part for now because we want to receive data we need to receive it not as a get method or get operation but as a post and you get out of the completion here as a post a operation now let's show this here if we save it if we reload the page now you can see that the previous version was a get operation and a get operation for article article id if we reload that now we can see that we have now a post http operation so with a post we can actually send data let's start by just sending some basic body we're gonna send a dictionary so we're just gonna receive whatever is in the body i'm just to work on that just to check it now we load here the page and now we can see that facility is going to be able to use these type annotations to detect that we want to receive some dictionary and we can try it out we can execute it we are just returning the same dictionary back so we can say a content world hit it and receive the content back cool but we could actually use this dictionary to say a two and then instead hacker and execute it and our application right now as it is is going to receive that and use that data and we don't necessarily want our application to go and make payments to mr hacker just because we only declared a dictionary so we want to declare what are the data shapes that we want to receive for that we are going to import from pride and peak import base model fast api is built on top of pydantic and uses by dante internally for everything for the data handling and fast api is so it is based on top of identity for the data handling and on top of starlet for all the web handling starlet is what provides all the performance uh too fast api starlets is like the the the main thing that is uh handling fast api underneath we are not going to use anything from starlet directly but fast api is using it on the mid so we import this pytham pic from identical import base model and we are going to create a class let's scroll a little bit so we have some space class article which is the thing that we want to have that inherits from base mode is the thing that we import and now we're going to say that we don't want to receive this pay too the thing that we want to receive is just content and content is going to be of type string now instead of receiving this body which is just any dictionary that we receive we're going to receive an article because the name is going to be more intuitive and we declare it as of type article the class article this is the standard python syntax to say that this parameter article is going to be an instance of the class article with a capital a and we just return the same article if we save it and we reload the page here we're going to see now that class api is going to be able to read this class that we created and show us in the documentation what is the data shape that we are expecting in the server so we are the claim that we want to receive a json object that contains a content which is the same content which is a string which is this string you can see that this syntax is very much similar to the data classes we are just inheriting from base model because base model is the thing that is going to do a bunch of extra things for us and that fast fp is gonna use underneath for example the automatic documentation now if we try it out we can execute this hello world and we received the response back let's actually return something different so we can see that we are actually reading and interacting with that so we're gonna receive a message and return the message so the thing is that this article with a credit has been an instance of article fast api is going to make sure that we actually receive an instance of article this is not going to be a dictionary this is actually an instance of this thing so we get auto completion automatically we can say content and get it right away if we forgot if we didn't have this let's say that we had the previous dictionary body then we will have to do square phrases and then put content but we might forget if it was content or if it was text or if it was message and we wouldn't have an easy way to know if you're writing something wrong but here because we are using this class and the standard files and type annotations we're going to have to completion and it's going to detect the now i'm saving it let's execute this again sending the same content hello world and we receive now the message hello world so we can see that we are receiving this response great what is next so let's actually say that we want to receive not only a content ah well and we we didn't check this thing that we want to say what happens if i send this and i say pay to mr hacker and we execute that api is not only gonna do and it's not only gonna document the data that we are going to receive with this class and it's not only going to convert the data to have the shapes that we declare for example to receive an integer when we declare integer it's also going to filter the data so that we receive exactly what we declared so we pan even if someone passed something else we are not receiving uh that extra stuff now let's declare that we also want to receive some comments my comments are gonna be several per article so this is going to be a list we just add these comments here declare a list and now if we reload here we're going to see that the example shows that comments is effectively a list that can contain some stuff can execute it yeah that's a list let's actually return the message and return the comments just to check them it will be article dot comments and we get completion everything nice save execute this again and we get comments the comments is no we actually don't want to receive no comments we want to receive how much there a string oh well uh actually first let's try this what happens if someone sends a content without comments like an article could have no comments but if someone tries to send it then we're gonna receive an error telling us hey these comments are required so facebook is going to check and validate these data but we actually don't want the comments to be required so we can make them by default have an empty list so we can use this empty list by default and now if someone sends the content which is required it doesn't send the comments that's going to be fine because comments by default is an empty list great but now let's say that we want comments to be not just a list of anything we want it to be a list of strings so for that we can do the following we can import from typing typing is part of the standard python library so this is standard we're going to import list list with a capital l and then we're gonna instead of using the list that we're using here we're gonna use the one with the capital l that we imported and we can pass in square brackets what is the internal type of this string this is the standard python syntax to say that comments should be a list of strings and now if i reload this check that i i actually don't even have to do anything here just update the declaration of what is the this pythagorean that i'm gonna use to read the body and if i reload the page here now i see that content is a string and comments has to be a string if i try it out execute it yeah well i get the stuff that i was expecting i get it right back great now let's actually do some machine learning over this thing so we want to uh first extract a document of processing with nlp we're going to process the content of the article so we can do autocompletion article dot content here it is nice quick and easy great and for this we are gonna do the same thing that we were doing before so for end in dot dot ends we're gonna do something what are we gonna do we're gonna let's say generate a dictionary that has the text of this npt and it has the label of this entity but we want to put this dictionary somewhere and return it in the response so we're gonna store all the entities in some lists so hence it's going to be this list now we're going to put this inside of the list so what we do is ends dot append then we pass that dictionary inside so now we are reading all the entities and we are extracting them and we're gonna send them over here let's just return it and release ends great i reload here i get the same thing as before if i try it out let's actually send some text that will contain entities comment hi we execute that nice it works we get the message that contains the same content that we were processing we get the comments and now we get the ends the entities that we were extracting we get apple which is an organization uk geopolitical entity one billion which is money so we are already we already built a basic machine learning api that works right away without problems now let's do something let's do something more let's say that we don't necessarily want to have a single article we want to be able to process multiple articles per request because that's what the client needs or something we can do these same standard python type annotations that we've written before and we can say we're not going to receive a single article it's going to be articles and articles is going to be a list of instances of this article so now we are going to receive this is going to be effectively a list of these instances now we can iterate four article in articles and we can just move all this code a little bit to the right so it's inside of the for loop and we can see that article even though is each one of the elements in this list and we are iterating we still have condition we still have the completion and type checks and everything because we are using this standard python type annotations so that's great now we are not going to return the message because we are going to receive multiple articles and for now let's not return the comments because we are going to achieve multiple articles but we can just return the entities now the entities are going to be extracted for all the articles so let's put the entities on top and we can now run this and if we reload the page here we're going to see that now fast api has a picture that we say hey we don't want a single article we want a list of articles and the documentation now shows that this is a list you can see the square brackets here or in json terms this is a json array that is the same as a python list and this is adjacent object or the equivalent of a python dictionary now let's try it out and let's try that let's try that with actually a couple of articles from here the examples we have here so we have apple buys uk started for one billion comments nice and the other comment it was expected or something then we have another article here that says microsoft creates new a project template with us api in space cool if you remember we said that comments was a list of strings that by default was an empty list so comments is not required so this will work but if you see this json is actually invalid because we have this extra comma here if we execute it then this is going to show us hey you are sending me invalid data i can't read that json.json is invalid but we get a nice error telling us hey this is incorrectly it's in this position line number column number blah blah blah then we can just fix it execute it and now we get the data back we can see that we have extracted the data from both articles of we were doing here in the code so we have a apple is an organization uk is a geopolitical entity one billion is money microsoft was extracted as an organization and because we are doing machine learning we are dealing with statistical models they are bound to have errors we have to have that into account so ai is being detected as a geopolitical entity which it is not up to now fortunately uh let's keep it that way so but the thing is that yeah like we are gonna have errors because we are doing machine learning we are expected to have errors but the model is working fine and like the api is working fine now let's actually do something and let's extract the comments so we are iterating for each article in this list of articles now we're gonna also iterate for a comment in article dot comments and we're gonna do something with this comment now notice this each one of these comments for each is a part of each one of these articles which is part of the list of articles that we are receiving so we're declaring that we receive a list of articles and each article contains a comments a attribute or a comments field which is itself another list that contains strings this is actually even difficult to say properly but it's very easy to declare all that and we are even having a nested for loop and still because we are using all these standard python type annotations the editor is going to be able to provide us with completion so it's going to be able to know that comment is a string if we try to make it in uppercase we don't have to remember was it like uppercase or capitals we can just complete and see that oh the method is called upper we don't have to remember everything by memory i'm not really good at memory so this helps a lot and we are just gonna store all the comments comments in a list here so we just do comments dot append and we're gonna append all the comments in uppercase layers just to make it look as if it was people streaming or something so we have call and we are gonna run it here comments is gonna be equal to comments comments girl yeah nice now if we execute this thing then we receive the entities that we are extracting and we also receive the comments in all capitals if we add a comment to the other article here comments this will be a list of strings and let's say oops no not that yeah let's say they say yay because it's a very professional comment or something and now it seems like they were screaming jay we are being able to collect all the comments in all the articles very easily also because the editor is going to be able to know that each one of these comments in the deeply nested data structure is a string we did something that is incorrect or invalid for python like chromaint plus 500 and we save the file the editor is going to be able to notice that we are doing something correct because we cannot sum a string and an integer and it's gonna show us right there in the editor we're gonna be able to see the error even before we execute this code we are going to be able to be safe from these types of errors very easily also notice that each one of these comments like all this is declared in our own data shapes this is not completion and type 6 and stuff for the things that we import from fast api this is completion and type checks and all this for the things that we declare so this this is quite nice and we can do arbitrarily nested and complex data structures and have all these benefits by default now what happens if we actually send invalid data so let's check all this whole example let's try to send this example completely this example this is valid json so the format json is correct there are no like extra commas or anything but the data is not valid with the shape that we declared uh let's see if we can spot it quickly before we execute it okay we execute it and then we are gonna fast api is gonna make sure that we don't have to protect ourselves from incorrect code from invalid code it's gonna protect us from it so for this code to be executed fast api already made sure that all these parameters that we clear and all the data and all the stuff that we occur is actually correct and valid it's going to return as an error telling us hey the data is invalid this is not gonna this is not gonna work and this is a it just returns the air to the client right away these errors detecting the data and validating the data based on this type completions and all that stuff this is saving us a bunch of lines here for each one of these class operations that we built this is saving us a bunch of lines but it's also not only telling us that hey there's an error it's telling us exactly where is the error this is also saving us another bunch of lines of code in here a lot of code and a lot of headaches that it's gonna save us now let's see what is this specific error that we are receiving it's gonna say this is the first error it says that the location is in the body in the index two let's check the contents so this is the same thing that we're sending before content is a string comments this is all fine this is index zero this is index one which is also valid this is index two this is where the problem is what is the problem let's see the problem is that content is a field required and if we see we are not passing the content we are passing this new thing text but we declare that content was a required uh parameter or required field or required attribute in this data that we are sending so this is invalid now let's check the other error this one over here so it says that also in the body at index three so if this was index two then this is index three inside of index three in the comments in index one so inside of index three which is this one in the comments index zero is this one is a string that's fine and index one what is the problem with this index one let's see the problem is that a string type was expected and what we said was oh no a json object we didn't send a string so we are getting uh data validation and errors automatically that tells us exactly where the problem is in the payload that we are sending and that again saving us a lot of code in here now let's say that we actually want to so fast api is using this standard python type annotation well the first benefit you get from standard python type annotations is autocompletion as we were checking here type checks as we were checking when we tried to sum the comment with a number or something that's by default with anything that you do on python and add standard python type annotations like this but on top of that fast api is using those type annotations to add documentation to our api for example to have an example of what is the data that we want to receive and it's also doing data conversion so that it makes sure that we receive the data that we declare but it's also doing data validation it's making sure that the data we receive is actually the shape and the types that we receive but we can also do more stuff to document our api and to do stuff to show what is in our api we can add a top string here like a standard string and we're gonna say analyze an article and extract entities with space we want this to look very cool and fancy so we're gonna add some sparkles and stuff and if we save it as api we extract this dot string and it's going to show it to us in the interactive api that we get here and we can actually use this is part this is all because we are using this open standard open api and all this user interface is provided by swagger ui which reads the open api generated that fast api so we are taking advantage of all these standards and all these things to to our benefit we can actually use mark them in this documentation so we can say a statistical models will have errors and we want to highlight the errors so we can make it uh bold using markdown and we can say we also want to highlight highlight will so we can make it italics and we can reload and see that now we have this in acrylics and this in a a bold case and we can also say like i don't know we want to extract a npts and scream comments because that's what this api seems to be doing and then if we reload that we get an actual list because this is just sold using margin so we get a we get to document our api we get to do the validation documentation of the data and actually pros or like textual documentation of what is in our api in our code right in our code we don't have to maintain a separate wiki or something like that or doing those extra tricks to keep in sync with our team or clients or anything cool i think that wasn't that bad we didn't have so many problems that run quite well uh cool we saw like the full experiment we have a full api with machine learning with fast api now a little bit of extra info fast api has the is in the top rank of performance for python frameworks this is a benchmark run by tekkenpower which is a third party and they benchmark a bunch of languages and frameworks and a lot of different tools with different combinations of databases etc and here we can see the blue la the blue lines are python the green lines are go lang go the programming language go is a compiled language and python is an interpreted language so go has to be a lot faster than python because it's compiled it should be a lot faster than python nevertheless fast api and a bunch of folders get to be even faster than some of the go lang frameworks so this is quite nice we are getting quite good performance and we keep our tools and our language that we are able to use for our machine learning and for all the other stuff which one of these is the fastest is not necessarily that clear relevant you can see there's some random error here fast api is built on top of starlet so of course the star of course started to be faster but you can see that the uv core which is the server which is not even a framework this is the thing that is running everything this should be on top of everything but yeah you see we get some random noise here the point is that we get on the ballpark of the best performance that you can get for python and we get data validation serialization documentation all that stuff and well you can see that flask is somewhere over here and django will be somewhere below there getting fit and screenshots uh but you can see that we are being able to handle 15 000 requests per second for this specific benchmark so that means that it's over 9000 requests that's that's quite fine fastapia has a bunch of other features this is a nice dependency injection system which is very simple simple to use but very robust we can do things like managing database sessions for any type of database we we use it could be a sql database or in a sql database we can do authorization authentication stuff with this dependency ejection system for example to validate that we have the user with the token or the cookie or anything that we want and validate that for all the path operations that we need it has tools to integrate with security tools like oauth2 so that we can build things like they do with facebook or github or those those things so we can authenticate with github or we can build something that other tools can use to authenticate so we have a lot of flexibility for that there's support for web sockets if you need that we can receive or send files there are lightweight background tasks so for things like sending email notifications these wouldn't be for training machine learning models because that will take all the cpu this is only for doing lightweight stuff like sending an email notification after after sending the response for example or saving a record in the database after sending the response for things like that you can use this integrated background tasks there's easy graphical integration if you're into that there's a bunch of other things like django like uh like templates like you would use on django or flask for rendering html on the backend although fast api will be better suited for apis but you know you can get like all this stuff that you will expect from one of these frameworks some other tools that you might want to check typer is for building command line interfaces it works very much the same way as fast api uh you get completion type sheets and everything you've got and your users are going to get a shell completion in their shell so tap completion for all the shells think is a functional deep learning framework that is compatible with your favorite libraries like tensorflow pytorch etc etc that's what i have for you thank you very much and you can check the documentation here if you have a questions later after the ones that we are gonna talk next then you can also go to the repository and ask there in the github issues uh yeah thank you very much awesome thank you very much sebastian so i i enjoyed the talk and i i think more than 200 people who were watching the talk really enjoyed it a lot too so awesome i'm glad to hear is it fortunately it wasn't that boring with errors you got to manage them under control it's always fun with errors yeah uh yeah i have had really bad experiences live coding but you handle it quite well i think yeah so we have it's probably a first yeah okay we have many questions um is it possible for you to uh turn off sharing turn off what sharing screen yeah show let me see this is over here it's the present thing i think there it is awesome yep so i think uh this this is much better now okay so one of the questions i mean this question has been asked so many times and you must be bored of this question now right yes you can guess the question so it has been asked in a different way and some people have asked the same question that you hear a lot of time so why should we switch from switch to fast api from flask but it has been asked in a very different way by one person so what can motivate companies to switch from flask to fast api for production and machine learning yeah so uh well like you will get if all the benefits that we that we just saw there uh but like the approaches to do that uh are like very specific like i wouldn't recommend you to go and migrate like i don't know thousands and thousands of lines of code from flask to fast api right away in one single batch what i would suggest is that you add a front-end proxy for example a traffic uh on top of it of like both applications the flask and the new fast api application and then add all the new features or new layers all little pieces that you need to refactor and to improve and then add those to fast api and keep all the rest in flask you can actually if you are using i don't know sql alchemy models for for the database or mongodb or whatever you are using in flask you can use the same models and connect to the same database and everything from fast api you have both applications running at the same time so you could get like the benefits of fast api for the things that you need to like new features of or the apis that you need to refactor and to update uh that will be the approach that i would recommend and then you can add traffic on top of it a and traffic will be able to also handle https certificates for you for free and do it like all automatically so you get like a lot of benefits from that and you can migrate very gradually to fast api if you want to get like those benefits of data documentation validation so you'll see all the stuff awesome i mean you have already answered this question but uh do you have do you have uh uh some kind of suggestions or best practices for reporting for parting to to flask yeah that's what i was saying like that that will be it like doing all that like doing the immigration slowly and gradually also if you have like an application that is already working you don't need to add new features there's nothing else new to add then it probably doesn't make sense to just go and migrate everything just because if you need the extra performance if you need the validation or for example if you need to make sure that you are really validating very well the data then that will be a reason but if everything is working fine and i don't know it's a very old application that just a few people are using then it probably doesn't make that much yeah okay um do you does fast api provide plug-in for extendable architecture yeah for i guess it will be something like this having the application separated in files so if the question is about separating the application in files like in flask it will be blueprints yeah there's something in fast api called api router in fact if you search flask blue prints fast api you will get directly to the documentation in fast api for that so there's a very nice way that you can instruct your sub routers that is like subapplications that you can integrate all in a single big application that if the questions refers to having multiple files in general about plugins in the case the question was about something different most of the features of the things that you need to achieve with plugins for other frameworks for example doing authentication or handling automatically json web tokens or things like that most of those things can be done with the dependencies in fast api so you actually don't need a plugin for most of the things because you can just use the library directly to for example to read the json web token and just use it in a dependency and you don't have to create or to find a plugin that does that json web token with fast api you can just do it directly in a dependency and it's very easy to achieve on the other side for many packages and many things fast api is built on top of starlet starlet is built on top of the standard ascii agi so there's a for flask and django the standard is called whiskey wsgi the new standard which allows a synchronous code execution is ascii and there's a lot of plugins and like things that you can and measure major work that you can connect with as for example like sentry datadog uh even elasticsearch uh atm there's a bunch of things that you can do in general for ascii and all those are compatible with fast api so there's also like a lot of extra plugins apart from what you can just build by just adding a function with a dependency awesome awesome and there are a couple of questions around authentication and authorization so one question uh is does fast api has extensions like out2 and can you discuss a little bit on how we can add authentication and authorization to fast api applications yeah so oauth2 is actually integrated into open api and fast api has tools to integrate these all through tools into the same application so that even the authentication gets integrated into the interactive user interface for the api it's like even to that level that you get like you can authenticate in the user interface to explore the api so that only authenticated users can use your api there's a bunch of tutorials in the documentation for doing specifically for doing uh authentication with json web tokens there's also tutorials for doing basic authentication there's a bunch of tools internal and fast api that help with building this off uh to tools and also it's authorization flows and these things you can also have cookies if you have like a more traditional session management system or like any of those things and you can combine them together however you want there's a bunch of documentation that for that so one question is it so the person feels like you have successfully built the end-to-end flow from code to deploy plus the rest api so what are the future plans what is the roadmap for fast api there's actually a bunch of things uh recently i had been playing with adding a little layer of this is actually not into fast api but like on the side but will be very easily integrated with fast api which is adding a little layer that makes a sql alchemy models that makes pedantic models be also sql alchemy models so that you can declare your database models with standard python type annotations and have validation and documentation for those models automatically that is something that i have been playing around with recently there's a bunch of extra things that i want to add to fast api one of the main ones uh that is not going to be soon but i want to at some point how a way to build it is to create an uh an admin interface that is independent of the api uh user interface that documents the api itself and uh just like a user interface that works as an admin based on the open api generated schema so that is something that is not based or attached to the database uh that this in that is independent of the database and that works specifically with open api i have that like as an integrated admin interface because fast api doesn't have like any uh attachment or like any compromise with any specific database or any specific orm so then it's not easy to assume that the user is going to have some specific rna and generate some admin interface for that specific orm but then we can build something on top of the generated open api schema that's something that i really want to play with at some point hasn't happened yet but yeah it's one of the like future plans for the whole fast api ecosystem awesome um one question is what orms would you recommend to use with uh fast api i would recommend you to try and whatever works for you if you are already using something for example if your asian sql alchemy then you can just use that if like if you're in a in a big corporate or something that wants to use something that has been tested for lots and lots of years or something that you can just use sql balcony if you want to have the benefits of async uh tools and have like the best performance that you can get that you can probably try something like tortoise rm tortoise rm recently added a integrated pedantic support for their uh for their models and this is in fact what what is thought in the test-driven io course of fast api so tortoise rm is a nice a nice one to try another one is gino which is based underneath on sql alchemy but has like the async utilities and like performance benefits uh it depends on like what are the the things that you want you want to to achieve but i will i will think like some of those are probably a good idea some of the older frameworks that were made mainly for python 2.7 and hasn't been updated and that depends on the specifics of threaded locals this is like a very technical ideas and and like a very very technical like nuances that some of these some of these all their frameworks depended on how threats were handled in python before there was support for a async ion for async and await and the way that they work it doesn't support like having all this performance and a concurrency that these new tools allow because they were storing some state and so for example a session each one per thread but now with this async and away tools you can handle more than one request per thread so there's conflicts in those but that's for only a few of uh of the of the orms that you can find out there because if you can go for the main ones then you're probably gonna be fine great uh then there is a question about uh how how did you generate the documentations for rest endpoint did you integrate swagger like implementation into fast api yeah it's actually very simple so fast api generates the open api schema which is just a big json describing all the open api and then there's just like one extra endpoint that uses swagger ui directly and uses it like yeah just the struggle ui javascript inside of a small html that reads that same open api so it's like the magic of using these standards and making the whole thing based on open standards that we can benefit from from all from each other and like get all the all the like yeah all these benefits of open source but yeah like the documentation is actually built by sugary and fast api comes with redux as well which is another alternative uh if you want to use that but yeah it's like very very easy to add these extra layers because they are all compatible because they are based on these standards okay great uh there are many questions and uh it's very difficult for me to filter but like one of the questions is um it's about platform so what would you suggest as a platform for deploying fast api and serving fast api model and serving models so um yeah is like sorry um just the second part of the question is is aws elastic uh inference or sagemaker good option i haven't tried directly sagemaker or or or elastic like i think i have done stuff with aws with like pure ec2 instances but i don't know if i have used like those extra services uh but yeah like you can so the thing is that these all these cloud tools like amazon google azure well like microsoft they all have like their apis for doing inference of machine learning with their own models and all the stuff so if you are using that then you can just deploy fast api whatever you want and just call those apis because you are not actually running them all directly if you are running the models then you're probably gonna want something that a that is not for example that that doesn't die very frequently so you wouldn't probably want to use lambdas or these functions as a service because they will take a lot of time to just start the server while they are loading the model which is like a big binary file so they will take a lot of time during the startup and then that will die right away and then it's going to take a lot of time again for the next request unless you have like a lot of requests and they keep it like they like each one of the instances live but yeah it's something that you can just like try and see how it works i'm personally a fan of docker and containers so that's what i would suggest there's an official docker image that you can use that has like everything configured so that you can have on one side the concurrency and on the other side the parallel listing so it's gonna it uses ubicorn the server that we were using that is managed through unicorn which is a yeah another tool that is going to spawn several processes and it's going to span the amount of processes depending on the cpus and the cores that you have available on the server and it's going to compute that and start like the amount of processes and everything depending on that and like uh like autotune automatically so so so you can you just have to copy your code to the docker image and you're done and it just works right away i think that's a very easy way to to do it and then you can add traffic on top uh and traffic can handle https for you and you're done that's like quite easy it's also there's also documentation for that in the in the first api's site but yeah you can also do like a bunch of other things there are tools to run fast api as in this function services like lambdas or these things it's just that for machine learning i think it might not be the best idea but yeah you can actually even run them as if they were as if it was a function framework awesome uh so let's take three more questions one related to fast ai and two non-fast ai questions so the one related to fast as uh um how do we go about integrating features into fast api what's your collaboration or contribution policies so like all the contributions are are very welcome uh and you just have to like go to the there's a lot of documentation on how to contribute how to set up your environment how to check stuff like for example there's an ongoing effort to translate the documentation of usda to several languages there's a bunch of people translating to spanish there's a bunch of people translating to russian japanese chinese there's there's a very fast team translating to japanese there's a bunch of languages uh that's a very easy way to contribute and to help your local community if you speak like another language that will be like a very simple way to contribute and there's also like a bunch of things that can be that can be done and a yeah like of course it's something for work for contributions that's for open source it's uh like contributions are everything yeah exactly and uh so our last couple of questions have taken a lot of time from you today uh but this is very interest these are very interesting questions so one question is what is the story behind the name the angola the name the angular i i think no one had asked that question before so my name is sebastian my family used to call me sebastian then just the last part tian instead of the whole sebastian just says and then tien got like some appendage because they are kind of crazy so it was tiango and then it was tiango lo just because they kept adding like sounds to that so he ended up being tiangoro and they have been calling me the angelo since i was a kid and it was like yeah well that seems like as a username and now like i cannot change it okay great uh and one one final question i mean uh this question got a lot of words and uh even i wanted to ask that so do you have any expert trips for growing and maintaining a moustache expert tips for growing a mustache so yeah you have to have a lot of uh from your nose you need to extract no it's just like hair gel okay great thank you very much sebastian uh it was a great talk and uh if you guys uh are not following him so go and follow him follow him on twitter uh you know the username tangolo and uh also star the repo if you haven't done it yet and uh thanks a lot for your time sebastian and i hope it was good for you it was very interesting presentation and live coding was amazing so uh hope to see you soon thank you very much thank you for the invitation and thanks everyone for assisting and for bearing with me while i told you i

Channel: Abhishek Thakur

Views: 15,969

Rating: 4.9824047 out of 5

Keywords: fastapi

Id: 1zMQBe0l1bM

Length: 75min 10sec (4510 seconds)

Published: Fri Jul 31 2020