Deploy LangChain apps in 5 minutes with FastAPI and Vercel

hi everyone I'm Patrick from assembly Ai and in today's video I show you how you can easily deploy your land chain applications with a very cool python package that uses fast API under the hood and in just a few minutes you can deploy this with a production ready server wherever you want in this video I'm going to use versel where we can host our API for free so if you followed this video in the end you will have a live deployed Lang chain application so let's get started so for this we are going to use an open source package that is called lancorn and lancorn allows you to serve Lang chain apps automatically with fast API and you don't have to worry about writing the backend code yourself so before we set this up together and by the way this is super easy it just takes a few minutes let me go over the features very quickly so as I said it uses fast API under the hoods then it easily allows you to deploy those apps it even provides all functionality then you have the high performance fast API AI framework you even have well documented restful API endpoints you can use asynchronous processing for faster response times and this is all thanks to fast API and the way you use it is you can pip install lancorn then you write your normal Lang chain applications and don't have to worry about the backend code and then for example you can start a server with this command or you can set this up in your code by saying from lancorn import crate service and then create your services that applies your chains here and then you can start your uvicorn server with this command and this is all you gotta do and it's no magic behind the scenes so the code is not very long actually the main important code is this API dot Pi file and here for example you can see it defines some data types for the requests and the response model us and then the important function is the create service function and here for every service and every chain you define it finds the input and output fields and then for each chain it creates a post endpoint with the endpoint prefix and then the slash run endpoint so this is how it works behind the scenes and now let's set this up together so here I already prepared two Lang chain scripts and the first one is a simple example with an llm chain that takes a prompt template and then applies a open AI model and the second one is a more complex example where we use a conversation chain and we also use memory so this also works and now of course you can also run these scripts locally but now we want to host this and for this we create one new file and let's call This main.pi And in here we say from lancorn we import create service and then we Define our app and here we call create service and now as a string we use the name of the file so this is llm llm chain and then a colon and then we need to use the name of this variable that we use for our chain and this will then be run in the end point so here we say colon chain and then we can Define multiple ones so let's also use the second one so this is called conversation chain and then colon and here we called the variable conversation so we need to use this as well and now we can run this so this is all the code you need to write so in order to run this you can now use this command UV corn main colon app and Main refers to the name of your main.pi file and the app is this app instance and then you also Define the host and the port and now you can see we have this running so let's open this on the Home Route you will get details not found because this endpoint is not defined but you can go to slash docs and this is a cool feature from Fast API that you automatically get those documented API endpoints and here you can see for each chain we get this slash run and point which is a post endpoint we also have a health check and then we can click on this and here you find information about the input schema for example so as you can see we need to pass in a string that will be the product and it gets those information from the input variables from The Prompt template so lancorn automatically gets the input and output fields for you so let's try this out we can click on here try this out and then for example for the product fact we want a ice cream as a product and then you can click on execute and now as you can see the response code is 200 and we get the output and then of course you can also um try this with a curl command or Implement a front end so whatever you want so now let's deploy this to versl and the pointer of a cell is also pretty easy so for this we are going to use the python runtime with serverless functions and this also lets you for example deploy Chango or flask apps here we have a fast API app so for this let's create a new directory and I call this API and now let's move all the files into this new folder and then we need to make a small change we need to start all our service names with API dots and the same for the second one and now let's save this and now in the root directory we need to create a require men's dot txt file and the only requirement here is lancorn this will then also install link chain and fast API and uvicorn and then we need to create a ver cell dot Json file and for this we can find the code in the Lang corn repository here you will see the Versa Json file so let's copy all of this and paste this in here and then let's change this so that it will correspond to our own naming scheme so we call this um Main and then app and the source will be in API and then main dot pi and the same down here so let's replace this and this is all you need to do and now you need the Versailles CLI so you can can install this with npn I minus Chi versus Cell and then you can simply run verse cell and then it will ask you to log in so I always log in with GitHub and now let's hit enter and then you will go through the process of setting this up and deploying this so I simply hit enter to keep the default settings to enter and now it will deploy the app and you see it is deployed so this is the link where our API is now deployed and we can open this and then again the Home Route is not defined but if we go to slash docs then you see all our endpoints are live and before we try this we also have to set the open AI API key so on my local system I have this exported as a environment variable and now we have to also do this in our project settings so in your Versa dashboard go to settings and then you find Environment variables and then here is a key type open AI underscore API underscore key and then enter your value I just did this and refresh this and now for example we can try the end point and then send again let's send ice cream and then click on execute and now you see we get a response so this is working and here you can see the full endpoint where you can send your post requests to and now you have a live deployed length chain application with lancorn and fast API so yeah I hope you enjoyed this tutorial if so then drop us a like and consider subscribing to our Channel and then I hope to see you next time bye

Channel: AssemblyAI

Views: 14,424

Keywords:

Id: iFvCZD4iS2w

Length: 8min 39sec (519 seconds)

Published: Thu Jul 13 2023