Deploy LLM App as API Using Langserve Langchain

hello all my name is krishak and welcome to my YouTube channel so guys yet another amazing video for all of you and this is something related to creating apis for your llm app in an efficient way using an amazing module that is available in Lang chain which is called as Lang serve now this tutorials that I'm actually creating it will be super beneficial for you let's say that you are already working on a software product and you want to integrate any generative AI application into a software product then creating apis for those generative AI module will be super beneficial so in this video you will completely get an idea that how easily you can probably create this kind of apis and integrate with any independent software product also as many of the companies are doing the specific work now again thanks to langen for coming for coming with this amazing model which is called as langa and as we go ahead langin will also be making sure that they're coming up with oneclick deployments and for that also you really need to learn Lang serve so please make sure that you watch this video till the end uh we will keep the like Target to, uh guys understand all these technical videos will actually help you to clear interviews will help you to work in amazing software companies in amazing AI companies itself and just by seeing if you're following me in LinkedIn you can probably see a lot of transition stories that I posted recently people are able to clear jobs in generative AI fields and all I want to make make sure that all the recent updated things that are probably coming up in the field of generative AI I should be uploading videos related to that in my channel so let's go ahead and let's start this particular practical implemented video and then we will go ahead step by step we'll start the implementation so let me go ahead and share my screen so this is probably if you're following Lang chain recently I made a video with respect to Lang Smith you know why Lang Smith was specifically used we had understood if you don't know about Langs Smith it will basically help you to uh create an application in such a way that you'll be able to monitor that entire application debug it test it and do multiple things so let's go ahead with respect to Langs Sur so here you can see Lang surf helps developer deploy Lang chain runnable and chains as rest API now the best part of this is that just by using this Lang serve module you'll be able to create all the necessary things in the form of apis and later on you can also create a client code where you will be able to access those apis so that both the things I will be showing in this specific video okay so let me just open this and let me show you that how things will be going ahead but here you have lot of options with respect to this you'll be able to capture logs you'll be able to capture uh invoke streams you'll be able to call multiple apis multiple different apis it is just not generic to open AI any kind of apis that you specifically want to work with or any llm models you want to work with you'll be able to work with this itself so that is the reason I say Lang chain is a common framework and it is an amazing framework because it is compatible to almost almost all the llm models out there okay so let me quickly go ahead and open my uh coding file over here and now the first thing what I'm actually going to do is that I have updated my file uh and in this file um you know I have made sure that I have also added my API key okay now with respect to this particular API I key I have this in requirement. txt you have all these libraries that I'm going to specifically use then here you also have Lang chain and the best thing is that we also going to use fast API so if I go ahead and show you the documentation Lang uh Lang Ser this library is integrated with fast API and uses piden for data validation okay so this is a library amazing library in Python which will actually help you to perform data validation in addition it also provides a client that can be used to call it into a call the API on the server itself okay so for this a JavaScript client is also available in langen .js but we'll try to focus on the python part okay so the requirement. txt is there everything is there now let's go ahead and start the coding um as usual what I'm actually going to do uh first of all I'm going to import from Fast API uh I'm going to import the fast API I've already done the installation of all the requirement. txt then I'm going to use langin from langin do prompts I'm going to use something called as chat prompt template okay chat prompt template so that I'll be able to create my prompts then from langu Lang chain dot chat models I'm going to specifically use something called as chat open AI chat ch8 chat models chat open AI so let me just copy it from here from the documentation I'm just seeing the documentation because I really need refer so many libraries over there right then from Lang serve I am and just by using this example you can create any kind of applications that you specifically want okay I'm going to import something called as ADD routes now this add routes will be super beneficial because in the form of apis if you think right in in flask also right the number of routes that I specifically add that many number of uh apis gets created right so uh from Lang chin we going to import that ad routes along with this we are also going to use uh import uvon okay just let me know in the comment section what exactly is the use of uvon also you'll be able to I hope many people will be knowing because I've already explained this in all my previous tutorials where I've used Fast API and all so import OS and then what I will do I will sa os. um what I'll do is that uh I will go ahead and set up my API key so os. environment okay and this will basically be my open AI _ API uncore key okay so this open API key I'm going to use it and here I'm will write OS do get EnV and I will call this for my environment variable open AI andore API _ ke okay so here any kind of API Keys you can call you can also use open source llm models so right now I'm just going to use openi API key and try to use the openi model itself now in The Next Step what I'm actually going to do is that I'm going to create my app and this app will be a kind of the fast API app three information is basically required in this one is the title so I will say uh in the title I will probably say this is my Lang chain server after this I will set up my version number so here I'm going to write version is equal 1.0 okay and one more parameter let it Let me give it over here like description here I'm going to say um description I'll say a simple API server okay so this becomes my API server over here and this is it okay so this is my app a fast API app now the next thing is that I have to make sure that I create apis okay now with respect to open AI key like what for what all purposes I will probably use I me create another route for let's say Pro providing me jokes or I may also create another route uh I may tell the chat uh open open a means chat open AI to probably create jokes or text summarization different different functionalities I'll make sure that I'll add some kind of routes over here so here what I'm going to do I'll say add routes and inside this routes I'm going to first of all use my app then I have my chat open a and along with this what I'm actually going to do I'm also going to create my path and with respect to this particular path slop aai okay so this is basically my path for the chat open a I've initialized chat open AI over here okay so this is one kind of functionality I'm probably giving it okay so this route is responsible of taking care of this open AI part and this is the model that I specifically have okay now what I will do I will also go ahead and create my model so here I'll say chat open AI okay now this is just one route what I will do I will make sure that I will also add some more routes okay so here I'm going to write add another route okay and inside this route again I'm going to give my app I'm going to also make sure that let's let's give one prompt so just by giving one prompt you'll be able to understand what I'm actually saying okay uh so here I'm going to write prompt so this will be my first uh first API functionality just imagine in that way so this will be my prompt and here I'm going to write chat prompt template from template uh from template and here I will just say tell me a uh tell me a or provide me an essay provide me an essay for some functionality or I will say about whatever will be my topic name that will be coming over here so I'm just saying that hey this is one of the functionality that you really need to do whatever topic I give over here you need to provide me an essay okay so this becomes my first prompt now let's say that I'll give another prompt one more prompt I'll say Okay so this will be my prompt one and I say hey provide me a uh a poem about okay and whatever topics is there so I will use this prompt in another route and this prompt in another route so that I have multiple services in the form of API so now in the next app in the next add route what I will do I will add my prompt over here and I will combine it with my model okay with my model of chat open then I also need to give the path so here I'll write path is equal to uh let's say I will say this is my essay okay so this is becoming my essay now one more route I want to probably create over here and this time instead of writing prompt I will write prompt one prompt one/ model and this time it will be/ poem okay so two apis I've have created One is/ S one is / poem and one is about the generic open AI okay now all these things will be getting listed in this fast API that is the most powerful thing right you can actually create the llm app anyhow but the main thing is that how you able to create this apis right so once you have added all the routes now what I will do I will go ahead and write if uncore name underscore uncore W equal to underscore uncore maincore uncore then I will go ahead and write uvon do run right and then I will write app is equal to host is equal to let's say host I will say local host or 127.0.0.1 and then the port that I'm actually going to use is 8,000 okay now see see the magic as soon as I run what will happen Okay so I let me open my terminal and this terminal over here I will open my command prompt okay now let's go ahead and run this I'm already inside my V EnV environment I will go ahead and write python app.py okay uh okay I'm not getting any error so see all the things are coming see Lang serve playround for slay for slop SL poem everything is visible now let me open the URL oops let me copy this okay now now see over here uh you'll be able to see that right now I'm getting details not found okay so in order to access all the apis I have to just write SL docs because it is using that entire uh functionality with respect to the fast API as I as I said penic Library also it uses right so over here you can see I have this open a API key open aai config okay if you go ahead and see each and everything what is the input schema right uh what will be the output schema what it returns everything you you will be able to see over here now along with this you'll also be able to see two more functionalities see Slash essay right and slash poem right so both of them are there see SLS a/ poem now over here the main thing is that you'll be able to see okay in the essay what kind of input is required input is a string okay output will again be a string because I'll be saying that hey write me a essay about this particular topic right so all this information that you are able to see just by writing some lines of code you'll be able to see the entire things over here but still the question Rises Krish how do I hit this specific API right and that is what I will be writing the code with respect to my client end now understand this in this particular URL already this apis are running okay now what I'm going to do over here is that I will go ahead and create one more file and this file will be my client.py okay just to give you an idea how I'm going to hit this particular files okay or hit this apis okay and uh you know it it it's just like a client SDK okay now over here what I'm going to do is that I'm going to write import requests and as I know the URL where it is probably running right so here let me go ahead and create my response and here let me write request dot post it'll be a post request and here the first thing that I'm actually going to do is call my URL right and whatever Services I want let's say over here I'm saying essay okay now just imagine over here what I'm using over here guys this is My URL where my API is running and if I use slash invoke right then it is going to call that particular API okay understand this this is a important important important things because tomorrow if you're going in any companies this is the best way that how you can actually create an API so this is my API URL okay right slash invoke I just need to do SL invoke now the next thing that I actually require is my Json okay so here I will write Json and I will create my key values right so my input will be nothing but the name that I have actually created is topic and topic wise what I will do okay let me just see what mistake I'm actually doing okay I have to close this okay input colon topic is equal to and here I will just give my topic name as okay see guys I'm also seeing the documentation and based on that I'm just writing it let's say the documentation um there another example is given so I'm trying to just make sure that I I do some of the changes over here and that is the reason and it's okay to see the code and all I don't need to even byad it so topic is equal to uh since I really need to uh write an essay I'll say my best friend Okay so so this will basically be my topic over here um some errors okay I will close this yeah this looks good okay input topic my best friend okay and then I've closed this over here right so that basically means whatever thing I will be calling over here write me an essay about I have to call this SL essay that is the reason we have written slash essay SL invoke and here it is expecting a topic right and we are giving the topic over here that is nothing but my best R now after calling this particular post request I will go ahead and write print response. Json and here I'm going to basically let let me just go ahead and print this response. Json now I will not close this this is already running in the background uh in the form of API I will open another command prompt and let me go ahead and executed it from here so now I'm going to write python client.py and let's see whether we'll be able to get it or not okay perfect see the entire response is coming output is over here content is over here my best friend is something over here so let me directly call this message okay so for calling this message I know it is inside my output key output key and inside the output you have the content and key okay now I will go ahead and call it now let's Print it Now directly I should be getting the message no need to give get all this information you know I just need to get the message so here you can probably see I have my my best friend friendship is a special bond and all the information I'm able to get right so if you are following my entire langin series and probably developing multip application I think you should have understood the important purpose of this but at the end of the day you are creating amazing apis uh from an entire llm app in a very simplistic manner so I hope you like this particular video uh this was it for my side uh I'll see you all in the next video have a great day thank you and all take care bye-bye

Channel: Krish Naik

Views: 20,447

Keywords: yt:cc=on, langserve tutorials, langchain tutorials, generative ai tutorials, krish naik gen ai tutorials

Id: FgTYqJVNdkU

Length: 17min 48sec (1068 seconds)

Published: Thu Mar 21 2024