LangChain Beginner's Tutorial for Typescript/Javascript

all right this is Val and this is a brief tutorial from chat with data YouTube channel on how to use Lang chain to build complex simple AI applications for JavaScript on JavaScript or typescript so this is a JavaScript typescript beginner's tutorial for line chain you kind of think of Lang chain as a framework that makes it easier to build uh these large language model applications so let's jump into it first of all what is Lang chain so to understand what Langston is we have to understand the problems around building applications so if you've ever tried to build one of these AI apps that we see popping all over the place where you know someone plugs into the open AI API and you've got you know apps allow you to write you know create images and do all these wonderful things but particularly when it comes to text you know that there's a lot of challenges so one of them for example is the issues of scalability so what do I mean well if you jump into the playground for example we can see that yeah it's cool we can effectively uh you know the because the playground is assimilation of when we use the API so I could come here and say you know write a tagline for my business click submit and that's going to pop out you know something right now but when we jump into looking at the max length of basically how much can we send to the large language model per request we're looking around 4 000 tokens that's more or less I think three quarters of a uh three quarters of a token is is a word right so you're looking about 3 800 words to squeeze in per request and that's a combination of your prompt so your prompt is the instruction you'll give it a large language model and the response have to combine together to um more or less 3 800 words and if you don't meet that then you need to start to do things like chunking and splitting your text because let's say for example you wanted to summarize a 10 000 word uh PDF or ten thousand word um book you know it's 20 000 or you wanted to extract insights right now you've run into issues where you need to split the text and then how much you split the tax by and then how do you make sure you don't lose context and so on and so forth so if you've ever tried to build or you can imagine that very frustrating and very difficult to deal with the second issue is you are probably going to get relevant of misleading information without proper context what I mean is when it comes to cost of data data that relevant to your business data that's relevant to your work or something specific you're looking for that's real time on the web these large arguments models are not going to necessarily give you the best result here for example this is me earlier try to look at this uh particular method or of putting together a chat bot using Lang trade and this is the response to chat gbt he gave me was effectively is a natural language processing model and Lang chain is a blockchain which is not true now it's talking about smart contracts and all these things that have nothing to do with Lang chain so if you've ever tried to use chat ubt for specific things of your work or custom data or specific knowledge you've probably been frustrated to to see that it's not really tailored towards you it's just very general it's very good at General things and that's what they were made for so that we can quote unquote uh fine-tune it for our specific use third pain point is around crafting you know complex prompt workflows so you can imagine this is very basic but imagine if you had a situation where you wanted to describe you know something a bit more where you you provide in some sort of an identity uh you say you know uh let's imagine your a business owner based in New York um you are you have a 20 years experience you know build in businesses what you'd want to expand businesses in in retail what you want to expand to fashion and blah blah blah blah right you can imagine like a prompt where you give you know identity and then after you give an identity you probably want to give like instructions of of what not to do or what not to say and then you probably want to give you know examples and you want to give context right so there's a lot that you can feed them the model ahead of time to get a more precise result the problem is as I said earlier it doesn't scale because you have I mean it's difficult to to do all of that precisely within the same chunk but the other issue is it's just very complex to manage because imagine all these different things are inputs from users right so you could have you know this being a variable input you could have this being a variable input then you have to deal with testing and log in and managing your tokens and your costs and all these things that are you know problems as well that's another issue then pre-processing and formatting on structured data which you know the large model is going to best understand text um the mod uh the the stores which I'm going to talk about in a second where we put the text uh in number format so that we can retrieve also prefers this kind of text clear text format so if we're coming from the world of PDFs images we need to pre-process we need to even with text we need to chunk it up we need to make it in a way where it's easier and it's efficient to work with our data and then the third and then the final thing is the cost of apis right as demand Rises the cost of the API increases as well because of inefficiencies and all these other areas so now we understand what the problem is let's see how long chain can help to solve so as I said it's building AI is powered by custom data Made Easy I mean that's the most simple way I can explain it so we're looking at use cases question and answering documents custom documents track PPT for your data like I said you know you have that chat experience but it's tailored for your data and there you've got personal assistance to use external tools for example you can have be able to chain together instructions that you know effectively a bot is able to go and search a real-time data on Google and that may be use openai to generate some sort of instructions or response and then take that response and plug into slack or maybe book your calendar for you right so there's so many use cases they got summarization of large documents so you can kind of think of Lang chain as a library or framework to coordinate all these different use cases and all these different complexities that we spoke about earlier that were problems so these are the components of Lang chain so as I said you got your prompts or your prompted the instructions to the large language model as to what to do uh then you've got the the large language model itself you've got memory the ability to memorize previous conversation you've got indexes where you store the data document loaders how you convert the docs to text you got agents which is where integration to external tools comes in you got chains basically chaining together all these different components where the name Lang chain comes from so what does this all look like you know for example let's jump into their docs right so now this is the Lang chain uh docs for JavaScript and typescript which we're going to focus on in this tutorial and what you can see up here is a um a search bar right so if I jumped in here and I said okay how can I help you and I say well uh I want uh how do I uh create a chain for a chop bot for my documents okay and here we go you can use LINE chain to create one of them is conversation chain which is pre-built and it gives you the code and it gives you verified sources to check it out so you can see these are actual verified sources from within this website itself that's basically what is capable uh what we're capable of doing now is basically you turn your docs your website your newsletter your application your database into um this format that we can have a chat or conversation with so this is basically what the power is and you can see how this is way more precise than you know something like this Okay cool so let's jump into the code right so we're going to start off first of all with a simple chain so simple chain is effectively where we have an input from a user or just a general import a query just like we would with chat gbt and the large language model like open AI will take that query process the query return a response so here's the code base for second let me just clear that and so you can see the structure here so we installed the library Lang trade which you can do um in by just simply doing npm install line chain I'll put all the instructions in the GitHub uh repo after this uh after this tutorial but effectively all we're doing is just like you typically would you you know you would uh initiate the model uh put in the temperature or whatever the parameters you want to you want to use and then you get the response right this is just the syntax using openai so in this case I'm just simply going to go npm run stop and I'm going to go here we're just going to do a basic call Dot TS so I'm going to run through this um I already have my environment variables my openai key is already in play and we're running and we should see Paris okay so we got the response back from the large language model that the capital city of France is Paris okay cool so that's a basic call now let's move over to another example I'm going to go through two examples but let's start with this one where effectively we have a user and instead of just simply saying what is the capital city in France where I'm the one who's going to hard code this time we're going to make France variable so it's going to be variable to what the user inputs and this is not any different from a typical AI application if you built one you're always looking for a way to have your prompts which is your instructions but then you have pockets of those instructions that you want the user to be able to input which customizes and makes it more are tailored towards them so the user is going to give an contribute here and be able to change the country then we effectively create this chain with the large language model who then output so it's basically identical to this except the user we have these variable this variable here we can call this a prompt template and we can call this the chain right so let's hop back to the code and we're going to go to prompts so Lang chain has these prompt templates as I explained to you earlier that are effectively just these uh just some more customizable format of what we would have to type out manually so we wouldn't normally we would have to in this case here imagine it being more complex but this is the equivalent of saying you know um so variable text and then you would have your what is the capital city of country okay and so where country is effectively the variable Dynamic variable that is going to be passed in from a user's inputs that can come from the front end or it could be a request to the back end however you want to do it this is how it would typically look like so that's the equivalent of what's going on here except you're passing this template in to this object and you have this country here as what is variable right and now we can pass in the country so in this case we're going to pass in France and if we run through this you'll see that uh let's just do that second so that's npm Run start I'm going to upload all this to GitHub so you can play around with it let's see what happens now so this format what it does is it formats The Prompt given the input values right so if I change this to to Scotland or Spain then it would be what is capital city of Scotland respect okay so effectively this just shows you the the format of the prompt and you can see how you know this when things get a bit more complicated this would be very useful so let's jump to the few shot case so what's a few short prompt well a few shots is just basically examples you provide to the large language model to help it generate a better response so for example let's say you wanted to um you know you wanted to give context on different countries and capitals so that you want the model to have a pattern to use in this response so you're still going to ask it the same question then you provide these huge shots or these examples to help it have a better response so in this case we're providing these examples to the United States Washington DC Canada Ottawa and then we have a formatted template which is effectively what the uh the template is that we're going to pass over and then we have the example prompt so again you can see these are variables that are going to change right because uh they're going to come from these examples here and this is the template that we created here so remember this is the same as what we had here right with this prompt template that basically just outputed something like this basically what is the capital city of of France right once we formatted it or saw the format so we're doing the same thing here we're making this thing here so that we can input the country on the capital and when we console.log we get country United States capital Washington DC obviously I'm formatting using these examples here so you can see how it effectively just inputs into those gaps there and now this creates an example that we can use so finally we create an object so this is a few short template object um that we create instance from that line chain provides and these are you can read through this later but basically you can provide prefix or what text you want to go before what text you want to go after um and so effectively you will end up with something like this where you have you can see what is the capital city of the country below so this is what the actual text will look like so let's hop over to maybe a format that will make more sense to you and this is what it would effectively look something like this right what is the capital city of the country below and so in this case we are providing uh these things called diffuse shot examples as I said they're just examples to help the model but he knows anyway he wants to seize the pattern of country capital country capital you provided France as a country and now if I just press submit it should uh say Paris okay so this is how you use few short prompts by basically providing your question or your prompt and then the few shots but in the case of JavaScript and line chain you can use these few short prompt templates which effectively take your examples which you provided earlier um the example prompt the prefix the suffix and help you to effectively scale this because this can this can get very very tedious that's a few shots and props okay cool so let's move over to chains so this is a basic chain now if I hop back over or rather actually let's let's start with agents because that's that's a bit more exciting and also a bit more a bit more requires a bit more thinking about so what is an agent so an agent you can kind of think of as a personal assistant that uses the large language model to help you take actions and it can help you either observe or use external tools so what do I mean okay so let's start from here so we have the user and the user says okay how many countries are there in Africa to the power of three I know this is a a crazy example but just just kind of stick with me here so they send this question to open AI for example the model and the model basically speaks to this agent or this bot and the bot thinks to itself well I need to find each country in Africa and add them together so that's the first thought that this bot has now it then goes and uses something like a serp API which is a real-time API to access Google search results performs a search action to find all these African countries based on live results and then maybe finds an article that gives you know credible article the fight that concludes the observation that there are 54 countries in Africa then the bot says well okay now I need to calculate the total number of countries in Africa to the third power so that's what he's thinking and then it uses another tool to do calculations there's times they can calculate action it observes this number it thinks itself now I know the final answer and this is the final output that is you know this number is the number of African countries to power through okay so there's different components here there's user there's a model there's the agent there's thinking there's tools there's searching there's actions there's observations and then there's the final output but in essence all it is effectively is a chain of a sequence of interacting with the large language model for instructions and then interacting with third-party tools to get whatever outcome you're looking for so let's jump into the code okay so I'm not going to run this one here particularly because you can just um get a sharp API free sub API API key that you can test this with but effectively again we initiate the large language model we initiate the tools so we got the calculator and serp API so Lang chain provides tools that you can use there are many many more so if I just hit here you've got Bing dad jokes IFTTT where you can connect to third-party tools and so on and then you've got agents here now this agent is going to take tools the model and this framework you can change this and read up on it it's just a framework that kind of goes through this kind of pattern of thinking that we described where it has a thought and then a follow-up observation and so on and so forth then you have the input what are the total number of countries in Africa raised to the power of three executes and calls and this is kind of what the final result looks like right so there's a lot of use cases that we can think about here um besides search you know interacting with your calendars and you know going to do maths and then taking that data and interact with other tools but uh the use cases are immense when it comes to agents Lang chain and external tools all right let us jump into memory okay so what's memory so memory is just basically given the chatbot the ability to remember previous conversations if you remember one of the big challenges is when you're interacting with these chat Bots or you try and make your own chatbot they don't have extensive memory and sometimes no memory so imagine someone you ask a question and then you ask a follow-up question but it doesn't have contacts the response is not going to be great so what we want is to give the ability to take previous conversations into context for future conversations but to do it in a pain-free way a line chain gives us the ability to do that so here we are effectively uh where we're saying you know we initiate this buffer memory class uh we create the new object called memory a new conversation chain again Lang chain has these components in different libraries and this library is free then you call with the input hello I'm John so this is the user base he's saying hi I'm John you get a response and so on and so forth so let's run this so you can have a feel for what's going on here so let's clear that PM Run start and I'm going to run to memory basic but yes so effectively what's going to happen is I'm going to say my name is gonna say hi I'm going to ask you what my name is and it's going to respond there we go so hi John I'm an AI nice to me can I ask you something what do you do for a living and here I go what's my name and the chatbot says your name is John so you remember that my name is John and I try to just put this in diagram by the way this image will be available for you to use and review after this video as well but like I said gives the chain or agent the ability to remember information from previous conversations so you got the input from the user previous chat context and then that is packaged into an instruction or a prompt in a structured way the line chain helps you to do behind the scenes you pass that over to the model and you get a final answer so what's pretty cool about Lang chain is all of this is abstracted and you don't have to worry about figuring all this stuff out yourself and how to structure the problems or what prompts you use and so much so much complexity going on under the scenes especially in terms of structure and has been handled for you from the library so let's talk about indexes so before we jump into indexes it's important to understand the concept of embeddings and what is an embedding I'm going to do another video on embeddings and how to work with effectively the language of the compute the computer or the language of the large language models and when it comes to interacting with documents because that's really the goal behind this channel is how do we you know build applications that help to interact with data so one of the things is that these models are very much focused on numbers right they don't necessarily understand text per se in the sense that they usually under the hood convert to numbers that they can use and this is particularly important when it comes to embeddings when when it comes to q a with document chatbots because effectively what you're doing is you are converting your the query or the user query into text form into your number of vectors like this that you pass over and store in a place called a vector Store where the vector store already contains a bunch of these for your document and so it's actually matching these numbers against the existing numbers to then decide which ones are similar it gives a similarity score using things like cosine similarity which I'm not going to get into right now but in a nutshell embeddings is just converting text into numbers in a way that large large language models can easily talk to and we can easily do analysis to compare find similarities and the keyword analysis and so on and so forth so that's pretty much you can think of an index as a collection of this embeddings these Vector stores and the process of getting to that point so here we can imagine we have your PDF that we load the docs into text right and then we take that text we may split it into chunks because it's too big as we spoke about but what's cool is Lang chain handles this for us create these embeddings using an embedding function so we open AI provides an embedding function that we can use to effectively convert all this text here into these numbers that we can then store and we can then retrieve later down the line so let's jump into code and see what that looks like so here we have embeddings and like I said you can grab open AI and here we are creating a function where we are embedding the query hello world and we get a response and we also embedding documents so document from Lang Chain's point of view is simply just text and metadata combined together so what I mean by metadata is just in just description or something useful to know what about that particular text they can use to retrieve later on so let's clear this out let's npm run start and we want to run here which is uh indexes okay indexes Dot embeddings.ts So This Is Us creating embeddings using open ai's embeddings function you can see like I said land chain provides that I just want to show you what it looks like in real time so you can see that we've created these numbers as I said so literally this these few uh these few words have created 1000 each Vector has one thousand 536 dimensions so that's what we're dealing with here and so you can see all these numbers so you can imagine if I did a search or query what would happen is you know if you had a large document that you had already embedded we would embed the query so the query could be hello world and then we would compare the vectors the numbers to the numbers already in storage and whatever is in storage that is closest to hello world is what is is what sent back so that's how it works while go more into that in detail in a different video here we have a document loader called recursive text splitter and like I said these are just useful super useful tools that you would have had to figure out the logic yourself previously but Lang chain provides facility for that so in this case if I just clear here I run you can see this is the text and then I can just do Cuts is text later.ts by the way I want to give major credit to ideas behind this this is actually a huge folk from Lang chains example repository so I just made a few tweaks but a huge shout out to the team there for this so please make sure to check out what they're doing I'm going to leave the dogs you can join the Discord join I'm part of the Discord as well and it's just a great you know a lot of innovation going through so please um check out what they're doing support it and you know get involved as well if you can so you can see here we create these documents as I said contain page content metadata but the key Point here is you see we split this into you know um different chunks right and here we defined we wanted to be 10 characters per chunk and each overlap between chunk would be you know one character so this is what this thing called the recursive character text bitter can do for you using long chain now we have the vector stores now what happened here was I already ran through this function you can run through it yourself but what I I drawn through this function to store this text here it says State of the Union text speech so this is how it will typically look like in real time you would have your text you know already ready to go and then you would have the vector store so we would load the text load the model now we're going to use Pinecone pinecorn is a vector store so remember the diagram where we store the embeddings so we're using Pinecone there's others like chroma and leave it and thighs and there's all kinds of other ones but for this example we're using pinecart so you need to go on pycon is free you can sign up and create your own Vector base I'm going to put instructions for you to do that so I initiate that I load the file then I split as I showed you previously I'm splitting into chunks of 1000 characters then I create a document object um this is the index I created on pinecon which I'm getting I embed these documents uh running through these functions and there we go so I'm not going to run through this again because that's going to override what I have so you can run through this and see what happens obviously make sure to change this because this is mine but effectively what we're doing here is loading the document and storing it in the vector base as per the diagram now speaking of document loaders you also have you have a lot of choices here so you can you can do uh you can load you can literally scrape websites you can Hacker News IMDb IM sdb you can know documents so there's loads of documents and every day or every week you know you have people effectively making pull requests to add more but here we have an example of a tax loader so it takes a string a five part a file path string and then it loads it and here you see the example data this is an example data to see how long chain loads will text and that loader is led to this output here so I'm not going to go through that again but I'm just showing you what the output looks like when I run through it the first time clear this here cool so let's jump into so we've gone through uh uh basic chain well actually we didn't go through a basic chain so yeah this is just a basic chain I think we went through very similar to the previous example with the prompts where we're just chaining the template The Prompt template with the large language model so you can see you should be familiar with this now then we create the chain so the large language model chain which is a combination of the model and the prompt we asked what country is it France and he says Paris we can also have conversations as well using the conversation chain so let me play around this so you can start and we are going to go with conversation.ts so here we go so again we initiate open AI Lang chain provides a conversation chain which also includes prompts that you don't have to worry about yourself and then we have the chain dot call um and here you see hi I'm Jim hi Jim it's nice to meet you my name is blah blah blah blah blah how can I help you today and you have uh another response that comes through when I say what's my name so you said your name is Jim so similar to the memory case I'm just showing you how it turns into a thing called a chain summarization obviously very useful so in this case it's just a simple case of using uh loading again Lang chain provides utilities for all these things but here we're you know I just do the short text it could be longer it could be a document whatever it is you want to do you load uh in this case I'm using analyze document chain because it's a small piece of text but if I had chunks I would use a different chain again the docks are very very useful for this and you call and then you get a response like so I'm not sure there's any necessity to run through this but I think you get the gist anyway so I'm just going to pause on that so here we have a question answering so we got question answering situation where you have a document here which could be much much more than this but Rachel went to Harvard time when Stanford this is effectively what it looks like as I said it contains page content and optional meta data and I'm just going to run this so you can see what's going on you can see the call the chain call takes Docs and also um the question and it passes that to the model as well so let's see what happens now so we asked the question where did Rachel go to college and we get the response Rachel went to Harvard which the text did say Rachel went hovered cool let's jump to the final example I believe and this is where it gets very interesting now before we jump in I just want to say that you you know there are some useful utilities here what I'm trying to do here is just kind of I guess break down the magic behind it it's not too much but it's just the basics of what's going on so you embed in a query you effectively embedding using this function called embed query similarity Vector search so again this is where we want to look at what it's currently in the vector store and compared to our query and the closer the numbers are in terms of similarity scores then that's what we want to get back and so again you can just view this lib so just have a field behind the backend logic and again much credit goes to the Lang chain team for inspiration for this let's go back to a final example and it's what really the magic begins to stand out and this is where you start to q a document so what's going on here so effectively we have Pinecone we're initiating as a storage we have the uh just cross checking that the particular index that we're index be in the store that has our data again this is this index has been pre-loaded with data from that speech that I showed you earlier that was in the in the the large text and this is the query so the query is how many new jobs the economy create let's go and verify so if I do control F job created so this you can see it says uh let's see jobs in fact our economy created over 6.5 million new jobs just last year okay so you can see this is a large body of text there's a lot going on so let's jump back and let's see if we can get a useful answer from this large document so we're going to call Vector DBA chain again don't follow this syntax too closely because this is almost like a custom function that I made that I showed you previously but always effectively doing is taking the query it's taking the index which you saw earlier so it gives you the operations a typical crud operations to create read update delete the namespace so what's the name space where the vector is stored and how many results do you want to return that are similar we pass all of that we run through a vector search function and we do some other stuff nothing too crazy but here we just focused on first principles so effectively we've run that chain which includes the large language model query the index the namespace all all the things you can imagine would be necessary including prompts provided by Lang chain to structure and to try and find the relevant information so let's clear this and if I just go to question answer docs.tf so let's run this so this is a question and let's see what the response will be so hopefully fingers crossed we end up with something like that see is he gonna figure it out oh oh there we go we got the answer right here over 6.5 million jobs and what's pretty cool is now let's let's run it back let's see what's going on so what's going on you can see we have vectors here we're splitting the text um then we've got uh yeah so now we've got the the results of the similarity search which was in that pre inside this function here you can see this is you know uh each Vector has an ID score metadata which contains the text you can see that this snippet of text which was a chunk that we split initially is the most similar in terms of vectors to this question here that's why I was the first one to show up because these are the vectors of this query and these vectors right which are 1536 Dimensions where compared to the vectors in the database and if found that ID 11 was the most similar so this is the text and then ID 17 was the second most similar and so on and so forth so that's how the uh the vector similarities work and so we keep going down down and you can see just more stuff going on but eventually we land with the final uh the final response we pick the top scoring one we passed that over to you can see this is the template provided by langchain that I did not have to write use the following piece of context if you don't know the answer just say you do not know you pass that as context then we ask the question which was here and then we leave a gap for a helpful answer and voila so that's how it works in a nutshell it's not too crazy but it's very exciting very interesting and um yeah effectively everything that you would have had to abstract for yourself figure out the problems the structures how do I extract data how do I you know effectively integrate different tools with the large language model how do I scale my application how do I make it more cost effective uh line chain is really working on that so I hope this video was useful for you as a typescript JavaScript developer because there's not much uh resources out there for us to build this stuff but I believe as the ecosystem you know keeps growing it's going to get better so if you have any questions just hit me up in the comments and um that'll probably also give me good ideas for the next video cheers

Channel: Chat with data

Views: 23,627

Keywords: gpt3, langchain, openai, machine learning, artificial intelligence, natural language processing, nlp, typescript, semantic search, similarity search, gpt-3, gpt4, openai gpt3, openai gpt3 tutorial, openai embeddings, openai api, text-embedding-ada-002, new gpt3, openai sematic search, gpt 3 semantic search, chatbot, langchainchatgpt, langchainchatbot, openai question answering

Id: bH722QgRlhQ

Length: 51min 54sec (3114 seconds)

Published: Tue Feb 28 2023