Learn LangChain.js - Build LLM apps with JavaScript and OpenAI

this course will help you master Lang chain a revolutionary AI first framework Lang chain enables developers to build context aware reasoning applications by linking large language models with external data sources for advanced natural language processing applications starting with the basics of text processing and vectorization and advancing to the nuanced aspects of laying chains expression language and retrieval techniques you'll gain hands-on experience in building real world AI applications Tom chant developed this course he is an experienced developer and course creator at scrimba hey there free code campers and welcome to this interactive course where you're going to learn how to use Lang chain JS to build a context aware chatbot that can answer questions on a specific document we provid it we're starting with the basics so you don't need any prior experience with Lang chain JS or even with AI the only prerequisites for this course are a knowledge of working with apis and vanilla JavaScript now Lang chain has a reputation for a steep learning curve but using its new expression language we make the journey much easier this is a Project based course with challenges and that means you're going to get your hands on the keyboard writing code throughout and if you're wondering how you can access the project code don't worry we've got you covered from the interact active version of this course on sca.com you can pause the video edit the code and run the projects right there in your browser or you can download the code from the scrims and run them locally if you prefer the link is in the description and one final thing if you enjoyed this course do hit the thumbs up right here on YouTube and if you'd like to get in touch reach out to me I'm on Twitter or X at TP chant okay let's Dive In hang chain to build aiiowed applications I am super excited to bring you this course on using one of the most mind-blowing Technologies in the emerging AI Universe we are going to study embeddings we'll be working with Vector stores we'll be building templates and creating prompts from those templates next we'll look at setting up chains and we'll do that using Lang chains expression language which is a more expressive and accessible way to write Lang chain we'll be looking at the pipe method to join elements of a chain together we'll be retrieving data from a vector store and then we'll use the runnable sequence class which is a really cool way to create complex chains in Lang chain and there's loads more plus challenges this is our project we are building a bot which is knowledgeable on information we give it so this bot is going to know all about our platform sca.com and we're going to feed it 3,000 words of information and then we'll be able to interrogate it on this information this is one of the most powerful uses of AI and it's a process that Lang chain makes easy now in this course we're going to focus on the Lang chain syntax the project is built in vanilla JavaScript and you're welcome to refactor it into any framework or Library you wish I'm assuming you have a solid knowledge of vanilla JavaScript but you don't need to be an expert if you've worked with apis and asynchronous JavaScript a little bit before you're going to be absolutely fine my name is Tom chant and I will be your tutor for this course you can find me on Twitter or X or whatever you want to call it I am @ TP chant and you can click this link and visit me there it's always good to hear how you got on and any feedback you might have now before you begin why not head over to our Discord server and meet the community and let everybody know that you're starting this course in the today I will Channel and again this slide is a link to that page on our Discord server before we we dive into the concepts and code let's get an intro to Lang chain from the founding software engineer of Lang chain Jacob Lee Yes L chain is a framework that helps developers build context aware reasoning applications Lang chain was born out of the realization that the process of developing sophisticated AI powered apps could be significantly streamlined by factoring out some common abstractions while much of current AI app development akes place in the realm of python there's a clear need for better tooling for the more webd focused JavaScript Community too hence L chain comes in two flavors and one of the aims of Lang chain JS is to make large language models and techniques around working with them more accessible to this broader audience in this course you'll be learning how to use Lang chjs to build your own apps it focuses in on conversational retrieval over document giving an llm access to the specific information in that document so it can answer questions and continue a logical contextualized conversation on topics Beyond its original training data along the way you will use models prompts and Alpha parsers some of the basic building blocks of lank chain we'll create chains of calls that enable us to connect up the various stages in the process need to get the desired output we'll be using L chain with superbase and the open AI API but one of the beauties of this framework is that components are easily swappable so you can work with a myriad of databases Vector stores and llms switch between them and find the one which works for you best good luck and I really hope you enjoy this course and enjoy using L chain to spin up really powerful AI web apps of how this app is going to work in this first section we need to work with our data so this chatbot is going to be knowledgeable about a specific Topic in this case the scrimber platform so we're going to start off with an information source which holds the knowledge we want the chatbot to have and we're going to pass this document to a splitter and the splitter is a lang chain tool which will split the document into chunks and then we're going to use an embeddings model from open AI to create vectors from each chunk and we're going to save those chunks to our Vector store which in this case is going to be a superbase vector store when we get to this point the vector store is established it's got all of the knowledge we want it to have and we won't need to repeat this process unless we want to give it more knowledge by adding some new data now if you don't understand the justification for each step or you're not really clear on vectors and embeddings yet don't worry we're going to go into a lot more detail when we actually write the code this is just an overview now once we've got the vector store set up we need to create the app to use it and the flow of that app is going to look like this we start off with a user which we've got represented right here and that user will input something probably a question they have about scrimber and we're going to do two things with that question we're going to save it to a conversation memory store which will hold the entire conversation and we'll use an open AI model to convert it to a standalone question and that just means we're going to reduce it to a very concise question with no unnecessary words we'll then take that Standalone question and we'll use an open AI embeddings model to create vectors from it we'll send those vectors to our superbase Vector store and we'll get back the chunk or chunk with the nearest match therefore the chunks that are most likely to contain the answer to our question the last stage is to use an open AI model to get the final answer and to give it the best chance of getting a good answer we're going to give it three pieces of information we're going to bring down the nearest matches from the vector store we'll give it the original user input and we'll also give it the conversation memory which will be the entire conversation that has taken place so far so that might be a very very long conversation or this might be the first question in the conversation now we're going to take the response that we get and we'll store it in the conversation memory ready to continue the conversation and also we'll give it back to the user by rendering it to the Dom now again this is a very high level overview and it might well cause some confusion for example what actually is a standalone question and why are we using it right here but then down here we're using the original user input and not the Standalone question but rather than get caught up that right now and give very theoretical answers what I want to do is keep referring to this diagram as we write the code and we're going to tackle those questions as they come up one by one so if this diagram looks horribly confusing right now don't worry at all we are going to take it step by step okay in the next Grim let's go right back to the beginning and think about getting some data into our Vector store but before we do that I'm going to bring in a scrim from my colleague Gil who's going to give you a detailed overview of embedded ings so sit back and take a few minutes to watch that next powered search shapes many parts of your daily lives every day you interact with platforms sifting through massive amounts of data from text and images to audio and video think about Amazon recommending products or search engines refining your queries social media platforms curate tailored content while services like YouTube Netflix and Spotify offer suggestions based on your preferences now Advanced AIS despite their capabilities don't truly understand the real world as we do they can't grasp the actual meaning or Nuance of a video title song or news article so how exactly do AIS and platforms like Spotify Netflix and YouTube truly get us how is it that they appear to understand predict and respond to us as effectively as if not better than people well the magic behind this capability involves a blend of algorithms AI models and huge amounts of data but a larger part of the answer involves embeddings you see when you present a question to an AI it first needs to translate it into a format it can understand so you can think of embeddings as the language that AI understands the term embedding is a mathematical concept that refers to placing one object into a different space think of it like taking a word or sentence which is in a Content space and transforming it into a different representation like a set of numbers in a vector space all while preserving its original meaning and the relationships between other words and phrases AI systems process lots of data from user inputs to information and databases at the heart of this processing are embeddings which are vectors representing that data transforming content like search queries photos songs or videos into vectors gives machines the power to effectively compare categorize and understand the content in a way that Almost Human so how is all of this possible well it isn't exactly as easy as just turning data into vectors so before we go any deeper let's take a closer look at what vectors are think of a vector as a coordinate or point in space and to keep things simple we'll have a look at this 2D graph with an X and Y AIS let's say that a word like cat is translated into a vector like 4.5 12.2 which is this point this Vector encapsulates the meaning and nuances of the word cat in a way an AI model can understand and then we have the word feline represented by a nearby Vector of 4.7 12.6 so we'll place that point on the graph now words that have similar meanings are numerically similar and tend to be closely positioned in the vector space so this closeness implies that cat and Feline have similar meanings now let's say we have the word or vectors for kitten which might also be close to cat and Feline but maybe slightly further apart due to its age related Nuance now a dog dog is different but still in the same general domain of domesticated animals so the word dog might be represented by a vector that's not too distant but clearly in a different region let's say 7.5 10.5 and even a phrase like Man's Best Friend which is a colloquial term for a dog could be represented by a vector that's close to the vector for dog on the other hand a word like building is not related in meaning to any of these so its Vector would be much further apart let's say 15 .3 3.9 here's another example that demonstrates how embeddings might capture semantic meaning and relationships between words let's say we have the word King represented by the vector 25 then man is the vector 13 and woman is represented by the vector 14 now let's do some quick Vector arithmetic we'll start with a vector for King then subtract the vector for man to remove the male context and add the vector for woman to introduce new context after performing this Vector math our resulting Vector is 26 so we'll plot that point on the graph and let's say there's another word in our space queen represented by the vector 2 6.2 right here well this Vector is extremely close to the resulting Vector so we might identify queen as the most similar word based on that Vector just as a trained AI model would now a two-dimensional graph is a massive simplification as real world embeddings often exist in much higher dimensional spaces sometimes spanning hundreds or even thousands of dimensions for example the actual Vector embedding for the word Queen might have values across multiple Dimensions each Dimension or number in this Vector might capture a different semantic or contextual aspect of the word Queen for instance royalty Cleopatra or even chess this is what allows the AIS to recognize and differentiate between these contexts when the word is used in different scenarios now imagine embedding hundreds of thousands of words and phrases into this high-dimensional space some words will naturally gravitate closer to one another due to their similarities forming clusters While others are further apart or sparsely distributed in the space these relationships between vectors are extremely useful think back to spotify's method of embedding tracks in a vector space tracks that are positioned closely together are likely to be played one after the other all right so what else can we do with embeddings and how are they used in the real world well you can imagine how embeddings have revolutionized our daily experiences for example search engines have evolved to understand the essence of your queries and content moving beyond mere keyword matching and recommendation systems with the aid of embedding suggest products movies or songs that truly resonate with our preferences and purchase history for example Netflix uses them to create a tailored and personalized platform to maximize engagement and retention also in the healthcare industry embeddings are used to analyze medical images and extract information doctors can use to diagnose diseases and in the finance World embeddings help with analyzing financial data and making predictions about stock prices or currency exchange rates so every time you interact with an AI chatbot every time an app recommends something behind the scenes embeddings are at work translating data into meaning all right so how are these embeddings actually created well let's dive into that next that Lang chain integrates with and if you go to the Lang chain docs and check out modules retrieval Vector stores and Integrations you will see a whole long list of them and by the way this slide is a link through to that page in the docs now for this project we're going to use super base and superbase is a really popular and very userfriendly Vector store but one of the beauties of Lang chain is that it's actually really easy to swap out your vector store so if you want to experiment with various possible Integrations that won't be too much trouble at all now our first task is to set up a superbase account so so head over to superb.com and again this slide is a link to superb.com and let's select start your project and there you can sign up with your email or GitHub and once you've completed the sign up you'll end up at the dashboard so let's go to new project and we need to give the project a name I'm going to call this one scrim bot and we'll also need a database password and I'm just going to allow it to generate a random password password for me and lastly I just need to select my location I'm here in Western Europe not that far from London so that will do just fine then we can scroll down to the bottom and just click create new project now you'll wait a while while it initializes a couple of minutes at the most and eventually you're going to end up here and that shows us that we've got one database successfully set up now if you click on this tables icon on the left hand side here in the table editor tab is where you could manually set up a table but we don't need to do any of that because Lang chain is going to do it all for us if we go back to the Lang chain docs to the super base Integrations page and we just scroll down we get this chunk of code and we're going to run this in our superbase database and it's going to do everything that we need now I've just pasted this code into scrimber just so it's easier to look at and just to be clear we're not running SQL or SQL right here in scrimba we'll be running it in super base and I'll show you how in just a moment but I just wanted to put it right here so we can have a look at it in a bit more detail so what this code does is it enables a PG Vector extension in superbase it then creates our table with everything that we need in that table and it sets the embeddings to 1536 and that's an important number because the open AI embeddings model use 1536 Dimensions it also gives us this match documents function and it's this function which actually does the job of finding the the nearest match so later we'll be using this function to take the vectors from a question and find the nearest vectors from the text chunks because that will identify the text chunks which are most likely to contain the correct answer so all we need to do with this code is copy it go back to superbase and come into the SQL editor tab which is this second one down and paste it right in there now at the moment it's just called Untitled query which is not a great name so I'm just going to change mine to match documents because that's what this main function in it is called now it tells us click run to execute your query so we can do that right here and there we are it says success and if we go to the table editor and we click on documents we can see that we've got our empty table we've got the ID the content the metadata and the embedding I.E the vector and that is what we asked for right here where we said create table documents so that has worked just fine we've got this warning about allowing Anonymous access but don't worry about that right now this is going to be absolutely fine for prototyping now if you'd like to take a deeper dive into vectors and embeddings and exactly what superbase is doing here do check out this blog post that I've linked to right here it's got quite a lot of information okay so now our database is ready to go so in the next Grim let's start getting some vectors into the vector store next which is going to be the knowledge for our chatbot and split it into chunks and the idea is that each chunk will be big enough to hold a piece of information so what you want to avoid is having something like this one chunk where we say we update our course is and a second chunk saying on a regular basis that piece of information needs to be in one chunk else it's pretty much useless now a chunk will often not be that small in fact lots of our chunks are going to hold whole paragraphs so there'll be something much more like that and it doesn't matter at all if there's more than one piece of information in a chunk and in fact that's very likely going to happen all we're trying to achieve here is being able to give an AI Model A smallest chunk of text from which it can find the answer to a question what we could do is just upload a massive document with every request to say the open AI API but that would be very very expensive with tokens so what we're do here is much more economical much more performance and much more scalable okay that's the theory and this is the actual text that we're going to be using for our chatbot it's about 3,000 words long and I've checked and vetted all of the information myself that's really important if the information in the document at the start is faulty you're never going to make a good chatbot now I'm using a text file here for Simplicity but Lang chain has actually got several tools for working with different formats so you can click through to this section in the docs and it will show you options like passing PDF or extracting just the text from HTML to give you just two examples but here we're going to keep it simple and just use a text file so in index.js then I've got this TR catch we're fetching in scrimba dasinfozentrum knowledge source for our chatbot and now we need to split it and we're going to do that using a tool from Lang chain which will do most of the work for us so I've got the Lang chain dependency already installed and in scrimber we do that using a three dot menu that appears when you hover over dependencies you can't see it in the recording but when you click on that three dot menu and select ad dependency a dialog box appears and I can type in Lang chain and it does the rest for me outside of scrimba of course you can use npm install Lang chain okay now Lang chain offers us a couple of tools to split text there is the character text splitter and the recursive character text splitter I'm going to use the latter which is just a little bit more sophisticated but to be honest you can use either and I didn't see a big difference in performance between them in this app so let's import the tool from Lang chain and if you want the more basic character text splitter it's just going to look like that okay let's save a new instance of this recursive character text splitter to a const splitter and then we'll save our output we'll call splitter and use the create documents method now we need to pass create documents an array and inside the array we can list out what we want to be split we've only got one file text but if we wanted to upload to a vector store from multiple files we could list them out in here okay let's just log out the output and see what we get and I'll hit save and then in the console we can just see a promise and that's because this is an async process so we need to await it and there we are as soon as we do that we see our chunks of text and I'm just going to copy one of those chunks and bring it over to Output MD so we can have a look at it so it looks like this we've got page content and that has got the actual chunk of data now it's actually much much long longer than that the scrimba console truncates it heavily and then we've also got the metadata the metadata is quite interesting it gives us the location of that text Chunk lines 1 to 14 so what we're actually looking at if we go back to scrimba info. text is right from the beginning from 1 down to 14 so that is our text Chunk now we W be using this location data in our app but it is good to know for future reference that if you need to refer to where some information came from you have got that information right there provided to you by this text splitting tool now this text splitter that we've used the recursive character text splitter has made some assumptions about chunk size it actually defaults to a th000 characters does chunk size matter well yeah you bet it does larger chunks get more context smaller chunks get more granular semantic info if you go to either Extreme Performance could suffer and if chunks are really big sending that off to the AI models will get expensive now of course there's no best way of splitting and you do have to consider the text you're working with I've experimented a bit and I want to go for a chunk size of 500 not 1,000 so I'm going to come in here with an object and override the default settings so I'll set my chunk size to 500 again we'll hit save and what you can see is where before the first chunk went from lines 1 to 4 14 now we're going from lines 1 to 8 so the chunk is a little bit smaller and I'm just going to use Chrome Dev tools not scribus console to get the text from the first two chunks okay so there we are we've got the first text Chunk and the second text Chunk right here and what you'll notice is that these text chunks are actually split quite nicely into paragraphs this first one ends at a very natural point and this second one does as well and that is not an accident and you'll also notice we've got some overlap here this is a repetition of what we've got right here and again that is not accidental so if we just go back to this object and I can come in here and show you a couple more defaults firstly we have these separators and these separators are an array and they go in order so we've got the double new line to break a paragraph the single new line the space and then no space so what this recursive character text splitter does is actually quite complex it uses those separators to split text into chunks based on the size but prioritizing keeping paragraphs then sentences then words together intact so that's why in these text chunks we've got right here we've got more than one piece of information but that's absolutely fine because they're both contained inside a chunk and they both finish on a natural break and we've got some overlap now we can also override the overlap and it actually defaults to 200 which is quite a lot so I'm going to come in here and this is called chunk overlap and I'm going to set it to 50 so 10% of our chunk size which is a good rule of thumb to start with and you can always experiment now you won't always see overlap and that's because when the paragraphs fit neatly into the chunks and the overlap size is not big enough to include a whole sentence then the overlap won't be applied it will actually only be applied if the chunk size does not end with one of the first two separators so the double new line or the single new line so that is all a little bit confusing and I wouldn't recommend that you worry about it too much do play around with it try different strategies also try the more basic character text splitter and just see which one gets you the best performance with your chatbot and remember you can always come back and adjust it later if necessary now I should just add that if you've got some other separators in your document it's quite popular to use say the double hashtag you can also just add them to this array it does accept custom separators okay I haven't got any of them in our document so I'm going to delete that and the next thing to do then is to take these chunks of text and get them uploaded to the vector store so let's look at that in the next scrim to store and the first thing that I'm going to do is come in here and save our superbase URL and API key to consts and we can get all of the information that we need from the dashboard if you come down here to settings and select API here we have got the project URL and the API key and I've saved mine in my environment variables so I'm just going to come in here and set up the consts and of course you'll need to use whatever you called your URL and API key when you set up your M variable now I'm also going to bring in the open aai API key as we're going to need that in just a moment next I need to set up a superbase client and we do that using the create client method which we get from superbase so before we can use that I need to add the superbase dependency and there we are the superbase dependency has appeared and I'm just going to put the the name of the dependency in a comment just so you can see it a little bit more clearly okay now we've got that superbase dependency we need to import the create client method from it and now we can use create client to set up a client now let's use create client with our URL and API key to set up a superbase client in the final stage of this we do two things at once we're going to create our embedded and we're going to upload them to the vector store to do that we need two tools from nchain the superbase vector store and the open AI embeddings class so let me just import them quickly okay first we're going to use the superbase vector store class so down here I will say await because it does work asynchronously super base Vector store and we're going to use its from documents method and we need to give this method three pieces of information the first one is the output that we've got right here so remember the output is the chunks of text that we've split next we need to tell it how to create our embeddings and we're going to do that with the open AI embeddings that we've just imported so we'll set up a new instance of that class and we just need to pass it at open AI API key which we've got right here now we could pass that as a key value pair like this but as the key and the value are actually the same we don't need to do that we can use the shorthand version Okay the third piece of information is an object holding our superbase details so it needs our client and again we don't need to write out the key value pair we can just use the shorthand version it also needs a table name and if we go back to superbase we can just see that when we go to the tables tab our table name is just documents and just before we run into an error let's remember that this needs to be a string okay let's hit save and see if it works and actually we should have logged something out there with a success message but I totally forgot but now we can go back to our superbase dashboard and I'm just going to come up here to the tables Tab and let's go to documents and we can see it's worked we have got our data we've got the ID the content which is the text for each chunk the metadata and the actual embedding here are our vectors and remember this field is huge we have got 1536 Dimensions here okay now we've got our vectors in our Vector store we can start work on the app that's going to allow us to query this Vector store so when you're ready for that let's go already have over on the right hand side you can see that we've still got our dependencies from Lang chain and superbase now this event listener at the top is listening out for user input so it's basically picking up clicks on this button when a user submits a question we've also still got our API key here that we're bringing in from our environment variable and that just leaves this function progress conversation socalled because it progresses the conversation onwards and also does the heavy lifting of updating the Dom so the user can actually see what response they've got back from the AI now it's all fairly straightforward JavaScript but do feel free to pause and take a look through and also do check out the HTML and CSS if you want to again it's fairly standard stuff okay so that's the code we're starting with so where do we go next with this project well let's head back to the diagram and see where we go first so we take the users input we've got the event listener listening out for that and then we do two things we've got the conversation memory right here and the Standalone question right here I want to focus in on the Standalone question first so let's start by unraveling its Mysteries and see what it actually is and why we want it so we'll come on to that in the next scrim question and just to be clear there are two things we need to do with the users's input they're creating the Standalone question and adding the user's input to the conversation memory now we're actually going to deal with memory at the very end so let's just ignore that for the time being and concentrate in on the Standalone question so what actually is a standalone question well a standalone question is just a question reduced to the minimum number of words needed to express the request for information okay but why do we want one well we can't control control what a user asks our chatbot and actually a user could easily ask a question in a way that's likely to get a vague or even an inaccurate response and we really want to avoid that so imagine you have an online clothes shop and a user asks something like this I'm thinking of buying one of your t-shirts but I need to know what your returns policy is as some T-shirts just don't fit me and I don't want to waste money it's a perfectly reasonable question but let's remember how chatbots work we're trying to find the nearest matching vectors and therefore the chunk of text that will likely hold the answer to our question we want to create an embedding from this question and find the nearest matching vector and therefore the chunk of text that will likely hold the answer to the question in our Vector database but there's so much going on with this question that the vector for it will be polluted remember a vector represents the semantic meaning not the precise words so what we want to do to maximize the accuracy of our chatbot is just extract the intended semantic meaning from this question what we want to focus in on is I need to know what your returns policy is and we can reduce that to a standalone question that will simply be this what is your returns policy that is what the user wants to know and so that is what we need to search for in our knowledge document so that will be our first task and to do that we need to take a look at how prompts work in Lang chain and also how we send them off to the llm the large language model so let's do that in the next scrim chain to set up a simple prompt and just for demonstration purposes we're going to move away from our main project and imagine we're building an app which generates a promotional tweet for a given product we've got the Lang chain dependency already installed right here and of course we're bringing in the open AI API key next we need to to import two things from Lang chain first the chat open aai class and we also need the prompt template class so let's set up our llm and save it to a const so we'll take the chat open AI class and pass in our API key now we could instead pass in nothing and if we do that Lang chain will check process. mv. openai API key and use it if it's available so we could actually delete this line of code as well but to be honest I think it's clear if we just add it manually so I'm going to do it that way and also if we wanted to override any open AI defaults we could do that right here in this object for example we could change the temperature setting so now this llm will use a temperature of 0.5 but to be honest I don't want that I'm just going to leave everything at default for now okay now let's create a template for our tweet and I'm going to save it in a const tweet template and I'm just going to say generate a promotional tweet for a product from this product description and we need to give it a product description so I'm just going to put the product description variable inside curly braces now as soon as you see the curly braces you might think that this is a standard JavaScript template literal it's actually not there's no dollar sign needed here this input variable will be picked up by Lang chain and of course we could have several input variables here if we wanted to if this prompt was going to be a little bit more complex we could also have for example price and anything else we needed but let's just keep this really simple okay now we need to turn this template into a prompt so I'm going to set up a const for the prompt and now we can use the prompt template class and that comes with the from template method and then we just need to pass in our template and before we go any further why don't we just log out the Tweet prompt and see what we've got I'll hit save and we we're getting an error let's just check the console and it says chat open AI cannot be invoked without new okay that's a pretty clear error we needed the new keyword right here let's try that one more time and there we are we've got our prompt and just so we can look at it a little bit more easily I'm just going to bring it into a markdown file and format it okay so we can see that we've got our prompt right here just as we wrote it and the interesting thing is that we've got this input variables property and that is holding product desk so that's telling the prompt which inputs to expect and this is an array so if we had lots of input variables they would just be listed out in this array and now that the prompt is expecting them it's going to throw an error if it doesn't get them another interesting thing to notice here that the template format is an F string and an FST string is something which comes from Python and it is the kind of python equivalent to a template literal okay so that is our prompt ready to go so in the next Grim let's take a look at how we can use it to generate some content template is time to set up our first chain in Lang chain I'll set up a const tweet chain to hold it and what this chain will consist of is our prompt chained to our llm and to join the two parts of the chain together we use the pipe method and I'm going to pass in the llm so the pipe method is joining the two aspects of the chain it takes the output from the first which will be the prompt and it passes it to the llm it's a really simple chain with just one connection again let's just log that out and see what we get and again I'm just going to paste that into our markdown file okay we've got quite a lot going on here but I want to draw your attention particularly to runnable sequence because that is a theme that we're going to be coming back to quite a bit in this course but just by way of an intro what we've got here is a runnable sequence and we can see that this object right here is called First and that is the first element in our chain we've got our prompt right here with our input variables and then we've also got last which is the last section of the chain and here you can see that we have got our open AI llm or large language model now if this were a more complex chain here in the middle we would see some more steps but for now it's just enough to know that this runnable sequence exists okay so if we go back to index.js what we need to do next then is invoke this chain to start the sequence in motion and we're going to do that with a method called invoke so let's set up a response and we need to await our tweet chain because this is an asynchronous process now let's call the invoke method on the chain and we'll just log out the response now if we hit save right now what do you think is going to happen well we're getting an error down in the console it says missing value for input product desk well that figures because look we're expecting an inut variable right here product desk and we're not actually introducing a product description to the chain so what we need to do then is pass in an object right where we invoke the chain so you can think of this as passing something in right to the beginning of the chain and what we need to pass in is a product description so product desk will be our key and then the value will be whichever product we want a tweet for okay let's save that again and down in the console we have got our tweet and in fact we can Zone in specifically on the Tweet by saying response. content and there we are we've got our tweet it's even got some emoji and some hashtags so this is working okay this would be a great time just to pause make sure you understand this code maybe generate a couple of tweets perhaps make this tweet template a little bit more complex with multiple input variables and then when you're ready we'll go back to our main project where we need a prompt and a model and that will give us a great opportunity to have our first challenge so when you're ready for that just move on prompt template and chat open AI already imported we've also got our open AI API key and the llm already set up so here is your challenge I want you to create a prompt to turn a user's question into a standalone question and I've just put a hint here the AI understands the concept of a standalone question you don't need to explain it just ask for it then create a chain with the prompt and the model and lastly invoke the chain remembering to pass in a question and for now we'll just log out the response to check it's working now just to be clear in this challenge we are not going to be wiring this up to the chat interface yet if we were going to do that we would have to actually come down here into this progress conversation function and update some of the logic in here it's a little bit of unnecessary complication at this point we will be coming on to that later also I've given you the four consts that you're actually going to need to complete this Challenge and we're just logging out the response right here I've also given you some hints above each const and I've done that because there is quite a lot of new syntax here and of course you are welcome to go back to that scrim to check the syntax if you need to but hopefully with these hints that might not be necessary now when you do this just remember that the question you pass in should be quite a long- winded question because the idea is to reduce that question down to just the bare minimum so you want to make that a long question with some unnecessary words it's only then that we'll be able to see that it works okay pause now take all the time you need and I'll see you back here in just a moment okay okay so hopefully that went just fine so we'll start off with the Standalone question template this is going to be a string and we're just going to ask for what we want so I'm going to say given a question convert it to a standalone question and then I'm going to say question colon and here I'll use the curly braces and we'll introduce the input variable which is question now I'll invite the model to provide us with the Standalone question by saying Standalone question and just finishing on a colon so that's like an invitation to complete right now we need the prompt so let's set Standalone question prompt equals to prompt template and we'll use the pr template method and pass in Standalone question template then we'll set up the chain so we'll take the prompt that we've just created and we're going to link up the chain with the pipe method and we're piping the llm and remember we've got the llm already set up right here okay to check it's working we need to invoke it and pass in a question so we'll take that chain and this is an async process so we'll await the Standalone question chain call the invoke method and we'll pass in an object we've got the input variable that the prompt is expecting right here it's question and for our question I'm going to go for something quite longwinded I've asked what are the technical requirements for running scrimba I only have a very old laptop which is not that powerful okay we're logging out the response so let's hit save and see what we get down in the console and there we are can a very old laptop meet the technical requirements for running scrimber it's reduced that question down to a standalone question it's just asking for the precise information and it's removed unnecessary words okay that is the first part done so now is a really good time to take a break and relax especially if you've been working at this for a while and when you're ready to go on we're going to look at how we can take our Standalone question to get chunks of matching text from the vector store but there is quite a lot to do that there are several steps in that process and some preparation so for now take a moment to chill and come back when you're ready and see where we are and we are right here so we've taken the users's input and we've created this Standalone question from it and the next step will be to create embeddings from that Standalone question and then take that to the vector store to find the closest match so if we have a standalone question like what is a scrim we'll use the embeddings model to create our vector and then in the vector store we will search through all of the vectors we'll find the closest match and we'll take that chunk of text now this diagram is a little bit of a simplification we might not just take one chunk of text we might take two three even 10 chunks of text and a bit later on we'll look at how we can exert control over how many chunks we retrieve from the vector store okay before we can do any of that we need some basic setup so I've bought in a few Imports here which we've seen before we've got the superbase vector store class I've also bought in the open AI embeddings from Lang chain and create client both of which we used before when uploading to the vector store so those are all of the Imports we need for now and then down here I've set up a new instance of embeddings I've passed in the open a API key do remember that if you don't pass anything in Lang Chain by default will look for process. mv. openai API key in this format and do it for you I'm going to pass it in here manually just for clarity okay we've also got our superbase API key and the superbase URL and we've used that to set up the client and again that is identical code to when we were uploading to the data store okay now things get a bit different to what we did before let's start a new instance of the superbase vector store in a const and and we're going to pass it our embeddings model and we'll also pass it a configuration object in this configuration object we only really need to pass it the client but again just for clarity I'm also going to pass it the table name and our table name in superbase was documents we can see that one right here in the superbase dashboard and I'm also going to give it the query name and again if we look at the superbase dashboard we've got this query match documents which we named after this function match documents so let's pass that in right here so these two are both defaults but I think it's a good idea to put them in here right now because if in the future you're working with a more advanced use case perhaps you've got more than one table or more than one query name it's important that you know that this is where you can configure your vector store okay now this is quite different to what we were doing when we were uploading to the vector store we actually want to retrieve something and without Lang chain we would have to actually write a ton of code to do that but now all we're going to do is create a new const and I'm going to call it Retriever and I'm going to set it equals to our Vector store and I'm going to call the as retriever method on it and that is all we need to do this as retriever method knows to go to the vector store and instead of inserting more data to use the matching documents function that we added here to find the nearest matches so that makes it nice and easy and if you'll remember this match documents query came straight from the Lang chain docks so all of that provided by Lang chain just makes life really easy okay this retriever is now finished and that means it can now be added as an element in a lang chain chain so let's do that next we're just going to come in here down to our chain which we've got right here and we're just going to pipe the retriever on the end so what should now happen is that we invoke the chain and we pass in this question at the top of the chain the Standalone question prompt creates its own prompt from that question it's piped to the llm to get the Standalone question and that should now be piped to the retriever to retrieve the nearest chunks from the vector store well we're logging out the response let's hit save and see what happens and if we open up the console we're getting an error and it's one of those kind of vague errors e. replace is not a function so I think what we should do next is isolate our Retriever and see if it's working so I'm actually going to delete this pipe that we put on the end of this chain and I'm going to set up a second response right here and this is just a test so I'm just going to call it response two and what we're going to await here is our retriever we'll call the invoke method and we'll just pass in a standalone question and I'm going to follow on from this question here and just say will scrimber work on an old laptop okay let's just log out response 2 and see what we get and look we are getting some chunks and we can see that we've actually got four chunks and the first one starts what are the technical requirements so that is looking good for an answer to this question so I'm thinking that our retriever is working just fine so why is it not working in this chain well I think if we log out the original response as well we might see an answer to that question so let's run this again and now we've got quite a lot of data in our console too much to see easily so I'm just going to paste it into a markdown file okay so what we can see is that where we were logging out the original response we have got this object right here and we've got the Standalone question what are the minimum technical requirements for running scrimba can I use it on an old less powerful laptop so that was the Standalone question here we've got our chunks and we've got four of them and that's great so what exactly is the problem if you want just pause for a moment and see if you can figure it out okay so maybe youve figured out that if we have a look at the chain what we're actually doing when we pass things along the chain is passing them along in their expected data type so we invoke this chain with this string and the Standalone question prompt is expecting a string and the llm is expecting the output of the Standalone question prompt it knows what format that's coming in but when we try to pipe the retriever on the end what we're doing is we're passing along this object but what we've just seen from our experiment is that the retriever works with a string so the only problem here is that we need to be passing along just the string from this response now we could do that with DOT or bracket notation but Lang chain actually gives us a better way of doing it and it introduces us to the concept of an output passer and that is something that you should know about so let's look at that next so Lang chain offers various output passes to suit some specific situations so for example you might need to Output in Json or you might even need binary data we're not doing anything so complex here but we can use the most simple output passer which is the string output passer and as the name suggests it's going to pass the output as a string so if we go back to our code I've put this chain back to how it was we're piping in the retriever at the end and we're just logging out the response now we already know that that doesn't work we're getting this error so now let's import the string output passer and all I need to do is add a new instance of the string output passer to the chain and it's going to come right in here so we're taking what we get back from the llm and that is what we want to pass as string so let's come in here and we're going to pipe in the string output passer and the string output passer is a class and in here I just want a new instance of it it and let's invoke that and when we hit save and I open up the console there we are it is working we have chained everything together and now we are getting the four chunks back from the vector store and just to see the string output pass in action let's quickly delete the retriever so I'll just delete this off the end now when I hit save the output from the llm is just a string before without the output passer we were getting an object okay let's just put all of that back so that is pretty cool we're definitely making progress however you might be looking at this and thinking we've got pipe and pipe and pipe how many pipes are there going to be in this chain and also this chain is called Standalone question chain but we've just added the retriever on the end and that is a little bit strange so why don't we just come in here right now and do a mini refactor and I'm just going to call this chain chain and of course we need to update that where we invoke it shortly we'll be breaking this chain up and doing a bigger refactor but first I want to show you the limits of what we can do with just the pipe method so next let's think about using what we've got down in the console to try to formulate an answer to our question now I think we going to run into not one but two problems so let's go and check that out feeling it was starting to get kind of messy and we've got another prompt to add so to tidy things up a bit I've taken all of the retriever code and given it its own file here inside this utils folder I'm just exporting it and then importing it back into index JS right here the retriever is pretty stable now we will do a tiny bit more with it later but it's good to tuck it out of the way and we'll do that as we go along just to keep things here in index.js manageable okay let's go back to the app so we've got this chain down here which we're invoking with this question and the question becomes a standalone question which we're using to get relevant chunks of data from the vector store so if we have a look at our flow diagram we've done all of this we're right here we're finding the nearest match from the vector store and what we want to do next is create a prompt to get us the answer now we're going to do that using three pieces of information we need the nearest match from the vector store we need the user input which we're getting right here that's not the Standalone question that's the original question that the user asked and we also need the conversation which will later be stored in memory now we're not doing that until the end so I'm going to leave the conversation crossed out for now now it might cause you a bit of confusion that we're creating a standalone question here but we're not actually using it to get the answer so just to explain the role of the Standalone question is to get the most relevant information from the vector store the the original question might well contain more data which the llm can actually use to generate a more relevant conversational answer but we wouldn't want to use that data to find matches in the vector store let me just show you what I mean with a little diagram so imagine a user comes to our chatbot and they say I'm a complete beginner and really nervous is scrimba for me well we reduce that to a standalone question is scrimba suitable for beginners now if we use that question to generate our final answer our final answer would look something like this yes scrimba is suitable for beginners and that's a perfectly correct reasonable answer and it will leave the user feeling I think fairly neutral now if we use this original question when we form the final answer we can actually use the extra information that we didn't use when we were going to the vector store so that means we'll be able to form a more conversational response yes scrim is perfect for you we're a friendly community so don't be nervous and that will actually give the end user a much more positive feeling from the chatbot okay so that's why we're using the original question in our answer template so now it's time for a challenge and here it is I want you to create a template and prompt to get an answer to the user's original question remember to include the original question and the text chunks we got back from the vector store as input variables and you can call those input variables question and context and I've just put this warning here that if you go ahead and add this to the chain this is where you're going to find the limitations of chaining pipe methods together so don't be surprised when you get an error you might already see where the error is coming from and in fact there's going to be more than one problem to solve here and it will take us several scrims so really just take this challenge as prompt template practice and I've put here a wish list of what we want this chatbot to be able to do so you can and think about that as you're writing your prompt okay pause now get this challenge sorted and we'll have a look together in just a minute okay so hopefully you managed to do that just fine I'm going to come in here then and create an answer template and this is the text I've written you are a helpful and enthusiastic support bot who can answer a given question about scrimber based on the context Prov divided try to find the answer in the context so I've put that in there just to emphasize that the context is the knowledge that we want the chatbot to use if you really don't know the answer say I'm sorry I don't know the answer to that and direct the questioner to email help at sca.com don't try to make up an answer and always speak as if you were chatting to a friend so so hopefully we'll get a nice friendly engaging response from the chatbot now we just need to add in the input variables so we've got context and remember that will be the chunks we got back from the vector store and we've got question and then I'll finish the prompt with an invitation to create okay now we need to use this template in a prompt and let's come down here and we'll add it to the chain so we can just pipe it right on the end and is this going to work well no I've already given you a spoiler this is going to cause us an error and the error we're getting is missing Val for input context so that's pretty understandable we're not passing in context in the format it's expecting so what we need to do at this point is rewind and start sorting out the data we get back from the retriever it's a little bit of a process but let's start that in the next scrim the output from the retriever is this array of objects with metadata and all we want is the text and we're not trying to get that text into any special format we just want a string so what we can do is create a function to extract the text join it into a string and we can just add that function to the chain so basically what we want this function to do is map over this array get the text from page content and join it into a string so I'm going to call this function combine [Music] documents and it can take in docs as a parameter and that will be what we get back from the Retriever and now we just need to return the docs having mapped over them and returned the page content as we can see down in the console it's the page content that we're interested in and we need to join them together so let's use the join method and we'll take a double new line As the separator let's see if that works so I'm just going to add this function onto the chain and now let's open up the console at the moment we've got the array of objects and when we hit save we get back one one big string so that is working that's really good let's try chaining on then the answer prompt that we've got right here and I'll hit save and we get an error we're still missing the value for the input context and I did warn you there are multiple problems here so now what we see are the limitations of this pipe method we can give the answer prompt a string but we don't have an easy way to give it the object it wants remember it wants both context and question but luckily we have got another way to make Chains It's an alternative syntax called the runnable sequence we did touch on it earlier and it gives us more options now it's quite a big topic we're going to take several scrims to go over it but before we dive into that it might be a good time to take a break if you've been doing this for a while but just before we finish in my effort to keep this code base a little bit tidy I'm just going to take this function right here and move it into the utils folder right when you're ready and rested let's move on I've got a feature from a language app which is going to take in a sentence and we've got one here I Don't Like Mondays and it's going to do three things now you'll notice I've marked some errors in this sentence we've got a lowercase first letter we're missing an apostrophe in don't we've got a D on the end of liked which makes no grammatical sense and we're missing the uppercase first letter Mondays now the first llm call is going to solve the punctuation errors so we'll get uppercase I apostrophe and uppercase M we've still got the grammatical error but that's dealt with by the second llm call now the sentence is correct and we'll use the third llm call to translate that sentence to a given language in this case French now if that sounds a little contrived to you well it probably is I was just looking for a not too complex way of demonstrating a few things with runnable sequence so so just bear with me now we've already got the first two prompts here so we've got the punctuation prompt and the grammar prompt and then we're invoking the chain but at the moment the chain doesn't exist so let's build it using a runnable sequence and we're going to start that off by importing the runnable sequence now let's come down here and set up the chain so we'll say runnable sequence which comes with a from method and we pass in an array and in this array we can start building the chain so we'll start with the punctuation prompt and let's just open up the console let's just hit save and see if that works and it actually fails and that's happening because we've only got one thing in the runnable sequence a runnable sequence needs at least two elements in the chain so let's add in the llm and I'll hit save and this time it looks like it's working we've got down in the console I Don't Like Mondays and as you can can see the punctuation has been corrected the grammar mistake liked with a d on the end is still there okay next let's add the string output passer and again I'll hit save and there we are we're getting our string so this is looking like a nice neat way to do chains let's just add in the grammar prompt we'll hit save oh and that one fails and again we've hit the problem that the template in grammar point is expecting an input variable punctuated sentence so let's just rewind now these three elements in the chain work because what they pass along is in the format expected by the next element in the chain now one thing that's really cool with runnable sequence is that you can put an arrow function in the chain and log out exactly what's happening at that point in the chain to see what's going on so let's try that now so preve result is a parameter in an arrow function and I can call it whatever I want but preve result seems like a good idea as we're seeing the result of the previous action so let's hit save and we're going to get an error because we've broken the chain but we will see what we're logging out we've got the sentence property holding I Don't Like Mondays and the language of French and that is just logging out the result of our invocation that we can see right here so what I want to do next is move this line of code down the chain let's hit save again and we see the punctuation prompt let's move it down one more step and now we see the result of the llm and move it down to the end of the chain and we see the result of the string output passer and this is of course where we hit a problem because the string output passer is giving us a string and the grammar prompt which is what comes next in the chain is hoping for an object which looks like this punctuated sentence which is the input variable and then a string with whatever we got back from the string output passer in here so we could actually do that we could come in here and build the object so we've got the key punctuated uncore sentence and now for the value we can just take pretty much what we've got here although of course we're going to delete the console.log so we just return the previous result and look at that it works we're not actually seeing the final result yet but that's just because we need to chain in the llm and the string output passer let's hit save again and there we are now we've got our Center it is completely correct there's no grammatical errors and no punctuation errors so this has worked but I don't think it's very elegant and I don't think it's very intuitive so in the next Grim let's refactor this and we're going to find a neater way of doing it but let me just warn you before we finish this trick with the arrow function in a runnable sequence is actually very very useful in some situations and we are going to be using it in a challenge just a little bit later so don't forget it working but it's not the most elegant so let's refactor now one option is to go for a runnable sequence in a runnable sequence so I'll come right in here delete this and create a new runnable sequence right here and we do that in exactly the same way with the from method and now I just need to list these three inside that runnable sequence and of course delete them from here and let's just tidy up the formatting and actually that line of code is so long it's never going to look too good let's just hit save and see what happens so it does work it works perfectly well but likewise I think this is rather awkward so what I want to do is actually extract this chain so I'll create a const punctuation chain and that is going to hold this runnable sequence and now we can have whatever this chain returns be the value of this key value pair let's just bring this back onto one line okay let's give it a try and that works and by the way this could be a runable sequence or it could be a pipe chain it actually just doesn't matter at all you might find for these chains which are just a prompt an llm and an output passer that using the pipe method is just as good it makes the code slightly sh water and it's a matter of personal taste as to which one is easier to read now I'm going to leave this as a runnable sequence and I've just realized I've got an unnecessary trailing comma right there now I think this is going to look neater still if we also extract the grammar chain and I'll just hit save and there we are that works just as well Okay so we've got the runnable sequence working but we need to remember the last part of our app which is a transl ation and where we invoke this chain we are passing in a language French and we're not actually using that as yet in any of our templates but when it comes to having the translation template we are going to need it so I'm thinking I'll just come in here right now and I will add in that translation template and there we are we've got a translation template and a prompt which comes from it so what we'll be needing to do next is potentially give it its own chain and of course we need to add the translation chain to the main chain now do you think this is going to work well of course it's not we've been here before we're missing the input variable for language and that figures because we're not actually doing anything with language we're invoking it here but we haven't bought it into this chain and we certainly haven't passed it down the chain now strange as it may seem taking this original input and passing it down the chain is not a trivial thing to do and Lang chain actually give us a special tool to do just that so let's check that out in the next scrim slightly strange app we've added the translation chain so we should be able to get this translation of our original sentence now it's not working and we know why it doesn't have access to the input variables it wants it wants the language and grammatically correct sentence input variables now the grammatically correct sentence input variable should be coming straight from the grammar chain so we should be able to break this one into an object and add that really easily but what about this language input variable we know it's coming in here where we invoke the chain but to pass it down through the chain we actually need to use a special tool and that special tool is called a runnable pass through and we're going to import it from from exactly the same place that we get the runnable sequence so we can just add it in right here okay so we use it like this let's come to the first element in the chain and we've got this object and I'm just going to bring it down onto two lines and now I'll add a new key value pair and the key is going to be original input and the value is going to be a new instance of runnable pass through now let's come in here to this second object and again I'm going to bring it onto multiple lines and here I'm going to add a property original input and for the value we're going to use an arrow function just to log out what the original input is and just before we save that I'm just going to comment out this line and now we won't get confused by what we see in the console and we're seeing out errors but interestingly we're also logging out the original input and we can see what original input is in that object it's the sentence I don't liked Mondays with no punctuation or grammar and the language French so it's actually this object here that we passed in when we're invoking the chain so what we can do now then is destructure original input and now we're destructuring we need to use parentheses and then we should be able to access the language just with notation let's it save and see if that works and it does work we're getting the language French so let's just delete the console.log and will just return the language and of course we need to update the key here to language because that is the input variable that we're expecting in this template and let's hit save and see if it's worked oh and I think I forgot to reinstate this line which means we'll be waiting a long time let's try that again and it does work we are logging out our French sentence so we've successfully removed all of the errors and translated it into French and the really important thing here is that we've managed to pass an original input right down through the chain so we can get access to language in the final element of the chain now just before we move on I want to say a quick word about formatting it might well be that if you do use a runnable sequence here you think it looks better if we bring down the elements in these arrays onto their own lines and for me I think that probably does look a bit neater but that is a matter of personal taste now have a good look at what we've done here because everything that we've done here is aping something that we need to do in our app so when we go back to the app I'm going to give you a big Challenge and when you've completed that challenge the app should be fundamentally working bar one or two Loose Ends so have a good look at this code make sure you understand everything and then when you're ready move on we have studied everything we need to know to get this app to some kind of minimum viable product what I mean by that is by the end of the challenge we should be able to see an answer to this question down in the console and after I've shown you my solution we're going to wire up all of the Dom elements and then we've just got a few final features to add now I've imported the runnable pass through and the runnable sequence for you so you don't need to do that and let's just quickly check out the flow diagram to remind us where we are so we've basically done all of this and now we're going to be getting the answer using the nearest match and the user input we haven't wired up the conversation yet we're doing that last so I'm going to leave that crossed out you don't need to worry about it so your challenge is this I want you to set up a runnable sequence so the Standalone question prompt passes the Standalone question to the Retriever and the retriever passes the combined docs as context to the answer prompt remember the answer prompt should also have access to the original question when you finish the challenge you should see a conversational answer to our question in the console now I think the neatest way to do this is to create three chains one for each section so there'll be a standalone question chain a retriever chain and an answer chain and then you can bring them together in a runnable sequence but that said you might choose to do it a bit differently now I'm going to give you a choice here if you're up for a challenge you can go ahead and get to work but if you'd like a bit more guidance I've got a file up here called hints. MD and in here I've listed out the steps that I would take to get this working so you can refer to that if you need to whether you look at that file or not definitely feel free to go back to the previous scrims to check out the syntax as you do this and take as long as you need this can be pretty confusing at first but it really pays dividends to figure this out yourself either with or without the extra help in this file rather than just watching my solution so really give this Your Best Shot pause now get to work on this and then move on to the next Grim when you're ready to see my way of doing it and it's probably worth opening the next Grim in a new tab so you can keep your solution to hand for comparison purposes and massive congratulations if you did a few tricky bits to remember there now I'll show you how I did it and remember my way isn't the one true way if you got it working your way might be just as good or even better than mine so I'm going to come in here and set up three chains the Standalone question chain the retriever chain and the answer chain and then this here we will reuse as the main chain where we bring everything together on this first chain I'm just going to use the pipe method and I'm going to chain together the Standalone question prompt the llm and the string output [Music] passer now you could do this here with a runnable sequence it's totally up to you I've gone for the pipe method on these basic chains because it just keeps the code a little bit shorter now we could also bring these pipes onto new lines and that might make things a little bit neater still okay I want to add this first chain to our main chain so I'm just going to delete all of the code that we've got here and then the main chain is going to be a runnable sequence and then we'll pass in the Standalone question chain now let's just think for a second this chain will give us two things it will give us the Standalone question and the original question that we'll need in the answer prompt so let's create an object and we'll have the Standalone question and the value there will be whatever's provided by the Standalone question chain and we can have another property which is the original input and that will be a new instance of the the runnable pass through okay and onto the retriever chain and here we faced a bit of a gotcha the retriever chain isn't expecting an input variable but it needs the string we get from the Standalone question prompt so let's get at that using an arrow function and that means that this chain will need to be a runnable sequence so the first thing in this chain will be the arrow function that gets us the Standalone question so we'll take preve result as the parameter and we'll return pre result. Standalone question next in the chain we need the retriever itself and finally we want to extract the text from the array of objects we got back from the Retriever and we do that with the combined documents function so in this chain we're taking the previous result and we're extracting the Standalone question from it that's going to give us the string which we pass to the retriever the retriever finds the nearest matches and brings them back to us as an array of objects and the combined documents function extracts just the text from that array of objects and joins it into a string okay let's add the retriever chain to our main chain and what the retriever chain needs to pass on is the context because that is what the answer template is expecting as well as the original question so for the context that's coming here with the key as context the value is the retriever chain and then for the original question here we're going to take the original input that we've passed through with this runnable pass through and extract the question from it and now we just need to deal with the answer chain and the answer chain is going to be much like the Standalone question chain so we'll set that equals to the answer prompt and we'll pipe it into the llm and likewise we need a new instance of the string output passer okay that is the last link in the chain so let's just add it on here in the main chain and now I'm going to hit save and let's see if we get an answer and look we do it says scrimba is designed to be lightweight and can be used on low speec PCS so your old laptop should work just fine so what we're getting there is an answer to this question and that is really really good that is exactly what we want and the answer we're getting is accurate it's from information that we gave it in our original document and has been turned into a conversational answer by the llm so again congratulations if you got it to this point now the end is almost in sight in the next scrim I'm going to wire up the UI that's pure JavaScript there's no AI involved in that if you fancy it as a challenge go ahead and do it most of the code is done for you if not just feel free to sit back and watch on this one haven't really touched we've got this event listener which is listening out for a submit event which will fire when this button is clicked and that's going to call this progress conversation function now if we come right down to the bottom we can check out that function and all we need to do to get this working is to take this chain invocation right here cut it and paste it in here now I'm going to delete delete the console.log we don't need that and also we want to get rid of this hardwired question and we're going to invoke with whatever the user has inputed and we've got that stored in this Con right here question which is coming from user input. value and that is just being collected from this input on submit and that is all we need to do let's hit save and now I'm going to ask it a question and I'm going to ask it how long it will take to get a code review for a solo project oh and we're getting an error and it's saying result is not defined and I think what I've done here is I've called this one response and this one down here result well it doesn't really matter how we changed this let's change this one to response and I'll ask it the same question and there we are we get a great answer and that answer has come right from the document that we gave it at the beginning so that is really really good now I'm going to ask it if it knows about me Tom Chan is a scriber teacher that's reassuring it knows that now let's start again and I want to do an experiment I'm going to introduce myself and of course it doesn't know I'm a scrimer teacher we've just refreshed it doesn't know anything about me and in fact I might just make up a name so I've introduced myself and asked what is scrimber and there we are we get a great answer it gives us a lot of detail greets me by my name so it's being really conversational now I want to ask it what is my name and of course it doesn't know and the good thing is it's advising us to email help sca.com so it's got that straight from our prompt I think that was right there in the answer prompt here we are email help.com if you don't know the answer so that's working really well now the fact that it doesn't know my name when I gave it my name right here and it's repeated it once tells us what we already knew which is that this chatbot has no memory so the last thing that we need to do then is add some memory to this app and we've got all of this left on our flow diagram plenty of arrows going on so it looks really complicated but actually there's not much more to do but we'll come on to that in the next scrim some memory now there's two parts to this there's some JavaScript to set up and then we'll need to wire the conversation memory into the chain so that the answer prompt has got access to it now I'm going to cover the JavaScript setup in this scrim and you're going to have the job of wiring it into the chain in a challenge in the next scrim so firstly I want to create a const to hold the memory and this memory is going to be an array and I'm just going to call it conversation history so we'll come outside of the progress conversation function and this is where I'll set it up and I think I'll abbreviate it to com history now let's add to the conversation history every time a user submits a question and every time we get a response back from the chain so we'll come down here and we can say conf. push and we'll push the question and the response okay that deals with a couple of arrows on this flow diagram so the user input is now saved to the conversation history and the answer we get back from the chain is also added to the conversation history but at the moment all we're going to have stored in this array is a bunch of strings which is fine but it's going to help the AI understand this better if it's clear which strings come from humans and which come from AI so I'm going to write a function just to add a human or AI label to each string and then we're going to join them together into one big string which we can use in our chain now again in the interest of keeping index.js as clean and clear as we possibly can I'm going to create this function in utils so we'll say function format com history and it will take in an array of messages and then it's going to map over those messages now the Callback will have each message as a parameter and also the index and we're just going to work out whether it's a human or AI message by doing some basic maths on the index so what we're assuming here is that the human speaks first so the zeroth message will be a human and all of the odd number numbers will be Ai and all of the even numbers will be human now we just want to join them together into one string and let's just join them on a new line character now when we call the function on the conversation history what it would do is it will take the array of strings and convert it to this so there's one long string but it's really clear what the human said and what the AI said okay let's delete this and we'll export this function and import it back back into index.js right so the setup is done and in the next Grim you can wire this into the chain let's wire up this memory now at the moment on this diagram we need the conversation history right here where we get the final answer to our question but I think it might help performance if we have access to the conversation history where we're creating ating the Standalone question it might just help it create a better Standalone question in some Edge situations so I'm going to add another arrow to this diagram now we want our conversation history from our memory store in two places the Standalone question and the answer chain at the very end and that means it's time for a super challenge or a mediumsized super challenge it's not quite as big as the previous super challenge but there is quite a lot to do so I want you to pass history into the chain as comore history at the point where we invoke it and I've just put remember to make use of our format com history function now if we just go back to where we invoke the chain this is where you can pass in comcore history and I've just put the underscore in just to keep up with the same format that we've used throughout once you've passed that in and you've used the format com history function The Next Step will be to come to the Standalone question template and make use of con history in that template and all I mean by that is come here into the template we're going to add con history as an input variable somewhere and just update the instruction so it knows it can use the conversation history as a resource as well when you've done that we need to come to the answer chain and we need to make sure it has access to com history so again down in the chain you're going to need to do something around here once you've done that and the answer chain has got access to com history again you need to come to the answer template and use an input variable to add com history somewhere in here and instruct it on how to use it now if I were you I would just use wording in here that makes it clear that it can use the conversation history to help it find the best answer but that the source of knowledge Remains the context so I'll leave it up to you to think of some work wording there and then you can see what I did and of course in the real world you'd have to play around with this prompt template and just do some experimentation until you get results that you're happy with okay once you've done that we should be done so you can test it by giving the chatbot some information and checking in the next question to see if it remembers it so a really easy way of doing that is exactly what I did before tell it your name and see if it remembers your name okay quite a lot to do there so pause now give this your best shot and we'll have a look at it together together in just a moment okay hopefully you managed to do that just fine so let's come down to where we invoke the chain and I've already added the comore history property and the value there will be con history however before we send that off we want to format it so let's take our function format com history and we'll just pass in com history okay let's go back to the challenge so we've got com history in the chain and we've formatted it let's go to the Standalone question template to make use of com history right there and I'm going to convert this to back ticks and I'm just doing that because like with the answer template we're going to end up with a list of input variables so it's just a bit neater when it's back ticks and we can spread it out onto multiple lines now I'm going to update the text right here so I've just said given some conversation history if any and a question convert the question to a standalone question and then here we've got our requests for input and then here I'm going to bring things down onto a new line and we'll add the conversation history input variable so now we need to make sure the answer chain has access to com history and we need to edit the answer template to make use of it well that's actually two things in one let's come down to the chain and then right here we're using this runnable pass through to get access to the original question now we can do exactly the same thing with com history so let's copy that down onto a new line change this to com history and change question to comore history okay now let's come up to the answer template and I'm just going to make some changes here so I'm going to say you can answer a given question about scrimber based on the context provided did and the conversation history I'm going to leave this sentence intact try to find the answer in the context but I'm going to add in here if the answer is not given in the context find the answer in the conversation history if possible now I think that's going to work but again in the real world we would have to do some testing here try a few different ways of wording this and just see what got us the results that we wanted now let's come in here and we'll just add the com history okay it looks like we're done so so let's come back up to the challenge and it's just telling us to test it so let's hit save and I'll say my name's Tom what is scrimber and it's given us an answer to that question and now I'll ask it what my name is and it says your name is Tom so the memory is working and we have successfully created a chatbot knowledgeable about our document and with a memory and that is a pretty awesome thing to do when you remember how Unthinkable this would have been even just a couple of years years ago let's just refresh and ask it one more question and I'm going to ask it what is the scrimba community like and it says the scrimba community is a global community of friendly and helpful coders we believe in learning to code as a community activity so we've set up a Discord server where you can meet fellow coders share your problems and solutions and network and that is actually very true and you can come and hang out with us there so this project is done congratulations on making it through to the end and let's just take one more scrim to recap what we've learned well every question I've asked it I'm happy with the results but that won't always be the case so what can you do when performance is not up to standard well there are a few tweaks you can make firstly you could go right back to where we set up the original document and you could alter the chunk size we went for a chunk size of 500 characters you could go larger if you think your chatbot needs more context and you could go smaller if you think it needs more granularity you could also change the overlap size we went for an overlap of about 10% next you could look at the number of chunks which are actually retrieved and I did mention earlier that we would say something about that so let's quickly come over here to the utils folder and find our retriever now right here we've got this Vector store as Retriever and we we can pass a number in here and that will control how many chunks we get back from the vector store let's just see what's happening at the moment if we head down to our chain and in fact we want the retriever chain here we are so let's just come in here and I'm going to add in an arrow function and log out what this retriever is giving us now this is going to break the app but if we just open up the console and I'm going to ask any question now down in the console we can see that we've got four chunks let's come right back here to the Retriever and I'm just going to whack this up to 10 and I'll ask a question and there we are we get 10 chunks down in the console and we could of course take it down to as low as one and we get one chunk now I'm going to leave that empty so it defaults to four I think that is absolutely fine for this use case but do be aware of that setting because it can be pretty useful if you get more chunks you can provide more context but if you feel your answers are a bit vague and off topic you might want to reduce that number and just focus in on the best quality chunks because remember the retriever is going to give you the closest matches first okay let's just take that out of index.js because else it's going to cause US problems now you have got other options you can look to prompt engineering so just come here to where we've created our templates the answer template the Standalone question template and you can just think about how you can tighten those up I think Al promps are pretty good here but if you find you're not getting the performance you expect you can mix things up go into more detail add some examples and just experiment and see what you get now lastly you shouldn't forget the open AI settings again in index.js we've got the llm setup right here and what we could do of course is open this up and in here we can use any open AI settings we want so we've got for example temperature and when working with your own knowledge bot it might be a good idea to set that to zero remember the higher the temperature the more daring the AI gets and we don't particularly want daring here we're not trying to be creative so actually setting that to zero might be a pretty good idea now you could also change which model you're using GPT 4 GPT 3.5 any other new model that comes out and likewise here we've got access to frequency penalty and presence penalty and they might not be so relevant for this use case but they are there if you need to use them as well as any other setting that you can use with the open AI API so do be aware of that and you can always come in here and change things if you need to I'm of the opinion that if it's not broken you don't need to fix it so I'm actually going to leave this exactly as it was okay so those are five things that you can look at if you're not getting the performance you expect and so hopefully that will be useful if you do run into problems okay we're pretty much done with this so let's just take one more scrim to recap what we've studied ation on finishing this course you now have a really strong foothold in the world of Lang chain and you've got the skills you need to build powerful scalable AI applications quickly let's just recap what we studied firstly we used a text splitting tool from Lang chain to split our documents into chunks we created vectors for those chunks using an open AI embeddings model we then set up some templates and prompts and we can see examples of those right here when the prompts were ready to go we chained them to large language models and the string output passer using the pipe method and again we can see an example of that right here now those were the more basic chains and we came on to more complex chains a little bit later but first we took the user's input and vectorized it and then found the closest matches from the chunks in our Vector store we use those chunks to generate the final answer and we chained it all together using a runnable sequence which is a really cool way to build a more complex chain wow that was quite a lot so why don't you head over to our Discord Channel and go to the today I did Channel this slide is a link right through to that channel and brag to the world about what you've achieved and with that all that's left to say is thank you very much for taking this course do feel free to reach out to me on Twitter at TP chant it is always good to hear from you and always good to get feedback and till next time good luck

Channel: freeCodeCamp.org

Views: 58,578

Keywords:

Id: HSZ_uaif57o

Length: 99min 27sec (5967 seconds)

Published: Mon Nov 20 2023