Step by step no-code RAG application using Langflow.

by the end of this video you'll know how to build an AI application that answers questions from a website without writing a single line of code that's my hope we are going to do everything visually we're going to be using LF flow which is an open-source project and I'm going to show you how to build this AI application but my goal goes beyond that I also want you to learn how every component works so I'm going to build it here from scratch so you see me doing it I'm going to try to explain the reason behind every decision I make so when you finish this tutorial however long it takes you're going to have everything you need to just go UND do this with your own data right and use your own model for this this should be fun here on the screen I have llow again is is an open source project is pretty cool I'm going to be using the pre-release version 1.0 this has not been released yet as of right now that's the version that I'm going to be using there are several changes from the current stable version that's why I want to use the pre-release version it's super cool I've tried both I really like version one I'm going to install that one now to run L flow you only need to do a couple things first you need to install it on your computer obviously this is the command right here to install the pre-release version and you need to have python 3.10 so make sure your version of python is 3.10 if I go to my terminal window I'm going to type python version I have the 3.10.1 version so you need to make sure you are on 3.10 as well then you can run this command I already did I already have it installed it's going to take a few minutes to do so I didn't want to do it uh here on camera so I already did this and after you install it then this is the command to just run LF flow and you'll see what's going to happen I'm going to run it here on my command line and when I do this is going to open a web browser on my computer now I have here Safari I'm just going to grab that URL and I'm going to go to Chrome because I already prepared Chrome for this so I'm going to close Safari here really quick and you can see this here is the URL that you need to open in your browser and that is going to show us the actually this is this is the one it's the same one let me just make sure sure that's the case yes it's going to show you the landf flow user interface so when you start you have nothing this is what we're going to get I'm going to start by creating a new flow so if I go to new project I'm going to use blank flow we're going to start from the beginning and this is what I'm going to get here on my left I have the core components this is just a menu with everything that I can add here and there are many many components here so anything that you have in mind that you could do you actually can do it right it's very well organized and there is a section that's called extended and when you click there this is is just on on on and on right how many components you have to start building my goal as I stated at the beginning of the video is to build an AI application that answers questions from a website I have a website and I want to answer questions from my website so let's imagine that I want to build the chat bot where users that come to my website can answer questions and the AI application is going to use the content of the website to answer those questions it's a more modern FAQ right frequently asked questions that people can actually ask those questions uh online to do that I'm going to need a model that's the first thing that I'm going to need so I'm going to go here to the models section and you can use anything that you have in mind right so I have open AI here I have anthropic I have coher I also have Ama so if you want to use a local model mod or an open source model like llama 2 or mol you can use it or llama is right here so you could be using that model for this demo I'm going to use open AI because it's the one that most people are familiar with it's very simple to use uh so here's my model here I have an open AI you can see that I can select which open AI model I want to use I'm going to start with 3.5 turbo let's see if this does the track cheap it's fast I'm going to need to type an API key obviously and I can change the temperature and I have an input here let me grab the API key and paste it here so I'm going to paste my API key and here is an input so I can type something let's say I'm going to type 2 + 2 equals and yeah I can then run this component let's see if something happens it ran it says open a I built successfully but I don't see anything on the screen which makes sense because I need an interface right I need an interface to just display that output in order to do that there is a section that's called output and there is a chat output I'm going to be chatting with this model so let me grab this chat output here and what do we have here well we have the message that we want to output and we have the sender name like who's sending that message okay so that's that's awesome I'm going to grab the output of the open AI component and notice how outputs are on the right side and in this case I have the text output and you get input on the left side side anything that you can fit into a component is going to be organized on your left side so I'm going to grab the output of open Ai and I'm going to put it right here I'm going to connect it to the shat output whatever the model answers is going to go into the chat output now what is this chat output going to do for me well it's going to enable this interface do you see how the rum button is now enabled well if I click on this run uh run button now I get this interaction panel where I can actually chat with the model and that is awesome because that's exactly what we need to see the model working so I'm going to close this okay because I don't have any inputs so I cannot just pass any information to the model but I typed two plus two here so if I run this output if I execute the output the first thing that I want you to notice is that when you run a component automatically you are running every component that comes before that component in the same chain so in this case the chain is the model and the chat output so if I run the chat output we're also going to be running the model because comes before in this chain and if I click on the run right now you will see the answer from the model it says four because the question remember the question is 2 + 2 equals that's awesome but we need something else let's create the ability to ask a question right so I want to ask a question I want the model to answer that question and then when going to display it in the chat output to do that I can just grab a chat input component so I'm going to grab that here let's see what we got so we got the message that we can type and we got the sender name which is going to be the user so the user is sending the message that's awesome I'm going to grab the output of the chat input and I'm going to fit it into the input of the open AI model so now my chain has three components a chat input so I can actually ask something then that's going to go into into the open AI model and the output of the model is going to go into the chat output if I run now the whole flow is going to run and now I can ask a question you see how this is enabled right now so I'm going to say tell me a joke and see how it says the user ask tell me a joke and the model comes back saying why did the Scarecrow win an award because he was outstanding in his field just a bad joke that's okay this is the table Stakes so I have an input I send that input to my model and I capture uh the output from the model and display it on the screen table Stakes but we need more than that remember what we need to do is answer questions from a website how can we do that how how we're going to be able to do that this is what uh technically speaking we call a rack application rack R A stands for retrieval augmented generation applications and in order for us to get there I'm going to need to just create create a simple prompt that is going to tell the model how to behave in other words I want my model to only answer questions from my website I don't want the model to make up the answers right if I ask the model what is the price I want my model to just look at the context look at the website and say well the price is this much I don't want the model to be generating you know a fake price to do that we need to create a prompt right now what we have on the screen here I'm not typing any prompt I'm just passing the question directly to the model but I'm going to need to create a prompt to make that distinction to tell the model how to behave so I can just grab a prompt is under input uh under the core components so I'm going to just grab this prompt uh paste it somewhere by the way just one thing I've tried many many different visual tools where you drag and drop components landf Flow by far is the easiest one for me to navigate and that might be because they're using the mouse wheel and clicking and command and control all of the key shortcuts are the ones that I'm used to so that's probably why I feel that it's it's so nice to navigate but it's yeah it's it's great you you really have to try it for yourself but it's very easy to just you know go around and and just just zoom in and zoom out it's very very very cool all right so I have my prompt and this is what I'm going to do I'm going to click on this template section here when I click there and open uh so I can edit the prompt and I'm going to say answer the question using the context below okay something like that and then I'm going to specify well context is going to come here and then the question is going to be here in order to specify the the context I need to pass a variable because I'm going to be injecting the context into this prompt I can just use corly braces and I'm going to say well the context is going to be here and I want you to notice just by me doing this automatically LF flow recognized that I need a prompt variable that's called context it's basically taking whatever is inside this right inside the cly braises so I need one variable to inject the context into the prompt and I'm going to need one variable to pass the question so now I'm telling the model please answer sure the question and I'm going to be specifying the question below using this context right and I'm going to be specifying the context great I have two variables here so I'm going to check and save and now the component The Prompt component changed now the prompt component accepts two more variables one of them is the context and one of them is the question so what I'm going to do is I'm going to grab the chat input the place where I'm going to be typing my question and I'm going to be rewiring that so instead of sending the chat input directly to the model I'm just going to grab this connection right here and I'm going to send the chat input to the prompt and I'm basically connecting that chat input to the question variable here so whatever I type that will become the question and I need to specify the context I'm going to specify a fake context for now I'm going to say Mary is uh Susan's sister okay something like that so that is the context that I'm going to be specifying and the question is going to come from the shat input Here and Now The Prompt outputs text which is basically it's going to take that template that I created it's going to replace the variables using the inputs here and then it's going to Output the whole prompt so I'm going to get this prompt template and connect that the text the output of that prompt template connected to the input of the model so let's review here really quick what we have I have a prompt I have a chat input that chat input becomes the question of the prompt I fixed my context for now I just typed Mary is Susanna's sister that's probably good enough I'm taking the output of the prompt passing it to the open AI model grabbing the output of the of the open AI model and putting it or displaying it as a shout output let's run this whole thing and let's ask uh who is Mary and the model is going to say well Mary is Susanna's sister so this is actually working which is great right is getting uh that from the context which is cool so let me ask something else uh who is JN I don't know what the model is going to say it's telling me John is not mentioned in the context provided okay that's that's cool but I want something actually a little bit different in this case let me ask something more interesting something like uh what is the Apel Tower the aphel tower is a famous landmark in Paris France okay so that's awesome so this clearly is using the knowledge of the model to answer that question because there is nothing in the context that will help the model answer that question and I don't want that to happen because I think that might create some hallucinations so I'm going to change my prompt here and I'm going to specify that I only want answer is coming from the context so I'm going to go back to the prompt and I'm going to say answer the question using the context below cool if you can find the answer in the context reply I don't know okay this is much better this is now asking the model to ignore any questions that it cannot find within the context so anything that is not there the model should now say I don't know and this is a simple prompt people have created better prompt to do this so I encourage you to find one online if you want to make it a little bit more or or you know if you want to improve it a little bit but this should do it for us let's try now let's run it let's ask uh who is Susanna and there is we're talking in the context about Susanna so yes it's Mary's sister so that's great and now let's ask something like uh what's the Eiffel Tower and now I we should get I don't know EXA exactly so now the model is only answering questions that it can find in the context so this is much better all right we're just getting started here we need to get data from our website into that context so we need to pass the data from our website into that context so how do we get there that is the question that I need to answer right now to do that I'm going to first save this flow so I'm going to come here and I'm going to click on settings and this is a flow that I'm going to call sample I'm going to call sample flow and I'm going to create a new flow okay so I'm going to create a blank new flow here and you'll see why because in order for me to answer questions from a website I first need to load those answers or the content not the answers but the content of the website I need to load it into a vector store database what is a vector store database it's just a regular database that is going to be indexing information using embedding so we're going to get uh the content of the website we're going to chunk it out we're going to create separate chunks and we going to create generate embeddings for those chunks and what that will allow us to do is find information by similarity what that means is that when we get a question we can find any chunks any information any documents in that Vector store database that are similar to the question and that similar ity is going to uh happen thanks to embeddings we can have two embeddings compare how similar they are the more similar they are the more likely those two embeddings represent similar Concepts or concepts that are tied to each other somehow therefore if I generate this the embedding of a question and I get a similar embedding from the database there is a a very high likelihood that that question is being talked about maybe answered in that the document coming from the vector uh store database the first step that I need to do is load my website information into the store database that is the first thing so I need to get that data and put it in the vector store database how do we do that well there is let me see here what we got there is a data section here and I'm going to grab a URL component so this component it says the documentation says fet content from one or more URLs this is exactly what I need okay so this is the URL that I want to load information from this is my machine learning program the one that I teach I'm going to be grabbing this URL there is a lot of information here by the way price and instructor information and what happens in a couple of sessions all of that information I'm going to use so I type my URL here and this component of course right now is not going to do anything is just going to fetch the information from that URL now I need to do something with that information and what I'm going to do there is a ton of information in that URL so we cannot just grab unlimited text and store it as a single document in a database and the reason we cannot do that is because then when we try to use that context to ask a question to a model the model will basically blow up because the context window for the model is limited which can only pass so much information to a model therefore I don't want just to load the whole website in the database a single document I want to create multiple documents I want to chunk the information from the website so if this URL creates or loads all of the content in memory I can go and if I go to extended you will find text Splitters there so I'm going to go to text Splitters and I'm going to be us using a recursive character text splitter to generate documents out of my website so this is the way this is going to work it's very simple you'll notice that here it says chunk size so that means how many characters out of the whole content of the website how many characters are we going to use to create a single document and by default it says 1,000 characters that's awesome so I'm going to be taking 1,000 characters and cutting at that point and I'm going to be saying okay the first 1,000 characters that is one document that I'm going to store in my database as an embedding cool there is also here the overlap which means like how back do we go after we you know after we get the first 1,000 characters how back do we go to grab some other characters so there is certain overlap for the second document and I'm saying 200 well by default it says 200 you can change these you can tweak those depending on your application so in this case what's going to happen is that the first document is going to go from zero to 1,000 so we're going to grab or for one to 1,000 we're going to grab the first 1,000 characters then the second document is going to go back to 100 and it's going to start there for another thousand characters and then it's going to go back to 100 it's going to start there for another thousand characters so every document is going to have a little bit of overlap if I get this and I connect the output of fetching the content of my website when I connect that output to the input here notice there is a red dot it might be a little bit smaller let me make it a little bit bigger you see that red that means that that input is required right so there is no default value for it so of course the input is required in this case so now what's going to happen if I run this is I'm going to fetch the website content and I'm going to split it into separate documents and the record here what we get here are by the way when you hover over one of the inputs or outputs it's going to give you information of what the next components could be right so this could connect with Vector stores with text Splitters with chains Etc right it's going to give you that information which is pretty neat so what do we do next well now that I have all of those documents I want to upload them to a vector store database so how do we create a vector store database well there is a section here that's probably called Vector store I'm closing the these sections here uh Vector stores here we go so within Vector store you get a bunch of different options that you can use for this video I'm going to use Astra DB it's pretty nice so I'm going to grab that here and I have an account for astrab very easy to find just type Astra DB and if you just type that you're going to find just where where you can create it I already have an account I'm going to log in so now that I'm here I can just create a vector store database I'm going to go to databases to this section I'm going to click on create database and I'm going to type sample that's the name of my database the provider is going to be Amazon web services and I'm going to pick the region Us East to and then I'm going to click on create database cool so this is going to take a few seconds is initializing that database and when you have the database you are going to need a couple of attributes to paste here first we are going to need a token so astb is going to generate a token for us so we can actually authenticate with database we also need an API endpoint so we're going to have to grab that we need to specify a collection name of where we are going to store this information within the same database you can have multiple collections right to store all sort of information so different type of information will go to different collections which is awesome and we need to specify the inputs so what is the information that we're going to be sending to astrab to store and we also need to specify embeddings so what is the class what is the method to generate the embeddings for the text let's figure out the embeddings first and then we can worry about everything else so let me start by grabbing uh an embedding class there is a section here called embeddings and you can generate embeddings with whatever class you want I'm going to be using the open AI embeddings That's the class that I'm going to be using I need to use the same open AI key I mean I can use any open AI uh API key that I want I'm going to be using the same I'm going to paste that open AI API key here and then I'm going to connect this embedding with my astrab class the reason we add this again is because when astrab Astra ASB is going to receive the documents but astb needs to generate embeddings to store those uh documents in astb so he asking us hey what is the class what is the function that we need to use to generate those embeddings and this is going to become important later because at the time of searching astrab we also need to provide a query with the embeddings class so we use the same type of embedding during query in the database than we used here through an insertion now I have the embeddings that is cool let's go back to the database database is ready it's a vector database you can see it here there is an API end point right here this API end point I'm going to click here copy that endpoint go back to LF flow and paste it here that's my API in point now I need a token okay so somewhere within Astro DB I should be able to generate a token you can see it here application tokens let me make this a little bit bigger cool so generate token I'm going to click there and now I have a unique application token I'm going to click copy go back to LF flow paste it here so what am I missing here let me see I have the token I have the API in point the collection name I'm going to call it sample or maybe I'm going to call it a website I'm going to call it website because this is the information of the website and I need to pass the inputs of course so what are we storing here well we're going to be storing the result of the splitter right of all of these records are going to become the input here for astrab so what should happen now I have the URL going I fetch that information split it into chunks get all of those chunks and pass them to astrab and astrab is going to be using the open embeddings to generate the embedding so if I execute this flow here so far what is going to happen is that I'm going to load the information from my website into astrab pretty cool let's try that out so I'm going to go to the final component and I'm going to click play or run and this is running everything that came before uh let's see if it works let me just zoom it out a little bit this should take just a few seconds it's basically generating or you know loaded all of the data created separate documents generated embeddings for all of those documents and saved them in Astro DB all right so everything worked let's go to the database and let's check the database just want to make sure uh that Explorer okay there we go we have a collection that's called website I click on that collection here you can see the data which is cool notice that notice a couple things so we have 13 records in that database that means that my whole website was turned into 13 documents because we chunked the website content so I have 13 records the dimensions of the vectors representing the embeddings are 15 36 that's the number of dimensions and that is what you should see if you use the open AI embeddings model if you use something else I think coher embedding model is 4,92 something around those lines don't remember exactly a number but depending on the embedding that you use that's how many dimensions you should see here is the table you can see the vector that is basically the embedding there is an ID there and there is the content that's behind that embedding right the content related to that particular embedding and there is obviously meta data where is the data coming from etc etc you can also see the data in Json format this is pretty cool you can see the content the vector sorry I clicked on it the vector the metadata you see the the title of the page the URL Etc and you can see how you can grab information let's say from a whole website not just from a landing page and all of that is going to get reflected here even if you wanted to just site The Source where your data is coming from you should be able to do it through that m dat data all right so all of my data is now stored in my Astra DB database or Astra database I should say and so this is cool I'm going to save this flow and I'm going to go back to the previous flow I'm going to call this Astra DB save it and then I'm going to go back to my collection so Astra DB is the one that we just saved I'm going to open sample edit flow so what needs to happen right now is that I need to pass this context is not going to be fixed anymore this context should come from ASB but the question is what exactly am I am I going to be passing as the context well I'm going to be grabbing the question that the user asks generate an embedding for that question search astrab with that embedding return any similar documents to that question and those documents will become the context for the chat to answer the question let's try this out I need to search in astp that is a given so I'm going to go to Vector search and now there is an astrab search component there this is the one that's going to search the query from the user to return related documents or similar documents so I need to pass the token so I'm going to go to astrab going to go to the database it's somewhere here is my database here is my token I'm going to grab that token and I'll come here paste the token right here I need an API endpoint which is the same endpoint that we used before going to grab it here go back to LF flow paste my endpoint The Collection name where we are going to be doing the search is called website what is the input value this is basically what question do you want to be searching within that database what is the text that you're going to be searching on that database that is the question that the user is going to be entering so I'm going to go to chat input because we already have that and I'm going to be sending the input as well to that input value so notice that now the chat input it's been used twice we're sending that to the search on astrab but we're also using that as part of the prompt that's pretty cool so now I need to specify the embeddings and pay attention to this we need to use the same embedding model that we used to load the data in astb in the first place that is the open AI embeddings model so I'm going to just find it here open AI embeddings here it is and I just need to specify just make sure obviously that you're using the same model here I'm just using Ada 002 and the open AI key you just need to grab it and paste it here so here is my open AI key and I'm going to connect embeddings with this Okay cool so let me check again what's Happening Here I have astrab search I put the token the API in point the collection name is website I think that's the one that I created didn't I that Explorer yes website and I have the input value which is coming from the chat whatever the user enters as the question I have the embeddings using open AI embedding same model that I used before and the search type I'm going to use similarity there is an order option there you can keep it at similarity this is awesome so what astb is going to return are records so it's going to return the do documents that are the most similar to the question that we provided and those documents will be the context now instead of just keeping this context fixed that says Mary is Susanna's sister I'm going to connect this record to the context here and I think I have an entire flow now with everything we need to make this word Let's test let's click on the Rong button and now I can ask a question and see what happens so let me go to the website and I'm going to try to answer a question let's pick something general like what's the program about I'm going to ask a question and the gpt3 model if this works it's going to connect to Astro DB and it's going to return an answer the program is about teaching practical skills and insights to build machine learning systems that work in the real world this is awesome this is exactly information that's coming from the website let's go back to the website and do something else I don't know are live sessions recorded I even have that same question in the website so if I ask that here I should get an answer back okay waiting for the model to do this yes we record every live session you can decide when to attend classes live or catch up asynchronously later using the recording this is awesome so here is our flow we're not done yet but almost our entire flow I have not written a single line of code and I can answer questions from my website or any website or documents or files you can grab PDF files and you can load them into astrab and then search those files let's say you can do a lot without having to write a ton of code but then the next question is what do I do with this because this is running here on my laptop let's say I want to run that from an actual application where I can actually ask questions and return answers so let's do that that's going to be the only ity bitty piece of code that I'm going to write in this session here so notice there is an API button down here on your right side so if I click there landf flow is going to give you a bunch of different options to run this flow from outside llow you can see how you can call it using a Coral Post using the python API using python just python code that you can call directly from your from an application uh you can just create a chat widget and HTML and add it to a web page so there is a bunch of stuff that you can do here just with this screen so I'm going to do this I'm going to create a very simple very simple python application that is going to use this to answer the questions that we need so here is what I'm going to do I'm going to go to visual studio code and I have a folder and I'm going to create a new file and I'm going to call that new file sample. py okay it's blank right now I'm going to go to LF flow and copy all of this code and of course you need to be running this on a virtual environment that has access to LF flow obviously so you can see from LF flow. load import run flow from Json I'm going to format this file here are some tweaks I really don't like The Tweak name here for this constant this is basically if you want to pass certain settings certain or make certain tweaks to the different components of your flow you could just pass them here I'm not going to make any so I'm going to get rid of it so just the code is a little bit cleaner so I can just remove these tweaks now you can see that I'm going to be calling the flow and there is a name of the flow there I'm going to need that Json file so I'm going to grab that Json file in a second and here is the input value before I even do this I'm going to comment this out and I'm going to make sure I have something like question if I remember correctly I've never done this or haven't done it in a long time you can just use input to just grab something from python application let me run the python application now enter your question la la la la la and now I have the question right here so now I'm going to just turn this back on and I'm going to pass this input value I'm going to pass question right here so what's happening it says it's undefined oh you know what I just remove the the line here which I don't remember I'm just not going to even try to remove remember I'm going to grab it here okay we're back on track now I need this flow so you can go to land flow and you can download this Json flow so I'm going to come here I'm going to click export and that is going to give me this sample. Json file okay and that sample. Json is going to contain this entire flow something important when you click export it's going to ask you whether you want the API Keys embedded in that Json file so just be careful if you do that and you export those API Keys into the Json file and then you share that Json file other people will have access to that Json file just make sure you don't do that you can pass the API keys from outside using the tweaks uh I'm going to do it here just because it's easier for the video I'm just going to download this flow boom awesome and now I'm going to copy that flow to the folder where I'm running my L flow application let's go back to visual studio and now I have my flow right here that's the Json file that contains the flow and here I'm just running the flow and grabbing the result right here now I'm going to print the answer I never remember how to grab the answer I think is I'm going to have to grab the first uh output here but I don't remember what comes after that so I'm just going to even I'm not going to even try I'm just going to print whatever comes in here and then we figured out how to do the rest okay so let's now run the application and let's see what happens so if I run this it's importing land flow it's asking me enter your question and I'm going to say I don't know who is the instructor of the program that is my question and now is basically executing the flow trying to grab that question from the database and you can see here's the result the structure of the program in Santiago that's me so it's it's working and if I wanted to make this prettier you can see that I can grab result z. outputs and then grab maybe the first output uh let's see outputs and then grab the zero here do I need value no now this is fine let me try again so let me ask something else how many hours of recordings let's see what the model says results every session is recorded so there are a total of 18 hours of recordings blah blah blah okay so I need to print results that's what I need to print and now I this should be completely working the way I want it uh so what is the name of the program I don't know I don't even know if that information is there but let's let's just try what is the name of the program the name of the program is not explicit St ated in the context provided that is true that is true uh let me try something different but you can see now that now I'm getting just directly the answer from the flow that I'm running enter your question how many sessions are there by the way here I'm using GPT 3.5 turbo if you use a better model you're going to get better answers for sure you can see now there are six live sessions in the program which is which is true this is exactly the way it is this is working this is awesome so here I'm running the flow that answers questions if I wanted to maybe load information to my database frequently or periodically I could also just export the order flow that we created to load the data into the database and run that you know the sky is the limit very little code only just to run it from outside the application but you can even run it without doing that you can insert your whole flow and run it spot of an HTML application you don't even need to write any code I think this is just a great way to learn and to build applications without having to worry too much about code but if you like code something super cool about land flow is that you can also create your own custom components and specify the inputs that you want the component to show the outputs that you want the component to show you can connect those components with other components in landf flow you can do all of that behind LF flow they use l chain so it's really really powerful and while this is a very simple application like I said before landf flow supports all sort of components that you can combine to create agents complex chains Etc hopefully this video helps you out please give me a like or hit the like button below that helps the algorithm and it tells me that you really like this type of content and I'll see you in the next one

Channel: Underfitted

Views: 14,909

Keywords: Machine learning, artificial intelligence, data science, software engineering, mlops, software, development, ML, AI

Id: RWo4GDTZIsE

Length: 40min 52sec (2452 seconds)

Published: Mon Apr 22 2024