Build a Powerful PDF Q&A RAG System with Langflow, OpenAI & Chroma DB! (No Code/Low Code Tutorial)

hello everyone and welcome to another episode of generative geek today we are going to talk about langlow Lang flow is a web tool built in Python which allows you to build Lang chain applications Lang chain llm applications without writing a lot of code or in most cases no code right it they it gives you lot of composable blocks like these uh which you can then use to um build very interesting applications right uh I really loved Lang chain I have tried several tools but Lang flow is like you know amazing a lot of composable blocks are already available and in today's episode what we are going to build is this we'll have like a fully working chatbot using which you can actually query a PDF we'll also in the process take the PDF build chunks of the PDF using a recursive character text splitter then store those chunks in a chroma DB instance and then that chroma instance will be queried to build the context for our chatbot right so let's just say let me do a quick demo right uh how was infosis uh earnings this time right so uh my my chatbot has uh a PDF which is a transcript from infosis which is a publicly listed company and uh it now shares infosis earning for declined in revenue and I can do I can basically chat with this this PDF using the context that comes back to me uh through this entire pipeline that we'll build so this is the pipeline that we are going to build um and we'll we'll go through the entire thing step by step you will only be building these like you know we are not going to write a single line of code to build this right so let's get started so we'll first go to Lang flow. G and this is where you know you can see that the easiest way to create and share chatbots AI driven apps and things like that a wonderful tool but we don't need to join the weight list we directly need to go to the GitHub link so press the GitHub link it will open a new window for you and there what we have to do is we have to follow their instructions of how to get started so what they're asking us to do is they're saying hey you know what you should have python 3.10 or greater truth is I tried with python 3.12 and I was having some sort of challenges with the build so I downgraded to Python 3.10 and I'm going to show you using python 3.10 but my buddy he tried with 3.12 and on his system it worked all right so he was on a Windows I was on a Mac I don't know what the challenges were I did not debug that much uh but let's now get started so I'll open my visual studio code and we'll then create an environment install langlow and get started from there so I have my visual studio code open now what I'm going to do is just just create a new environment um let me just first create a file called main.py do a command shift p python create environment I want to create a VM and I'll select python 3.10 right I have various interpreters available but I'm going to select 3.10 here so once I do that it will create an environment for me and now what I'll do is I'll open Terminal with the terminal open now I will go back to the page page llow page and install the latest release right there is a significant difference in the stable version and the pre-release version I will be showing you the pre-release version demo I found that to be more intuitive uh so I'm going to use that right uh we'll type this in it will take a lot of time because there is a there are a lot of tons of dependencies if you see it's installing Lang chain it's doing something with pyop G2 which is basically um driver for post Grace so I don't know what all is there inside uh it's installing a ton of things let it install and once the installation is done I'll come back to you on how to run langlow so do this code along with me where you are installing things and we'll get started from there okay so the installation is all done I'll just type in clear and clear the prompt now what we need to do is again come back to the uh GitHub page and says run blank flow with python DM blank flow run I'll show you uh something that I learned in the process right so we we'll just say python dasm langlow run and when I do this um it's taking a little while hopefully it will all it will work post this it's saying starting L flow um there are situations where when you type python DM langlow run you will end up with issues like these right you know so I was earlier having some certain issue where uhuh okay so it's uh launching the Lang flow is launching but let me just show you the issue I was Landing in right so I was Landing in um issue something like you know uh under underscore main under underscore something like that uh where it was not able to find uh produces error things like that you know I was I was getting this very same error that you know L flow is a package and cannot be directly executed um in fact to this I even posted a comment some hours ago saying if you are on a pre-release just try typing langlow run so for me on this installation it has worked but if let's say you land up in an issue my recommendation is just type langlow run instead of python DM langlow run and you should be able to run it if you still don't are not able to run it come back to this issue type your comment and you know I'll try to help you there right so um now we go to this URL uh the URL is already opened we can see this is available here um this is where we are once your U once you have uh this thing installed Lang flow installed this will open up right so it's it's a beautiful UI and from here what we are going to do is we'll we'll get started with uh creating new flows right so we are building flows and flows are made up using components so we are not building components here we are building flows you will get to understand each of these things in just a while so I'll press start here and when you press start here you basically uh or let me just show you if you do a new project right it gives you a very nice uh templatized way of getting started that hey what is it that you want to do do you want to start building a vector store rag you want to build a Blog writer some sort of a document QA um we'll start with a blank flow right and once you start with a blank flow on your left you see all of these things right so there are uh there are inputs outputs if you look at inputs there's a chat input prompt text input for our use case as you can see on the screen now we'll first get started with we'll first take the PDF and we'll give this PDF to um a component which is basically going to implement the recursive character text splitter so it will chunk the PDF right so the PDF is going to get chunked and then using openi embeddings will store these embeddings into chromat TV so basically from PDF our chunks are now saved in chroma next what we'll do is we'll have a chat prompt and using the chat input tool we are going to ask user is going to ask a question whatever is the question that becomes your question right so and bases the question we are then going to send a similar do a similarity search on chroma DV and get the most similar chunks out from it this will become our context we are now going to add the context and the question to the prompt and maybe some history as well that hey whatever messages you have already exchanged We'll add that as well and we'll build a bigger prompt that prompt will be sent to open aai and whatever is the output that output will be displayed both in the chat in the chat text box as well as maybe I'll show you how other ways you can what are the other ways you can um display the output right so let's get started this is exactly what we are going to build without writing a single line of code so let's let me quickly bring make this slightly bigger and on this what you will see is there are a lot of components available on this left side right so all the things that you see on the left they are all the components right so first thing we'll do is we'll bring in some sort of a file reader because we need to take a PDF and that PDF is then going to be chunked into smaller pieces right that's what we are going to do so we have a file and you can even select a file here we'll do that in a while and now what I want to do is I want to bring some sort of of a text splitter right so I want to bring in um some kind of a recursive yeah recursive character text splitter and if you have seen my other videos where we actually had to code it by writing making lot of imports we had to specify the chunk size as a variable name and variable value and all of that here within this component you get all of that right so there has to be an input what are the separators if you don't give a separator it then defaults to some we can say hey you know what my chunk size has to be at least 2,000 and the chunk overlap has to be 200 and you can like you know customize this component so this is a component it needs to take an input so what we are going to do is I want this file to be input it to the recursive character text splitter so that now these things are then be the the the bigger file right you know maybe this is like a who knows 40 page file right you know 40 page PDF that will then be converted into smaller chunks right so that's what we are going to do now once this is done the next step is the load has happened the chunking has happened what you now want to do is you want to ingest them in a vector store right so the vector store that we are going to use is called chroma you would have seen other videos I have on chroma so we're going to make use of chroma and when you bring chroma earlier we had to do a pip install chroma we had to you know then import necessary libraries we had to let let the thing know what is the collection name what embeddings we are going to use here think of it all of that is abstracted all of that code is written this component has that code so we have to give the P um the name of the collection collection in chroma if you have not done any chroma video till now is like the table name or the database name which database do you want to use what is the name of the database where all of these chunks basically the embeddings of the chunks are going to get stored right so we'll say PDF chat and any collection in chroma needs at least two things one is the collection name the other is the embedding right what embedding model do you want to use um what we are going to do here is I'm now going to bring in an embedding model right so if you go embedding you see there are like tons of other embeddings available I'm going to use open AI embeddings right so we'll bring in this uh open AI embedding and I'll just uh put it here now when you use open AI embeddings what you have to do is you have to first select what is the model name right so all of these models have different characteristics different costs I'm just going with the default uh for demonstration purposes next I already have a global API variable if you don't have a variable you can create one I can come here and say hey open aior key that's my variable and it should be a credential right here you should now go out and paste whatever is your open aai key right which you can get from platform. open.com right so go to platform open.com and you should be able to see um various you should you should be able to see your keys or generate new keys from here uh you will have to obviously log in and then you can do that right so um it's fairly self- instructive just go there you will be able to build a key for yourself next once you have the key paste the value here I'm just going to pause this video here paste my key come back so my key is now pasted and I'm just going to save this variable right so I'm going to save this variable now I have open aore key which has the value of the key right and now what is going to happen is using this key it is going to make a call to this text embedding Ada 002 model and that's the embedding we are going to use so I'm just going to connect this embedding with chroma and this way now we have a collection name we have a embedding setup we can purist this somewhere so I can say chroma uncore store or chroma uncore DB that's my directory where I want to store this uh once this is done it also needs some input right so input in the sense what are what is it that it is ingesting now what it is ingesting is our chunks right so our chunks have to be ingested so we'll bring the chunks from the recursive character text splitter and input them bring them as an input for the chroma component right I know this might be slightly harder to visualize or to re uh to look at uh but that's the slight constraint with this drawing board kind of a setup right uh so what did we give we gave it the persisting persistence directory we gave it some embedding model um and we brought in the open I embeddings component we gave it the key and uh we connected all of them together so we are at this point where we took a PDF uh and we can even specify the PDF we can say hey you know what what what is the PDF so my PDF is uh in transcript infosis is a publicly traded company both on the NASDAQ and in India on the on the stock exchange is this is one of their transcript from a conference call I have just taken that you can take any PDF for that matter you can take a latest paper that has come out maybe some other Financial earnings whatever is of interest to you right maybe some book and follow the same process with this we are now at a stage where our PDF was chunked the chunks were embedded and that embedding has now got ingested into chroma DB now comes the part where we want to use this to build a chat agent with which which which using which we can then chat with the PDF right so we'll now get started with building the chat part of it right so if you look at it on the left side you have all of these components available and I'm just going to bring all of them now right you know so the chat chat input I want to bring I want to bring the prompt I also want to bring the text input right and you will you will understand what we'll do with each of them shortly uh the chat input is fairly standard you know what what it will be it is about the user typing in some message and that becoming the input right so the next is your prompt what I'll say is here you can actually Define your prompt so uh I want you to uh act like an expert uh uh at reading context information and giving answers basis the context right you are provided with the following context which which is context uh so we are saying you are provided with the context uh give a new line then you are provided with uh history which is nothing but message history um so whatever chats are happening we'll provide that as well so that it has some uh state that it maintains and um you have a question right but whatever the user types in that becomes our question right so uh and then I want to give it um um I want to ask it the answer answer and um I want to give it one standard uh disclaimer saying hey uh do not make things up use the context that is provided to answer the question uh use the context and history that is provided to answer the question if there is not enough information in the context and history then let the user no right so this is what I have this is the prompt that I have made and once you save this you can see that there are now three variables for us so there is a context there is a history there is a question right so the question is simply put whatever is the chat input right that becomes my question what about the context context will come from chroma DB right so the chroma DB thing that we just did we'll come back to the context in a while what about the history right and now for history what we what we need is some sort of memory uh if if you remember I had done a very detailed video on how do you add memory to chatbots this is that part right and but the simplified part of it uh we covered several ways of adding memory in that video but this is like hey how many messages do you want to like you know keep sending back as part of the prompt right but I'll stick with the default so we need to send back five messages so every time there are five messages this is like a rolling window five latest messages are going to get sent um you want to select just the machine and the user or only the machine I let it be like you know the machine and the user so that the machine has when the AI is looking at the entire thing it has enough information about what we were talking about right so uh then there is something called as a session ID now session ID is if you if you look at the info this is the session ID of the chat history so these states are maintained at a some there's some sort of an identifier to the state and what we can do is whatever is the text input we are giving here uh we can let that be B basically the the uh we can say uh ID and whatever ID we enter here that becomes the session ID right so uh that becomes the session ID for uh for this uh user right so sender name uh is user or you can even say you I can even bring another text input and we can say hey uh we'll inform the bot about what is our name right um so the text input is username um so we have now we have now like you know we have two uh inputs so the one input is about the usern name what what is the name of the user who is chatting with him with the bot that becomes your sender name and the second is where we are giving it some sort of an ID and that ID becomes the chat memory I could have still I I could have just used both of them as same I could have just brought this thing here and that also is valid right so it only accepts one input so um maybe not one input uh even this can be brought here right so so why confuse things we can say hey whatever is your name that becomes the session ID and that also becomes the sender name right so so this way we can connect things up so you take it's a simple drag and drop connect kind of a setup so now what we have is we have our chat input so we have connected and now whatever is the memory we'll connect that with history right so this becomes my history now I need to get to the point where we'll have context coming in so for context I need to bring in chroma again so we'll say chroma and you can see earlier we took the chroma Vector store now I'm bringing in chroma Vector search right because now we are searching whatever is the user typing I'm searching that right so so this becomes my chroma searge and you need to again let's just go back up for a minute and see what was it when we were ingesting things we we had the collection name as PDF chat and our persistence persisting persistent directory was chroma DB we have to use the same things here now you know when we are setting when we are now again going to search back from the database I have to search in the same database and I have to search in the same directory I can't have like you know search somewhere else so so let's set this up so our collection name is called PDF unor chat uh uh and our index directory was chroma uncore I'm sorry chroma DB is our um persistent directory so that becomes our index directory now and um the input is basically whatever the user types that becomes the input um because that's whatever is the user typing in that is now the question right so that is Basis which the search has to be done um so I'm just bringing this here uh sorry this might not look too intuitive but this is how it is yeah so if you look at look at it there was a chat input uh whatever is the user typing here whatever message that message we are taking in and making it as a question and also using that message to search in chroma right so if the user types in how much revenue did infosis do in the last quarter what what is going to happen behind the scene is that that message that input is going to be embedded and in this collection and in this directory it is now going to sech for most similar results so we going to say search type as similarity right it is going to pick whatever is most similar and it is going to give that output to us so now I have to also connect embedding um because and you you have to be always sure that you know when you when you ingest whatever embedding you use the same embedding you have to use here whatever same model and the same uh same model has to be used I'm just I can launch Another Kind component but I'm bringing in the same component just to show you that you know the same component can be connected again again right you know so so this way our chroma search is ready now if you see my embedding is connected with the same open AI embedding that we have above which we were using when we were ingesting uh I have brought in the same output here now I'm what I'm doing is I have everything connected right so the result now becomes my context so I'll pick the result come back into the prompt and connect it with the context right so if you look at this is how it looks now right my context is connected with chroma search uh my history is connected with uh chat memory and my question is connected with chat input right so with this now my prompt is complete now this prompt has to be sent to openi so let's just search for open a and I have this model here which is open AI I'm going to bring it here we need to give it the key you can select what model you want to interact with we'll let it be gbt 40 uh you can say gbt 3.5 turbo I'm just going to stick with gp40 and we're going to say open AI key what should the temperature be temperature will Define how much creative you want the model to become I don't want it to become too creative at this point and the input for us is just the prompt template right you know whatever template we had we're giving that as an input next what I want to do is I want to show this output somewhere whatever the model will output I want that output to be shown so we'll go to outputs and uh we'll say ch output right so I want the chat output to be done and you just connect uh sorry you just connect sender name is AI and you just connect it with the message right so so that's what um this is how you connect and this this way if you look at it my entire thing is now ready right so we started with the file uh we took a file we brought it to the recursive character text splitter we said 20 is the chunk size and then you have an open embeddings that we used and we sent those embeddings to chroma DB uh next we brought in the entire Chat thing and we said hey what is the username and we use that username as both the sender name and the session ID for the history we're going to use five messages from the history then we built a prompt template where we said this is going to be our prompt and when we declared these variables we saw that you know automatically three new VAR Ables came up for connections right so we could connect these variables here you can have even more detailed prompt template where you can have tons of variables and those tons of things will start showing up here uh next we took this output sent it to open aai and whatever came back from openi we then pasted here as chat output right now let's just try this entire thing out so once you are ready we'll press the playground button and we have to enter the username so I'll say generative geek uh this becomes my uh ID now if you remember and we'll just say uh how is infosis looking at the entire generative AI landscape and the opportunities uh it brings right so um basically infosis is um is an IT services company and there is not information in the provided context and message history to answer the question about how infosis is looking at the entire landscape and the opportunities it brings now uh one thing that we can do is in case if let's just say let's type another thing uh What uh is the future uh share some Mega deals some deal wins right from uh they talked about share some deal wins they talked about let's see what the what the model comes back with um we have asked the model to specifically output only from the context so if let's say there's not enough information in the context it will not talk about that based on the provided context and history here are some details the company reported very strong wins amounting 3.2 billion um 71% and all of that right so uh uh 71% of the 3.2 in Deal wins these various sap and all of that so so you can see right you know this way your chat bot is now ready what we can even do is uh just from a debugging perspective right if you are interested uh there is this this context that comes back we can even have it outputed in some sort of a text output right so I can have some sort of a text output which will display what is the context that is that is getting fetched and getting passed into the prompt template or I can even have the entire prompt template why only the context I can even have the entire prompt template uh um displayed right so instead of this maybe I'll connect the entire promt template uh into a into a text output right so uh so let's see now with this connection let's turn the uh thing again and you see an output coming up here right so this is empty for now um I have generative geek here which is basically now five last messages should be part of it um is there any update from the retail sector right so I'm just piing things up and asking it I have not read the PDF myself um but I know that as an IT services company they probably have a lot of clients in various sectors uh messages say there is no specific update or information regarding the discussion primar evolves on overall demand environment client spending Behavior digital transformation programs cost efficiency but do me retail sector specifically right now let's look at the output right um so if you look at it I want you to act like an expert at reading context information and if you look at it this is what the context this slend context external document um it is better now please go ahead so this is what it is fetching uh and this is the context um still at the context I don't see okay there is questions I don't see history right so the History part doesn't seem to be showing up and why is it so so good now when you start looking at it um it makes sense for us to now go back to it and ask and check why is the history not showing up right so uh let's again see what was it that we did we had the history come up from here this was the chat memory retrieve stored chat messages given a specific session ID and the session ID was our text right so whatever value we gave um that was the history uh let's again try it out I'll just change the input now so I'll say instead of generative gig change the ID because the ID now changes is that means um uh how are you how is infosis looking at margin enhancement right so we are trying to debug why we did not see the uh the output last time so uh we did not see uh anything on uh message history okay I can see the message history show up um let me just copy this uh and bring it on to some sort of a my maybe my uh uh I'll just close this here and we can look at history right so if you look at it the history now has uh it's still blank right the message history is still blank uh because this is probably the first message let's try more uh let's see what comes back after I type uh uh so it gave us some output on utiliz ization internal programs uh is there more info on pricing and value based selling right so let's just ask it some question right so I'm just look I'm more keen on understanding if the history has any uh data or not right so uh how is infosis okay so now let me just go back to the text output copy this again and uh come back to my thing paste this and see if history yeah so history now has is there more info on history is still coming blank and um that's slightly surprising so this means we are not sending any history right so we are only sending um we are only sending um the context and we are sending the question but we are not sending any history uh so okay uh I'll give this thing some uh debug and see why we are not sending history uh maybe instead of this have another text output just on the history and see why is the history not showing up right um it's slightly surprising uh history should have shown up I'm just connected another uh text output to history just to see right so we see now two outputs I'll maybe change the name here and say this is history so that I know uh so you see this is history now um and this is empty for now so let's just type uh tell me about uh specific specific clients and segments right so we I'm just trying to debug I'll just not take too much of your time uh maybe the last of the sessions on how to debug uh so we'll go to the outputs we'll see history nothing comes out uh it's empty uh which is surprising right so so okay maybe because this says this is still in beta maybe there some error with the component itself um or let's just say if I only want to send the user does it change anything right so uh hi right so I just want to see if history has anything or not yeah so far nothing the history thing seems blank and uh no right I'm just typing something just to see if history has anything because history ideally should have um something going back but no I don't see still don't see anything going into the history so so we leave it leave it at this this is what we built today um maybe the History part did not go out that well um but other than the history so this is a stateless bot right now uh and if there is any other memory no there's only chat memory right so so if uh instead of uh you know having a memory this is like a stateless bot uh which we built we took a file we ingested the file file um sorry this is like you know it gets messy so we took a file we chunked the file we use openi embeddings embedded them into chroma DB chroma DB and then we came back and we built the entire chat workflow right so we could have done now the way it is even if you don't have the chat memory block it will still work fine right so uh so this is what we built um I hope you learned something new please share your comments in the comment box and let me know how you how it went for you

Channel: Generative Geek

Views: 306

Keywords:

Id: NyQK1_0e660

Length: 33min 30sec (2010 seconds)

Published: Sat Jun 01 2024