Advanced RAG- Langchain and HuggingFace

hey everyone welcome back to this video series on building rat pipeline using open source large language models part two in this video we will dive deep into Advance ra topic that includes parent document retriever for year reing and land expression language so let's get started the very first step is we need to do the installation we have L chain we need to install chroma DB and one extra library that we need to install in part is unstructured PDF the reason why we are building unstructured PDF is in this particular use case in order to understand p and documentary D it will be good enough if you use some kind of complex PDF or document and this is where we have instructed PDF so how is unstructured PDF pred from PDF whenever you use a PDF loader there might be chances that whatever image data you have you're unable to extract the text from it and in unstructured PDF to some extent you will be able to extract text from the image so we will use unstructured PDF as our loader to extract the data and load it in our chk so let's quickly get started uh the first step is very simple we need to define the document loader which is unstructured PDF loader and then comes our data pre-processing where we do split in short chuny text split or from there we need to import recursive character text spitter and the next thing is we need to create the embeddings in order to avoid time to load our embedding model we use something called as pugging P inference API embeddings and once we have the embed we store that in Vector store which is our Vector database in this case it is chroma and next we have our language model from Lang chain we need to use LMS and import aing Face up that's it okay I made a spelling mistake it should be Ling we have few more statements that we are supposed to do in order to implement the advance rack but so far let's just use the components that we have used in our previous video let me quickly upload our data path and in order to have the data path let's upload our resume which is a PDF format let me click open and it's very simple in order to upload your PDF file either you can connect with your uh drive or you can just click on upload by clicking on this directory icon upload and upload your PDF file from your local directory once it is uploaded create a file path and what is my file path my file path is th Jane resumer PDF so this becomes my part of my data and the next step is we need to load it so in order to load we need to First call unstructured PDF loader it's unstructured PDF loader pass the file part and then I need to extract the content how do you extract you just use a load function and once it loads uh the entire content from my resume it will be extracted and it will be stored in a variable called content it's going to take few seconds and it will complete the process uh it's using nltk data so that it can extract all the tokenization and we have few more other techniques or using which it is able to extract the content so let's print what is inside our content when whenever we load any data it is in document format and we need to check the page content we need to check the page content so as you can see we have entire data of my resume over here cool cool so now let's proceed with the next step of uh data splitting but before we touch Bas on data splitting I need to use embeddings the reason why I'm delaying the process of using data splitting is because parent document R is somewhat related to data splitting and that's the reason why we are not going with data splitting as of now so let's quickly initialize ize our embed and in order to use our embed we need our aing pH access stoer so let's quickly get our a space access open we Define a variable called HF toer from where we get our get pass we will also store it in an environment variable so that we can later use it in a large language model so it should be aling face API it should be aing face up I'll just copy copy paste it should be again face upore aior toer and I'll just Define it with HF toer and you already know your you already know the step on how you can get your access toer just go to a.co click on settings click on access topen create new and just copy it I'm just copying one of the previously created topen and come back and paste my particular access topen so I've already added my access topen now the next step is I'll create my embeddings I'll create my embeddings which is huging P hugging face inflence and and we need to define the API key the API key is nothing but HF token and one more thing is we need to Define our model name and in the previous video we used the NLP larger which is GTE large that's it so this will create our embeddings now The Next Step will be creating the vector store and let's recall what we did in last video for Vector store we need to connect the document with embedding but in this case we are not connecting the uh document as of now because the document will be created once we Define the retriever which is parent document retriever so now we will only Define our Vector store using our embedding function that's it so we Define our chroma only with our embedding function only with our embedding function which is embeddings that's it we are not using any document we are only providing embeddings to our Vector store that's it nothing else so now let me explain you what is parent document retri we have the external document the external document is further divided into text splitting which is normal chunking in parent document retri what appren is we are splitting our text into a parent chunk which is further divided into a child chunk and whenever we do wey to get the retrieved context it will first check in the child uh chunk and then it will return the original Parent chunk so that we have the entire context in our llm so this is how parent document retriever works and I'll also tell you with the commands first we have the data then we create the parent Chun then you can pass in the child CH so this is one step this is one step that you can do and the another step is you can directly Pass the full document you can directly Pass the full document to the child son but this doesn't make sense to be honest uh most of the time why do we need this is because there are times when most of the data based on its larger context it is getting lost when we are doing the embeddings that's the reason why we need to create the parent sum and child s so that we have little bit of uh better use of our parent document retriever so let's go with the first step and let's create two splitter not just one rather two so the first one will be parent splitter and the second one with child splitter and for the both the case we will use recursive I'll just wait so that it will auto complete and here what you can do is you can Define the chunk size to be 512 and since we're creating two different CH I don't think we need CH Co so I'll Define some lab to be zero but again this is experimentation you can experiment and see if chanco lab is doing anything significant in this particular retriever but as of now I don't need uh I don't need any Chun Cola I'll just Define my CH size to be 200 and my CH overlap will be zero again um when I run this particular code for parent document retriever I will change this chunk size so that it will give you a better ination what is happening behind the seed just like what we did last time when we inspected what is happening with chab zero chab 20 in this video we will inspect what will happen if I increase my chunk size which is parent splitter and what will happen if I decrease it so it's going to be very interesting uh when we play around with this Tex splitting so let's run this code now we Define our first step which is retriever uh how we Define retriever we just call our parent document rri but before we jump into parent document rri when we are doing any text spitting there should be a memory nothing but a storage where we store this chunk and this is where we need to install or use Lang chain. storage Import in memory uh so we are using this particular storage so that we can store the chunk and now we need to Define our retriever we need to Define sorry it's a spelling mistake my bad Lang chain dot we have retrievers there is retrievers import parent document retriever so let's define the retriever in order to Define retriever we first initialize it and then Define the hper parameters so we have Vector store Vector store is nothing but what did we Define we defined it as Vector DB then we need to also Define the storage in our case the dock store in our case the dock store is nothing but in memory store I just copy this uh in memory in memory store I'll just call it and now we Define our child splitter I'll copy this again and then I Define my parent split that's it so I have defined my retriever now okay so what is this sorry I didn't use comma my bad that's it we have defined our retriever but there is something fishy what we did was we defined data we loaded our data why is it not scrolling my okay so we defined the data we created the embedding we also created a vector store we created splitting of our data stored that in memory but where are we passing the content so far if you look at Vector store in Vector store we have only embeds and in TT spitting also we have only chunk size in parent document ment in also we are only defining the storage and splitting but so far we never added what is our content so this is our actual content I'll just remove this this is our actual content we need to add this content in our retriever so what we do is we have a special function called retriever do add document uh it's not add document it is documents and I'll just write content and we have IDs to be none that's it so it's going to take few seconds uh until then we'll probably proceed with coer now coer is a again platform where it is famous for its schematic search and it also adds a feature where you can rerun now why do we need R let's imagine you ask a pro saying what is black hole in your document there might be cases where this might be repeated multiple times and you have four keyword search we have K value right K value so we have search keyword documents which is rning for R context and your actual answer might be in third or fourth and what reranking does is it will compress your particular context and it will rerank and it will also r a threshold value a threshold value is nothing but this for using which your retrieve context is run this is where we have ranking which will rerun your relevant context based on its priority so let's use uh the retriever uh but before we use coer let me tell you why we need it so that it will give you more intuition so let me quickly use this Retriever and ask few queries so we will test with few queries uh so that it will also give you understanding of how parent text splitter is working and character text splitter is working once we understand this then we will probably proceed with ranking so let's play around I'll ask what is the ruins role in y Planet okay so this is my query this is my query now what I'll do is I'll print print Retriever doget get relevant documents qu so as you can see it is returning my details where it is mentioning what I have done at the panel uh good great uh now what I'll do is I will try to increase is my retriever size I don't know if we can do that let me try retriever dokal to 3 okay we can do that no worries so let's only play around with what you call uh text Splitters so that we can see how it works I'll bet a variable and then I'll use response so that it will give me in okay now is not given so let me directly print response this is good so as you can see we have two documents we have two documents which is getting retrieved uh you can look at the length of it now what I'll do is I will reduce my spy 12 to be 250 and see whether there is any change and what we are trying to check is whether parent parent retriever is parent splitting is getting retrieved or child is getting retrieved so that's the reason why I only changed the chunk size of parent I didn't change it for child so let's see what is getting generated so I'll run this statement and I'll print response here so that we can compare the result so as you can see if I reduced my size of parent Tex splitter the context which is getting retri is reduced so now what is happening is uh the First Data we have it is converting that into a parent splitter that is getting converted into child splitter whenever I ask a query first it will look into child chunk and based on child child chunk if I have my query in that particular chunk it will return the parent chunking as you can see I reduced the size of parent and this is what it is getting retrieved so also let's change uh let's revert it back I'll make it fight well and I will make this particular CH size to be 100 let's make the chall size to be 100 which is SH now let me print the response this should be similar to this one so as you can see we have the larger context but it's only retrieving one different document it's not retrieving two that means this is working better when I'm giving some size to be 100 100 this is much better than 200 but if you look at the length of it this is similar to this one so the return context is parent exp splitter but the actual uh inspection which is going on if we have to verify it is child splitter so I hope that gives you an ination uh just play around with response and look into the chunk size where change your chunk and look into the responses so let's get started with reranking I'll just write rerank again we are supposed to import few uh statements few components let me check it out I don't remember things uh the first thing is we need to install pip install po but I've already done it I already I already installed ker so let me quickly just import it from for here I need to import plant and then we need to import retrievers we need to import retriever from Ling we need to import retrievers with which is contextual compression Retriever and from L chain again retriever we have something called as document compressor inside document compressor we have something called as coer aner these are the three step we need to install now what we need to do is we need to get our epi key so what you can do is you can head out to dashboard. for.com API keys and you can copy you can copy the default one once you copy the default one come back to your uh po notebook so we have already copied so what I'll do is I'll just copy the statement 4 equals to for plant and I need to Define my API key over here API key again I don't want this to be visible to anyone so what I'll do is I'll write goer API equals to get pass run it and I'll copy my API key and copy paste since we are using collar modbook we might end up with issues with P dantic so there is some custom corer rerun code which is available on GitHub I'll probably paste it over here and I'll read line by line so this particular code I poy it from GitHub which was an issue r. L chain we are supposed to use pantic which is a base model and inside base model we need to make this arbitrary type aot to be through and then use this forward D to update our poer ranking so once you update it we have created the class name as custom poer ranker we need to call it we just need to call it since uh Boer rank is a document compressor we are defining uh the variable to be compressor and for custom for your run and client to be Co so this Co is nothing but this Co here all right uh so I guess we are supposed to create an environment variable my b. environment variable it should be Co API k equals topi okay I hope this should work all right it's not working why did not spine for API key I guess we have already defined it okay my bad uh the K was small letter I've just uh copied it from here and I pasted it so it's the same thing the only thing was I made this y to be small letter I'll just reun this it should work it worked so now once we have the compressor we need to define the compress compressed R compressed retriever which is contextual compressor Retriever and we need to Define our compressor as base compressor which is Po and then we need to Define what is our V Retriever and V retriever is our parent document retriever so let me copy this my base retriever base retriever is nothing but coer I mean parent document retri so now we have already defined two Advance r one is parent document retri and we have connected that with our Co ranking now we are only left with the last part of our board which is Lang chain expression U language and since we have already defined redri our first part of rag is done retrieval is done second part is augment and for augment we need to create a template for our PR so we will use the same prom template that we used in part one I just hope you have watched the part one before watching part two so I'm using the same template from my previous video you can check out the link in the description as well for the part one and now we will Define the template from Lang chain PRS import it should be chat prom template but I'm not sure with this sping so I'll probably wait should Meer from template yes it is now what I need to do is I need to Define my prompt I need to Define my prompt from the template and if in case you need this particular prompt you need to check out the previous video the only extra thing what we are adding is this particular context that's it in my previous video this is how the promp will look like in this video this is how the br will look like we are just adding a extra line which is context so now from chat BR template round template I will Define whatever variable I have used here which is template so we have defined our template and we have two input variables one input variable is the query one input variable is the context so context we will fetch from the retriever query is what user will enter okay so we have completed Two Step retriever is done augment is done the last step is large language model so let's define our model again P up and we need to Define what model name we'll just copy the Zar model which is the reposit ID all right so from depository which is a H4 zire 7B model which is completely open source and we will Define temperature to be 0.1 in this case and maximum new topens what should be the length of your response it can be F well you can also make it 256 so let's this all right so it's probably working I hope we have created the environment variable let me verify it all right we have created the environment variable uh so we can run our model let's talk about uh land chain expression language L chain expression language in short it is also called as LC so this is something new which was introduced lately so what this particular expression language does is it can create fallbacks it can create bads it can create uh streaming so everything becomes very simple uh to build a complex chain and since you know L chain is made of multiple complexing with expression language it becomes very simple in order to run this and we have import statement which we need to use and it's very simple there two things that we need to use one is land chain do runnables uh we have something called as runable pass through uh this is something relevant to chain uh using which you can Implement expression language it's something called as runnable I'm not sure what is it should be runable pass let we gos it is runable pass through all right why is it giving me an error sorry my bad it should be lunching for dobles I guess this should work yeah it is from lanin core. rebles import renable pass through and we have one more structure so whatever response should be generated it should have certain structure where we have lch dot uh we should have parer output parer and from output parer we need to import string which is my output parsel so once you define your output parcel not again okay I guess there is some issues with Ling I don't know why but usually output parser should work directly when you use lch output parser but these days You're supposed to use l chenore code previously I have Abit of using L chain. output parer but it should be L chain _ po but that is fine now we will Define chain we will Define chain in the expression language so it's very simple uh we need to create the input variable so let me see what are my input variable we app cont text V app qu so I have context I have context which is my uh what is my retriever my retriever is comess Retriever and I also have qu qu is my second input variable and quy will be whatever user enter and that's where we will use run pass through so what run pass through will do is just pass that's it it's like normal python pass and when you run this flow you just have to invoke it I'll show you need to invoke it but for time being since we don't have user query we will run runable pass through so that we can continue with the chain and in order to create the chain you just need to use this particular icon uh I don't know what this icon is called it is just a exclamatory Mar but without doubt uh just a standing line I I don't recall what this icon is or what this special symbol is U it's next to your curly brackets just a line that's it and what is our first thing the first thing is prom the second thing is our model and the next thing is whatever our output parser should be so the output parser is string output parser function so this should do that's it guys we have created the chain and we have implemented three rag Advanced rag components so now let's finally invoke it our final response chain do invoke I will ask what is pars Ro at Planet let's see if it can answer or not I hope it doesn't take to was of time but let's just wait uh meanwhile guys please do comment it down if in case you are able to follow uh the part one and part two and all the source code will be available on the awesome brag the GitHub repository that I have created and if in case you have any more queries on future videos that you need please do comment it down it really helps me to create the content and do subscribe and like the video and obviously we wait for the response I'll show the response and if in case we have to debug something we'll do that but hopefully it should give the response it should give the right answer so let's just wait resp says done and whenever we use runable pass through for user input we need to invoke it this is what I've have done I just hope it gives me a good result tun's role at TI planet is developer relations and Community manager that's accurate so that's what is said in my content so I'll just print the content so where is my content where is my content yeah it's here so I was part of Google sum of Bo relation developer relations and Community Management this is what I've added great so let's ask one more query let's let's ask one more query before we leave this video so I'll use print chain. invoke what did tun do at Google somewhere of so let's see if it is able to answer it from my res according to the given context tun worked on an entirely new project exactly true I built a new project for our organization which is C microscope during his Google Su contribution p in L Domine uh that's exactly true so this is it guys uh please do subscribe and like the video Until then let's meet in the part three

Channel: AI With Tarun

Views: 4,059

Keywords: ai, langchain, genai, generativeai, largelanguagemodels, machinelearning, python, ai python, llm, huggingface, advanced rag, open source

Id: oUQHBNVVEH4

Length: 35min 50sec (2150 seconds)

Published: Thu Jan 11 2024