8-Building Gen AI Powered App Using Langchain And Huggingface And Mistral

hello all my name is krishak and welcome to my YouTube channel so guys uh I hope you are loving the entire langin series we have almost completed almost each and every module specifically to langin that is actually required to create end to endend projects and we have also developed multiple projects over here uh we have used both paid and open source llm models now one of the most requested video was that Krish how can we work uh considering Lang chin along with hugging face libraries and making sure that you use only open source llm models and try to create some amazing endtoend projects so in this video I'm going to use Lang chain hugging face and open source llm models like mral that is specifically hosted in hugging face and then we'll try to develop a Q&A rag complete rag app and uh I hope you will be liking this particular video please make sure that you watch this video till the end because there will be a lot of learnings that you'll be able to find from this particular video too so let me quickly go ahead and share my screen so here you can actually see I've created a folder called as hugging face and there are some us sensus data which I probably downloaded from the internet in the form of PDF and I have uploaded in this specific folder right so we will go ahead and probably read this again we we are going to use completely open source LM models like mistr using hugging face itself and then we are going to create the entire Q&A application so let's quickly go ahead and do this and initially uh as you know that initially we need to load all the data injection step that is loading all the library all all the PDFs itself so for that I will be importing from Lang chain uncore Community okay do document loaders import Pi PDF loader okay so this is one of the uh things that I'll be requiring then from Lang chain undor community. doent loaders I'm also going to make sure that I use a PDF directory also for that so here I'm going to import Pi PDF directory loader okay and then from since I after reading it I need to do the character splittings uh divide that documents into chunks so for that I'm going to use recursive character splitter along with this again as we know that we'll also use some kind of vector store over here so I'm going to use the vector store called as F and uh as you all know we also going to use hugging face so for hugging face as I said all the embedding techniques also I'm going to use from hugging face itself so for that in langin also you have this embedding techniques where you have hugging face uh BG embedding which is completely an open source and which is good enough right so we also going to use it um along with this I'm also going to import numpy as NP if I specifically require numpy I don't think so I will be requiring but I'm going to just use this okay um two more amazing libraries that I'm going to use from lank chain since I also have to make sure that I create a template so from Lang chain. prompts I'm going to import prompt template okay so this is also what is specifically required and one more right one more uh chains I will specifically require again from L chain dot chains UT retrieval QA right so retrieval QA chain we will specifically be requiring so all these libraries we are basically imported uh and we are going to use this for this particular purpose now let's go read the PDFs from the folder okay now here you have loader is equal to Pi PDF directory loader and here I'm going to give my sensus folder right so here I will go ahead and write dot start usor sensus okay so this sensus folder I'm going to specifically read and then I'm going to use this loader do load and I'm going to split uh I'm going to basically get my documents okay so once I probably get the documents then what I will do I will create my text spitter and for this I'm going to use my recursive character text splitter and for this uh the chunk size chunk uncore size I will be taking 1,000 okay and chunk chunk uncore overlap will be somewhere around 200 okay so all these things as you all know I've written this kind of code a lot uh and then finally my final documents okay will be nothing but I will be using this text splitter and then I will split all the documents so I will write split all the documents uh with considering all my documents over here and let's go ahead and see my final documents first final document I'll just try to see I don't have to display the entire thing so uh this will probably read all the PDF file inside that particular us sensus folder which is present over here all the four PDF files and then it will probably give me a thing so here you can see one p error is there so here let me go ahead and write it chunk size now I think it should be working fine and all the PDF all the materials will be available in the description of this particular video guys for you all of you right so here you can see and the total number of length of final documents you can probably see is somewhere around 316 so here you can see page content hand insurance cover status everything is available over here and this is the total number of documents that I'm actually able to get it okay now uh the next thing that I will be specifically requiring is the embedding technique what embedding technique I'm going to use as I said I'll be creating hugging or let me just go ahead and write see you can use AMA embedding anything but since I am planning to do it with uh embedding using hugging face so I will be using all the hugging face components and these all are also very good completely open source and available for you from hugging face so here I'm going to basically write hugging phase embeddings and uh the next thing that I will specifically require is is my model underscore name this is the first parameter that I will be giving and then model number underscore name will be nothing but I have explored some of the model over here that is this one right by this one is also there okay so one embeding technique is this one I can also use uh sentence Transformers okay so let me just show you instead of using this I can also use this so I'll write down a comment over here oops just a second I'll just write down the comment over here okay so this also you can specifically use let's go ahead and use sentence Transformers if you want okay and uh if you don't want you can also use this no worries it is up to you okay both any of them can be specifically used it is up to you okay and these are some of the free available embedding techniques that is basically available okay so now I have actually created this I will go ahead and write my next one so before that let me do one thing okay and okay so the model name is driven then I will be using model KW R GS and I know I don't have to even use GPU right because with the help of GPU with the help of CPU this is possible because I have very less documents so I'll be using device device colon CPU okay so this will basically be my device with respect to this model arguments col okay the next parameter that I'm going to specifically use encode KW R GS and encode _ KW RGS will be here I'm going to use a parameter which is called as normalize so you may be thinking Krish from where are you probably considering this I'm just checking the documentation from the hugging phase itself right so if you go ahead and search for this right and you'll be also able to see some of the examples right so here you can see this okay so here you just click on this and here you'll be able to get all the information there will be some code files also which you'll be able to see entrance Transformer is also there see everything is over here right all mini this one only I told right so similarly you can probably go and check for many of if you want okay now the next thing that you really need to do is that you have to set your access token of hugging face unless and in order to access all these models hugging face token needs to be there so I will go and click on settings in my hugging face account there'll be something called an excess token okay and just make sure that you create this particular new token so once you probably create it you'll be able to see it okay uh and then you can copy it and you can specifically use it but anyhow I will show you where we'll be specifically using now if I go ahead and execute it so here could not import sent Transformer P install sentence Transformers okay I need to install sentence Transformers so let me quickly go ahead and update my requirement. txt file see whenever error comes some type of things is always going to happen but it's okay we should not be afraid of Errors you know errors will probably help us to do anything that we want okay P install minus r requirement. txt so here you can see now the sentence Transformers will also get installed and finally you'll be able to see that okay everything is getting installed itself okay perfect perfect perfect perfect sentence Transformers is coming over here and now it'll be able you'll be able to see that okay on once it is installed then I think then you'll be able to execute this okay so we'll wait till this installation basically happens so if you have not subscribed and please make sure that you write something in the comment section how are all these videos for you all are you able to understand I hope many many people are a see whatever feedback that I'm getting from people with respect to the transition I feel that yes the videos are doing something in your life at least some amazing things in your life and I hope to do as long as I can okay but again my main aim is to teach you in such a ways that you learn learn learn learn things in a way that always it matters in your life right and that is what I'm actually looking at okay so let's wait uh sometime sometime sometime some amount of time usually takes for this particular installation but I think once this installation takes place we are good to go okay and I'll also show you how we can actually set up the hugging face you have actually created the hugging face tokens right but uh how to set it up as an environment variable still it is saying that it is not installed yes installation is taking some amount of time so let me pause the video till then I think it is going to take some time and then we will continue so guys the installation is taken place now you can go ahead and execute it and I think it should not give me an error now and your embeding hugging fish should be imported over here okay so perfect uh let's see uh it is going to take some amount of time again it is going to take somewhere around at least 1 minute because this particular embeding is quite big oh 13.1 seconds this is absolutely amazing and this is really good now after you have this hugging face iddings we will try to use this hanging phas embeddings and try to just check on one of the file okay so I'll just go ahead and write embed uncore just to check whether it is working fine or not emid query and here I'm going to basically take my final underscore documents of zero okay so I'll just take the first one okay first one um and uh probably I will just go ahead and use the pageor content because it will be available over here uh now what I'm actually going to do if I'm specifically using these hugging face embeddings okay I will also go ahead and import numpy let's see import numi as NP and let me convert this into an array np. array okay I think so it should not be giving him an error so we are just trying to see okay perfect so whatever was available in the first document of page content this is how the embedding is basically shown for the entire words okay whatever the words are specific specifically present in that particular embedding and uh if you probably go ahead and see this what is the shape of this particular array okay I will also print that specific shape okay let's go ahead and print this first print this first okay and then we will print it I will paste this do shape okay I think you'll be able to see the shape 384 okay and it is also getting printed so perfect now what we are going to do uh if this embedding technique is working fine then what we'll do we'll convert for uh the words into vectors by using this embedding technique and store it in a vector store so for that already have imported FES so I'm going to probably create my Vector store and I'm going to write my f do and then I'm going to say from documents uh did I import FES yeah it is imported from documents and from the documents uh what I'm actually going to do is that I'm going to give my final documents my final documents till let's say I give 120 records okay there are somewhere on 310 records uh I'm not putting all because it is going to take time okay just to show it to you but later on once once you have time you can go ahead and increase this number and here I'm going to basically use hugging face EMB biddings okay so Vector store will basically get created so if I go ahead and execute this again it is going to take some amount of time at least 30 to 40 seconds because there are so many documents over here and all this will be embedding right it'll get embedded into words and as you can understand every document will be getting converted into 384 based on the chunks that we have specifically 38 for vectors okay so this will basically be my this will be my Vector store creation okay and this is what is the step that we have specifically done in every step right Vector store creation perfect so let's wait for some time and I think it is 30 seconds it has crossed since uh 120 before uh when I tried to do it it took me around 30 seconds because there were only 100 documents okay so 43.4 seconds perfect now we will try to use this Vector store and perform query using similarity search okay so similarity search will be basically done over here similarity search will be done okay and once we specifically do the similarity search we will be able to see or we'll be able to get some kind of response over here okay so how to do the similarity search first of all I will go ahead and write my query uh with respect to the query you'll be able to see I will just write a question what is the health insurance okay now see this question that you'll be seeing I did not just write it by purposely or somewhere uh uh here uh what I I will be doing withit what is the health insurance insurance coverage okay I what I did is that I just went to this folder okay and uh when I probably go ahead and see this folder I will just open my first file okay and this is where I show this question what is health insurance coverage so I will take up this question itself okay and I will paste it over here okay now once I paste it so this is the query that I'm actually searching I will go ahead and write relevant uncore documents since I need to probably retrieve that result that is present over there so what I will do I will use Vector store Vector store dot similarity similarity undor search okay and whatever query I have specifically written okay whatever query I have written and let's go ahead and and print all the relevant documents let's go ahead and print all the relevant documents so once I print it I think I should be able to oh relevant relevant relevant relevant is this not the same thing okay so here you can see page two what is health insurance and here you'll be able to see the answer this brief prevents uh stage level everything uh you you are able to see this right completely right and in order to get this what I will do to exactly get the page content so I will write underscore page content so whatever question I'm specifically asking you'll be able to see this is the brief present state level estimates of health insurance cover is it same or not so this brief presents state level estimates of health insurance coverage using data from American Community s server survey American Community server so here you can see the US sensors and all everything is working absolutely fine okay uh sometime you also may be requiring uh you know more results right more let's say right now I'm just getting one result let's say I want two to three results now two to three similar kind of results now what I can actually do is that um or uh according to that particular text or query that I've asked the most relevant document I specifically want so what I can do I can basically write this Vector store okay Vector store and I will specifically call as retriever function I hope everybody knows what is as a retriever because we have discussed more different thing about it completely as retriever is just a kind of interface that gets connected to a vector store so that you'll be able to get more results more amazing results based on the similarity search and here I will say search type similarity but my search uncore arguments okay will be nothing but my search underscore arguments will be nothing but K is equal to 3 K colon three okay so this is what I will be probably getting the vector store okay and uh you can probably consider this as my retriever itself right so I will save this as retriever okay and then let's print retriever okay so here you can see F hugging face embeddings all the things and the number of search arguments like most relevant documents I require is three okay and now I can also use this particular retriever to invoke anything that I specifically want but before I invoke understand I also require an llm model now for those llm model as I said I'm going to use hugging face llm model now that is the reason the excess token that I actually created right I'm going to set up that excess token with my hugging face so I will write os. Environ okay and here first let me go ahead and import OS import OS os. environment my hugging face Hub API token so I'm going to set this up as my API token because from here hugging face Hub only we be downloading the model we'll be using the model and here we are going to specifically use my token key right I'm hardcoding the token over here because I've copied it from there itself but don't use this because I'm going to regenerate that particular token okay it will be of no use okay now let's go ahead and if if you don't know about hugging face Hub guys what exactly is a hugging face and all uh I'll be writing some description over here for you all right so hugging face is a platform with over 350k models 75k data sets and 150k demo apps all in spaces all open source and publicly available in an online platform where people can easily collaborate and build ml together so I'm also going to use this uh uh same hugging face Hub and then I'm going to write from Lang chain do community Lang chain uncore Community dot llms import hugging hugging face Hub right so I'm going to specifically use hugging face Hub I will create HF is nothing but hugging face Hub and now I will show you how you can call your models right the main parameter over here is nothing but you have to give your repoor ID okay now here one repo ID since you are going to use the open source model is nothing but mistol right so this is basically a skeleton for you you can go ahead and replace the model whatever you want right so we are going to use Mistral 7B okay again it is up to you whatever model you specifically want to use only you have to give your repo ID along with this one more parameter that you really want to give is model arguments okay so in the model model arguments here we will specifically give temperature parameter okay so temperature will be nothing but it will be 0.1 and the next parameter that you're specifically going to give is nothing but Max underscore length Okay and this Max underscore length will be nothing about 500 okay so these are the two parameters that you really need to create uh so that you will be able to use this particular open source model understand one thing this will be directly using from the U hugging phase Hub itself okay you you don't need to download it locally for downloading if you want to download it locally and probably want to check it out so I will also give you the code for that you can which you can try it out okay now let me go ahead and write the same query so I will say query is equal to um what is the health same query insurance coverage okay so this is the query that I have now in order to invoke it I will use the same H HF do invoke okay and then I'm going to basically give my query so here you can see now I will be able to get my response and this is the most generic response right what is a health insurance cover I have just asked this is the most generic response which mral knows but my main aim is to probably create a Q&A app right Q&A app specifically for rag application so still it is not communting communicating with my data that is available in my PDF okay so right now this is just given me a general uh General output okay General output what is available with respect to the knowledge of the internet now the next step what we are specifically going to do see understand if you want to make sure that we interact with our PDFs this hugging face uh llm models we have to probably create our prom template one more thing that I really wanted to show is that um if you want to download locally right hugging face models can be run locally through hugging face pipeline so here is the uh new code before we used to use hugging face Hub here you have hugging face pipeline same code for generation everything you can probably execute this and this will basically be uh able to generate it okay but I'm not going to use this since I've already trying to use from this right so it is going to take some amount of time with respect to execution perfect now this is done now as I know I need to use this llm model to interact with my PDF so for that I will try to create my prompt template okay now the prompt template will be nothing but I'll I'll be use the following piece of context okay to answer the question the question asked okay so uh here is the simple prompt that I have used use the following piece of context uh piece of context to answer the use the following piece of to answer the question asked please try to provide the answers only based on the context okay so I'm just giving a simple prompt okay simple prompt okay now my context will be over here Co n txt okay and then my question will be nothing but it'll be over here and I think we have done this many number of times in my previous video okay and uh if I want one more thing I can probably give I will just write helpful answers okay and I'll keep it blank okay so this is my prompt template that I'm going to use and then in order to make it as a this I'm going to use my prompt template prompt template uh my template will be assigned to this same prompt same same prompt uh I think this is assigned to this prompt okay so this will basically be my prompt okay and then my input variables will be having two things one is context okay context and the other one is question oops same question just a second question okay so context and question we are setting it as an input variable and this basically becomes my prompt itself okay so I'm going to execute this okay now understand if I want to combine okay I if I want to combine my Lang chain right my my sorry not Lang chain my my template with the llm models obviously what you really need to do so that we also get the context in the question it's create retrieval QA right so create retrieval QA chain right so if you remember that in my previous video one more way that I can specifically do is by using this retrieval QA right and in order to use this retrieval QA here what I will do I will just write from chain type okay from chain type and here I'll start giving my parameters that I require first parameter is nothing but llm llm will be nothing but HF whatever the hugging face library that we have created I will create my chain type as stuff okay so chain type will be nothing but stuff the next parameter that we have specifically going to do is my retriever which will be assigned to my retriever itself which is basically defined return Source document you can keep it as true or false so that you'll be able to understand from where it is basically coming up so return Source documents which will be assigned to True okay and the next parameter is nothing but it is my chain type chain type k w r GS okay and you can also check out this entire documentation from the from the Lang chain documentation so prompt will be nothing but it will be assigned to this particular prompt okay sorry this prompt so this is done like most of the things have been completed over here but understand one thing in retrieval QA you have to give your hugging face your chain type stuff your retriever return Source document so that you need to understand from where it is coming and CH type arguments is nothing but with respect to the prompt so this prompt will return some of the documents it will be setting up in the context then question will go over there and entire this retrieval QA will work okay so here I'm going to write it as retrieval QA right so this will basically be there let me quickly go ahead and execute it okay now it's final that you just need to run it in order to run it here is the question right so now finally we can use this retrieval QA that we have actually created and we are invoking with respect to the query that we have already written what is the health coverage and finally let go ahead and check our results so here you can see what is health and insurance covers the brief presents the state uh level estimates of this American Community survey the US cons US Census Bureau conducts ACS throughout the year the surveys asks responds to report the coverage at a time so every information is specifically given now you can go ahead and ask some other questions also if you want um let me go ahead and ask this differences in the uninsured rid by state okay so I I'll go ahead and ask this let me go ahead and write my new query so query is equal to let me just go ahead and close this in okay so here is my query so let me go Ahad and execute it now if I go ahead and see this it should be able to give me the response so here is a comparison of ACM measure of Health insurance so everything is probably matching over here see unur rates of this this this is coming up okay so all the information are specifically coming up from here this is perfect uh now we have I've also shown you how you can probably work with Lang chain and hugging face so let me go ahead and update uh this also if you want okay um insert sell CH sale to insert okay so perfect this is it uh I hope you like this particular video uh at the end of the day uh the one thing that I really want to say is that more you practice things more you see things definitely you'll be able to achieve multiple things over here so yes uh I hope you like this particular video this was it from my side I'll see you in the next video have a great day ahead thank you one all take care bye-bye

Channel: Krish Naik

Views: 8,447

Keywords: yt:cc=on, langchain projects, end to end huggingface projects, mistral llm project, krish naik langchain projects

Id: RZ2Vu8z-P1Q

Length: 29min 47sec (1787 seconds)

Published: Sat Apr 20 2024