OpenAI RAG Chatbot | Chat with PDF locally

hello everyone today I'm going to show you how to build a chatbot that will answer your queries from some document a PDF document in particular if you're wondering that why you need such such a service or an application well in my previous videos I have shown you how to build a chatbot with the llms and it was capable to answer most of your questions but a limitation of all these llms is that they are trained and they are and they are trained with the data at particular point of time so like Chad j 3.5 is trained with data till 2022 January and anything that has happened after that it is not aware of so over here you can see that I am trying to ask the question what is a snowflake container service the service was introduced after January of 2022 from Snowflake so naturally Chad GPT is unaware of so let me show you the reply from chgb so when I asked chat gbt 3.5 that what is snowflake container service it was unable to give me a proper answer so this is when we need to provide some document which is available in on the internet we can provide it and kind of train the or you know make the llms utilize the document and uh and respond to us so in other there can be other use cases also for this like you want to you know uh you want uh replies from your policy document or things like that there are so many large documents or legal documents you want to ask questions and get a response so you can feed in the PDF versions of those documents to chat GPT and get a response which can be very convenient do you find that interesting so let's get in into it before we go into the detail if you like the content like the video and subscribe to my channel so to start we have to find understand the architecture so in this case it is called a rag architecture which is retrieval augmented generation so we have some PDF document which will be broken down into Data chunks and stored in some Vector database this Vector database data will be provided to the llm as an context to uh so when the when the user asks a question to the llm this text will be referred and a response will be generated and the response will again be stored as a chat history so the next query when comes in from the user it will be the context from the PDF document as well as the chat history which the llm will refer and respond to the user query now let's get into the coding so in my previous videos I have shown you the structure is more or less the same so we have few libraries over here that you need to import and not all of them are being utilized in the code um most important them I will try to walk you through and uh so the first let me start with the streamlit application or the code for the streamlit uh front end that we have so as usual it is uh the chat chat library that is being utilized over here and I'm just making one call that is SD session State ask chat ask PDF so this ask PDF is nothing but um a function within my main embedding uh script so within my main embedding I have a function chat genin and with I I have a class CH chat gen and which I'm initializing over here and then I am making a call to the ask PDF function with the prompt and this prompt is nothing but whatever the user is asking so over here one thing we have to keep in mind that uh generally if we use this initial um these chat window then when we are asking a new question it tends to reinitiate initialize but if we want to hold on this session we need to initialize the I mean the object of the class only once so to prevent it from reinitializing every time we have to use the cash resource which actually prevents it from happening of reinitialization so just check out the init uh cash resource um capability of stream lead and this is very helpful so coming to main embedding we have the F AIS as the vector database you can you can use many other databases uh that are available like pine cone or chroma DB Etc now coming back I have make use of open AI this time and not any open llms why because so far whatever I have seen is all the llms open llms when we I was trying it on my local system the performance and the response both were severely horrible and I mean most of the time I am not getting any response and even if I am getting the response structure and the response even is not up to the mark So this time till here I had to do it with open AI so first thing is we have to have a chat history as I was saying which we will be populating with the question and response from the llm then uh the first thing that we are doing is preparing the document so basically as I showed you in the architecture um we have to load the using the PDF loader Library we have to load the PDF document and then split it so that is another Library which is character text splitter which in which I'm saying that chunk size is thousand so thousand characters will be uh you know the document will be broken down into th characters of Chunk and there will be a overlap of 30 you can play around with these numbers and see the response uh differing a bit so whatever is convenient you can do a trial and finalize and optimize the the numbers for yourself so after that we use the hugging F embeddings this is an again open- Source free available freely available embedding model so we use this model to create the embeddings so uh to store the data into the vector database we need to create embeddings so now with these embeddings we are storing the data into FIS which is a vector database base and this is again open source so after that we are um making uh storing the content in a local local drive and after that whenever this is called we are actually passing on the persisted Vector store the response from I mean extraction from these Vector stores are passed on to the called function I mean calling function next is the llm so over here I making use of azure chat open AI with u Max token 4,000 temperature zero but over here again a very important thing is the prompt so system instruction I have given that AI assistant you must answer the query from the user um from a retrieve content if no relevant information is available answer the question by using your knowledge about the topic then I am giving a template where I'm saying the combine the chat history and followup question into a standalone question answer from the context so this context will basically be the retrieved uh content from the Victor store followup questions and this is the question from the user after that it is the conversation retrieval chain that I am using from langen with which we um will get the llms response so we are passing the llm we are passing the retriever we are passing the document name over here so this is the snowflake container PDF document that I have extracted from an HTML uh from a HTML um URL so this I am feding in so that and now I can ask question about snowflake container container and get the response and uh then I'm passing the promp that I have set over here chain type is always tough and now the ask PDF the method that I am calling from my um stream L um streamlit script right so result uh is I'm just self since this is the class you see chat gen is the class so I have all these Define within self so I'm making it as self. load model which is basic basically load model which is internally then calling our uh over here load doc so the document is loaded and then we are asking it and then we are passing the query and the chat history so self. chat history basically is as we have initialized over here this is being utilized so that whenever there after that just after this line we are actually appending our answer and the question query uh answer is the response from the model so we are storing it appending it into our chat history so now the model will utilize the chat history the context retrieved uh to answer the question so therea it Returns the answer and we are getting it in our streamlit output so going back to our streamlit application if I continue with this chat and ask it like when was this introduced so you can see from the document it finds out that it was introduced in the snowflake Summit in 2023 which was around 20 uh December time so that is why snow um Chad G was unable to answer answer question related to it but the interesting thing over here uh that I told you that uh The Prompt is very important over here because right now if I ask it who is Charlie Chaplain yeah so sometime it does falter but uh it is able to you know still go and find out from its own knowledge if it is not able to find out you know the response from the PDF document provided it still goes out and try to find out the answer otherwise and provide you the answer so this is a more like a convers ation but if if you wish that always the bot will respond only from the PDF and not nothing outside in that case you can play around with the um prompt in the code and you can say that do not respond anything beyond the content of the retrieved document so in that case it will just say that um I sorry it is not provided in the content so that is a very interesting uh use case and uh this is how we can use or upload our PDF documents and ask our questions make it more convenient instead of going through the whole doc whole PDF documents manually we can make it interesting in a conversation way that's it for today if you like the video subscribe to the channel and like the video thank you

Channel: Joy Maitra

Views: 5,894

Keywords: chat with pdf langchain, chat with pdf ai, python chatbot, python chatgpt, langchain pdf chat, chat with pdf locally

Id: CTheO06NzD4

Length: 14min 1sec (841 seconds)

Published: Sun Jan 14 2024