Build your own Q&A KnowledgeBot using GPT-Index & LangChain - Document to Chatbot

hey friends welcome to one little coder in this video we're going to learn how to build a question and answer bot using GPT index language this video has been coming for a long time I got sick so I could not make this video so what we are going to do is we're going to take a very popular book in this case meditations by Marcus earliest we're going to take that book and then give it a GPT so that you can ask questions in the book which um the book the bot will try to answer from the book so we're going to build a q a bot very simple nothing sophisticated we are not going to build a UI I wanted to keep this video very simple so that we can focus on the actual concept of building a q a bot then probably if there is more interest I can show sophisticated UI using stream letter radio in the future videos but today now to Simply take a book that is dot EXT file a lot of people have been asking like let's say you are a company that you've got your own txt or PDF how can you build a bot on top of it so that's exactly why I'm making this video so I've got a txt file like a folder full of txt file so how can you give this folder to whatever code that we're going to table and then use that to ultimately build a bot where we can ask questions so I'm going to show you a demo very quick demo so I'm going to ask what is this book about so okay let's first try to understand what is this book about so it says what do you want to ask the bot and I said what is this book about and then it is going to actually get the answer so it says this book is about importance of living your life of contentment and purpose and how to achieve this by following the principles of art and adapting to the events that happen in life so I can ask more questions I can I can say give me five points to live peace free and then it is going to actually take the book content and then it's going to find a place where this kind of content is there and then it is going to give me five points okay it has given me five funds for focus on your own actions and decisions rather than worrying about what are those things so it has given me Five Points so this this is exactly what we are going to build so let me go long back and then give why we are building this and then give some more context first of all for us to understand since chat GPT has been launched there has been enormous amount of interest in the conversational AI bot space everybody wants to build their own conversational agent people want to see what is the point or part of life that they can automate whether it is in your personal life whether it is in your business life if you particularly talk about a business life then this is more closer to what customers support agents do in a lot of companies so you ask a question they reply give you a reply and then you they try to fix it for you like for example I ordered something on Amazon I got a wrong book so I go to Amazon and then say you know I got a wrong book they are going to tell me which order are you talking about I select the order number they're going to give me a return or they're going to give me some details this is typically quite popular workflow in a lot of companies um but for customer support agent to make a decision or give you a response they need to be trained on something called sop sop stands for standard operating procedure in typical customer support workflow you have something called standard operating procedure so like that if you've got a document where you've got knowledge base of your company of your product documentation how can you take the document how can you take the dot txt file and turn into a DOT like let's say a bot like in this case that could be a CLI bot that could be a full-fledged UI based bot like whatever that bot is so for us to do that we are going to leverage large language models large language models like in this particular case models from open AI but instead of directly going to open Ai and then building something on top of it we're going to use two very popular libraries which I've covered already in a separate video called Lang chain and GPT index in this particular code at least we're going to use Lan chain to access open AI models and we're going to use GPT index to build an index like build a vector search where we're going to take the document convert it to some kind of embedding store it and then retrieve it that's what we're going to do and all we need is this Google collab will have the entire code so you will be able to get started immediately without any hassle hopefully I've tested this code multiple times so this should ideally work without any issue so I'm going to take you to the start of the code and then explain you what we are trying to do first thing is we need to install the two popular libraries we just mentioned which is GPT index line chain once we have these two libraries installed then the next thing that we need to do is we need to import from GPT index we need to import the simple directory reader to read the folder and the GPT list index GPT simple Vector index llm predictor prompt helper from long chain we need to import open air that's primarily to connect to open Ai and if you want to store this whatever that you are going to build the index inside your Google drive or if you want to keep the files inside your Google Drive then you can disable this code or uncomment this code and then use it but in my case I'm going to do everything collab so I've commented it and finally to set the environment variable the key ideally I'm showing you the key because I'm going to disable this key after the video is published ideally you should not share a key with um with whoever you're showing like you you cannot expose your secret token anywhere on the internet anywhere in the world so I'm going to just that's why you set it up as an environment variable especially if you're going to deploy it it will be helpful so yeah that's why we're setting it as an environment variable now that we know what all the libraries that we need and next thing is how are we going to get this key getting this key is going to be quite simple um you go to open AI website okay when you go to open a website you are going to go login First like go to open a platform.openai.com then once you're log in you will reach some home page then click personal and then from there come to API keys so click APA keys this is where you can create a new key like you can see that I've created a new key which I can delete it so go here click create new secret key and once you click create secret key you can copy the key here make sure that you copy the key because copy and paste it somewhere like just don't copy it leave it copy and paste it somewhere otherwise this key will not be accessible like if I click ok now that's it this key is not accessible now I cannot see the key I can delete it revoke it and then create a new key that's what I'm going to do for this key as well what I've shown in this video so make sure you create a key copy and paste it in whatever application that you're built that you're building um add it as an environment variable if you're doing it on local machine another important thing that I think I should highlight at this point may be quite obvious for a lot of people is we do not need GPU for this thing in fact like on Google collab I'm not using gpui because we are going to use large language model as an EPA call you're not literally running this model on a GPU so you don't need GPU so this is hence this is quite helpful let's say you want to develop a mobile application you want to develop a web application where you you don't need GPU you can still use it because all you're going to do is make a call to the open as Rover get things done one is fine tuning the second one is the inference where you are going to make a call and get it so both will be done the next thing is we need to now take the folder so okay let's rewind back we learned what are we doing we learned what are the libraries we need we learned how to get open AI API key at this point the next thing is we need to find a way to get the data that we need to use so in my case what I've done is I've gone to the MIT uh classical edu I've got the book The meditations written by Marcus Aurelius and I've got this is this is written one 167 EC so I've got this book I've got this book and uh the way I'm getting this book is by saying W get W get and then download the book like you can upload it you can go to the folder upload the dot EXT file if you want like click upload here and do it so it depends like whatever you want to do it upload it so you need to have a folder and you need to keep all the txt.txt files that you want to use or you want the model to learn you want to keep all those things inside a folder and keep it that's that's one thing the next thing is now we have a folder we have a folder full of dot txt file which is where exactly the knowledge is there and we want to build a knowledge bot based on this knowledge so now we are going to build a helper function we call it construct index and the function takes the directory path as an argument what is the directory path where the the folder where we have got all the dot txt file now there are certain parameters that we need to set because any large language model that we talk about it has a context um like the length so sometimes you need to chunk the text into multiple parts so that the large language model can consume it and get it so so we're saying maximum input size should be 400 and 4096 tokens number of output when it responds like 256 chunk overlap chunk size limit we can see this in detail in the future videos but right now let's go with the default parameters the next thing is we need to create a prompt helper instance so the prompt helper instance is going to use all these all these and then create the prompt helper instance that's going to help us in you know designing the problem to taking the prompt based on the input text the next thing is once we are done with the prompt next thing is we need to define the large language model in itself so what is the large language model you want to use um La Land chain can actually support uh open open source models like models from hugging face as well that means you don't have to only use open a which is a paid APA you can also use free open source models but in this video we're going to use open a model so here where we actually specify the large language model predictor llm predictor here you have to specify the llm in itself and what is the llm that we are using we are using open a we are using the cheapest model in open here so it's not going to be as good as chargpt in this particular case because I'm using it for demo purpose but when you are building it you can probably use text da da in c o o three you can use text davinci03 that's the most expensive one also the most advanced one but I'm using uh the cheapest one like you it's always a good practice to start with the cheapest one because you're going to make uh you're going to pay for the number of tokens uh so okay that's another thing I need to cover so the way you're going to be charged by opening is like when you give the input text for the first time whatever tokens you have got that for the tokens tokens approximately you can say one token is like two words one word half a word so somewhere in this range not necessarily towards your half forward or one word so for the number of tokens you're going to be charged so it's always important for you to keep in mind how much import input that you are giving so token cost optimization like that is another totally different topic altogether um so that's some Advanced prompt engineering technique how you can optimize cost but talking about Basics I'm using the cheapest model you can go to the open a website and then see how much each model cost I'm using the cheapest model just because it's a demo purpose I also like I don't I don't find it actually bad like I got some decent results but if you want to advance the model you can use text Darwin say O2 ticks down and say oh three depending upon your use case now that's where you define the large language model answer now once you have the llm predictor in place next thing is you want to use the simple directory reader which we imported before to take the directory and then load the data into something called documents like it has got a multiple.txt file I'm going to take everything and then put it inside documents now is when you're going to leverage GPT index completely because you're going to build something called a vector index so you're going to use the documents you're going to use the llm that you just used with the prompt Helper and build an index so this is more like the embedded so it's going to take everything um learn from it create an embedding and then store it as a DOT Json file so that's what you have to do now that's dot Json file you can store it in disk which means you can use it anytime you want like second time let's say you want to ask a question to the spot like five days later 10 days later you don't have to do the embedding process again unless until you have got new content so if you're a company you want to build this for your company let's say in one year you've got like let's say 20 documentation so for the 20 documentation when you build the dot Json index now next time you don't have to retrain it again because this for this 20 documents knowledge is already embedded inside the dot Json file like the vector index already has got it now if you get additional documentation let's say you went for a new product release some vulnerability patch then you are getting new files for that you can extract fine tune it and then you can combine it that's again totally different thing but for whatever you have already got that knowledge is going to be stored inside the index.json as a vector index so we built a very simple Vector index at the end of this function we're going to return the vector index into an object wherever you want you can store it in index and then we are going to save it so this is the first part of building a bot like a GPT border q a bot what is this first part called this first part is called fine tuning or training embedding so when you go to your openai cost usage place you can actually see that there are two kinds of usage so you have one usage called Model usage you have second usage called Fine tuned fine-tuned training usage so one is when you make a request to the model and then get a response that's called Model usage the second when you have text in place you can fine tune it and that's what so at the first point what we have done is we have basically taken the text fine-tuned it or created in embedding so it extracted the knowledge out of it and stored it in numerical format that's exactly what we finished doing now the next thing is how do we take an input like which is an English in from the user in this case English and then use that information from the user and use the knowledge that we have got and then take the information reply back to the user that's the second part so the first part is creating this knowledge from empty text which we have successfully done and the second part get the input from the user which is most likely English now take this and then use your existing knowledge and then combine this and then reply back to the user with something relevant go ahead now what are we going to do here we are going to build a new function called ask bot what is the input the tax ask bot is going to take so we're going to take the index.json file so that you know like in this current session wherever you have got you can use the index.json file now very basic thing just like we saved the index.json to disk we can load it from disk using the given path in our in our case the path is here the default is index.json so while it is true we are going to run like an infinity loop you are you can see what do you want to ask about now you can take the query from the user the English query and the response mode is compact leave it out for now take the query you create a response and use that response and get the response back so there is a query that query is sent to the indexed query you can see this is the user query whatever you type that's a user query you are using a method called query and that query is going to create a response back in return and that response is going to be displayed to the user finally very simple function quite simple now this takes us to the end of our bot if you want to build a UI on top of it so this is where you have to use like this is the part where you are going to build a UI on top because everything that happened before is actually you know behind this scenes like creating embedding storing the file Json file storing the vector index all this thing your friend users are not going to care about but this is where you will actually build the UI like I said we are not going to build the UI I wanted to keep this video simple now everything is run so let's go ahead and then invoke the bot so first thing that you need to do is you need to run this construct index and give the folder where the file is there so in my case the folder is in my current working directory I can go here copy the path come back and give it and because I want to give the entire directory I can give in your case it could be different software you are using Linux you are using Windows you are using Mac go find the folder where you have got all the dot txt file wherever it is and mention that folder path here once you run this it's going to take a couple of minutes so because you are constructing the index and it also depends upon how large how big your file is once that is done the next thing is you can go now at this point in the same folder where what you have specified you are going to also have a file called index.json and that index.jsonpath is what you need to give in my case both are in the current directory in your case it could be completely different now index.json and then you can start asking questions let me run this and then I can ask well I I think I already asked what is meditation okay let's see what is meditation practice of focusing one's mind on a particular thought or activity in order to achieve a state of mental Clarity and Tranquility I'm not sure if this is there uh where in the document that's okay so what is meditation so what um why am I always sad I don't know if there's an answer for this in this book but let's see if it can answer again um we are using the cheapest model so it shouldn't bother a lot it's possible that you are feeling sad because you are not living in accordance with your nature you may be feeling discontent with your current situation then it gives a long answer how can I improve my life so make you can actually make it conversational you can keep context window um we have already a different video using radio I've already done so I can link that if you want you can search for it to improve your life focus on being satisfied with what you have okay I've got 23 000 subscribers should I be satisfied with that not sure at justly and benevolently and use reason and Justice to make the most of the present moment be mindful of your thoughts and actions strive to be a part of the greater Universe by connecting with the reason of our common nature that's amazing so this bot is quite good so and it works like you as you can see this bot actually works it has used the knowledge that we built and based on that knowledge because see the index the index we loaded and index.query and then it is giving us the response that we want that means we have reached the end of the video where we have successfully built a question and answer bot using the knowledge that we have got like so this is like a knowledge bot using GPT index and land chain leveraging open ai's large language models and this is the core of it a lot of people have been interested about it a lot of people message me and ask me so I'm sorry for the delay but finally we have got it here I'll link the documentation GPD index luncheon in the description I'll link the Google column notebook in the description make sure you check it out if you take this what are the things that you need to do to use it for your own dot txt file like knowledge it's quite simple first thing is keep your dot txt or txt files inside a folder first thing second thing is add your own key third thing is install all the libraries and I think then you're good to go you all you have to do is run everything construct the index ask the bot with the with the right path and I think you should be able to keep this bot up and running without any issue if you have any issue let me know in the comment section happy to help if I know if I can help but otherwise I hope this was quite helpful to you in building a knowledge bot knowledge q a bot using GPT index and long chain leveraging open a large language models if you have any other question let me know in the comment section otherwise see in the next video Happy prompting thank you for watching

Channel: 1littlecoder

Views: 25,700

Keywords:

Id: Dhc_fq5iCnU

Length: 19min 58sec (1198 seconds)

Published: Sun Feb 19 2023