Build Streamlit HealthCare Chatbot with Llama 2&Langchain on CPU Machine with MEMORY|All OPEN SOURCE

everyone in today's video we are going to build this exciting project is the healthcare chatbot using number two line change and streamlined and it's going to have a memory that means that it's going to have a historical conversation and we are going to do that on the CPU machine so you don't have to worry about a GPU all right okay so this is how it looks like uh it's a data that was collected from Healthcare it is a mental health and mental illnesses data from the reset gate that I got from online downloaded that it has 13 pages okay so we are going to build a chatbot around this data by the help of the llm that is going to be the lava model okay so as you can see it has some data defining mental health and mental illnesses definitions okay yeah so I'm going to walk you through that step by step let me show you briefly how this works so I'm going to say you ask a question about um mental health related question I'm going to say how can I improve my Mental Health okay I'm gonna hit send because it's running on CPU it's going to take some time okay all right so as you can see we have an answer and it's saying that there are many ways to improve your mental health including seeking professional help from a therapist or counselor practices of care activities such as exercises so you can see that this is giving us an incredible answer and and also it also taking records of the history all right so this is an awesome project and we are going to build it step by step okay all right so first 15 first we are going to be using the uh llama model the jdml that is the quantized version it is nothing but the memory is being reduced so we are going to be using you can choose to I mean choose any model that you want to use here depending on your size or memory okay so in this case we are going to use the Q4 that is the quantite version okay so you can go here and click on download I have downloaded that already so thanks to the block for putting this together okay all right so once you have that we will go to hugging phase and get the API token access so when you go to your settings here settings and you come to access tokens you can get your API access key right here so you click on the copy right all right and we are also going to be using the percentage Transformers all right for the embedding okay that's so that's that is from the hanging face models okay so that is pretty good at um clustering or semantic searches so we're going to be using that for creating embeddings right okay so let's get into some coding stuff but before that I want to take you to the um to the architecture okay so this is our document so this is going to be the healthcare PDF so it's going to be extracted here it's going to be split into chunks so when it's done we will splitting them into chunks we're going to be so we can see chunk one chunk two we are going to create an embeddance so it is nothing but the representation of these text into numbers by that way we are going to create the knowledge base from there so as it has been embedded here so at this point we are going to use the hanging face embedding so that is the sentence Transformers to do our embeddings okay and it creates an understanding um here so is that the semantic searches and it's been stored in the knowledge base okay so at this point we are going to use the face so the phase is nothing bad let's see here it's nothing but um Facebook AI similarity search and it's a library for officials similarity search and clustering of dense vectors so we're going to be using that so you can use the option adoptions like the chroma DB so we are going to use the phase for today okay and add the knowledge base so when a user asks the question the question is also embedded it goes to embedding is so when it's embedded here we create a semantic judge so embedding just like here so exchanges to numbers and it goes into a semantic search and installed in another knowledge base here so it creates an understanding of the um of the question from the user and the knowledge base all right so we try to find a similarity in there and when it gets the answer there will be the ranking and it gets to pass it through the llms and the user gets the answers okay it's pretty easy all right so these steps are called the injection phase right so let's get into it so we are going to get into our coding we're gonna so I'll be putting um attaching this to a GitHub so you can pull that you can clone it so you can start installing it and working on it okay so first things first I've already uploaded my data okay it's running all right so this is the mental health data and I've created an environment okay so let me show you something real quick if you want to start a new project you as you can see right here I've have my environment created but you can do that by saying python V when you say AMV you hit enter I'm not going to get the environment installed here and we are going to create a number dot m so that is where we are going to store our secret API Keys as you can see this is my hanging face API token that I got from the hanging face all right settings okay here's to the next thing so this is the model that I downloaded we are using the uh the quantized version all right so we have that installed here we have that here and the requirements so these are the packages that we are going to install that's going to help us with the project Okay so you can see all this in here I'll be um attaching them in my GitHub so you can install them okay all right so we're gonna be importing these packages we are going to import screen let SSD for our API I'm sorry for the uh app extremely chat all right so we are going to create our messages boxes with a streamlined chart um we are also going to need a conversation now retrieval chain to create a conversational memory okay and we're also going to need the pi PDF and the directory loader for the PDF loading and assessing them hanging face from Reading C Transformers to load the model from hanging face so we're going to need the C Transformers to transform that and we are going to use the recursive character split to split the text phase to store bearings and the conversation about for memory so this is what is going to be used to create a memory all right so the next thing we are going to do is we are going to create a load so we are going to say load in your files okay so let's say say loader is equal to erect directly loader and we're going to say so we're going to Target this folder load the document okay and say ice we're going to be doing a lot of coding more than the token so bear with me yeah so globe equal to this not invite just to make sure we grab the whole PDF in the folder and loader CLS will be equal to Pi PDF loader okay so this ensures that we get the documents from the directory I'm going to name it commands equal to loader let's look okay so before I forget if we have to store the requirements right so within the air fastest way is to do a pip install at our requirements right and then txt okay so that way everything is installed I have that installed already so yeah just to let you know okay so after loading that next thing is to split so when we are done with the document we're going to split so split yes into chance so we're going to play the text it returns so let's say X speaker is equal to aggressive character splitter I'm going to say chance so tank size will be mostly dependent on your data here I'm going to choose 500. in the tank overlap overlap is going to be 50 you can choose whatever size you want depending on your use cases and the text size right so this is the chunks if you go to the text so we are splitting it now it's equal to split documents right okay so now let me show you here we've got the document we extracted it and we split it and here we are here now into chunks all right so let's move on create our own beddings say create buildings so embeddings nothing but presentation of the text into numbers so the machine can make minions from it okay so we're gonna use the hanging face embedders so that is where we are going to use the sentence Transformers maybe face to create our embed it so let me just go there and copy that right copy that and paste it in here all right let me do a comma you want to say running everything on a GPU it's probably going to take about but the whole thing is probably going to think about 9 to 10 gig or something yeah size CPU okay all right so the next thing is create a vector store with a vector store we say a store it's equal to base Dot from commence X so from the text chance it's creating the return patterns for each of them and installs them here in the backup store okay and the next thing is to create our llm okay I'm going to name it llm it's equal to C Transformers I'm going to load the model the model that's equal to llm right so guys you just go here and copy that oops grow here foreign name let me just copy this come here I'm going to paste everything here right and I'm gonna say Adobe I'm going to say model type SQL to llama and the comfort so we're going to choose the mark size token so the max you okay to go to we want to do 2128 temperature so it's going to take the the randomness of the outcome right so I'm going to do that 0.8 so guys you can choose the temperature that you want here depending on your use cases right let's see okay take this one out right perfect so the next thing is to create a memory so I'm going to say memory is equal to a conversation alpha memory okay so we are creating that memory key let's call it a chat I'm gonna store the chat history in our memory and return messages we wanted to return the messages we're going to say that to True okay because we wanted to turn on messages so historical messages okay let's just create a chain for two more so create a chain okay dots from nlm llm okay hello we say chain type we call it stuff so the other options that you can use here with the chin type using this for now so retriever it's equal to Vector store so I'm passing my um Vector store into the chain okay and in that that is where I'm going to search for the answers so I'm going to do a search equal to okay I'm gonna and we'll be so I will be retrieving the top two answers okay so you can choose whatever mean sizes that you want okay put two answers and then a memory memory it's equal to so we are passing our memory here so we pass our memory here memory pass it inside the chain right so the next thing is to create the streamlined application say title I'm gonna name it out okay what and for a big an emoji in there let me see okay let me copy this guy okay this this guy here so let me save it and let's run it let's see a few minutes Quran up the pi see just like there's an arrow shank size 916 let's see okay okay I think I had an X here so let's save it and run it again and not this one let's run it again okay let's see let's try it again yep so that's the initial output so let's move on to the next one so we want to start a creative conversation so we say conversation query attention chat on the parts in the query correct and I'm going to print the results say results I'm going to pass in the chain and the question let's say put that code query so the queries whatever the user is going to put in so it's going to be the question and pass in the chat history that is going to be the the section history okay three okay right so let's move on to this section stay restriction State and I'm going to pass in the history say it's free now we append query the results so this is nothing but just greeting the conversation or chat where we are going to pass in the query the chat history and we want to retrieve the results okay and it's going to be there very and the answer so we want to put all that opened out to the history and now you're going to return the results pull this down the results as the answer right the next thing is created on the function to initialize the visualize the section we want to legalize the station state so we have three sections that we want to initialize we want to initialize the history so the conversation chain is going to be going to involve the history and the generated that is the answer we are getting and also the past okay so if history is not in the states section if you return an empty list so you press no screen nothing should be written right if the generated in section of the section we are going to say that Texas state it generated generated the vehicle to say something like hello okay X asks me anything that's me do anything right the next one is the other condition is if if a SD prescription speak I'm going to say for the past is equal to comparison something like hey and create a lot of functions to display the cuts history at history put in my at least we want to see the reply container just reshape the output to be in a container okay so you want to say streamlight.container guys I'm gonna put that description to the Streamlight documentation it's quite an interesting one so you can also go through that so we say container is equal to St container so we pass in your container initialize uh visualize the container so container is equal to SD instance is that and now so we're gonna apply the container so with container now let's create something and with we want to create a form the user to input the query I'm going to say key is equal to line four okay I'm going to say clear on submit is equal to true so anytime a user inputs a question you should clear the press in the Box uniform you start a new input so the user input is equal to create a text box now where the user inputs query the question right so this is nothing but form Apex input today I would put a question okay I want to say this holder face holder you want to get this in the in the holder so that the user will know what to type about okay so something like winter out and we are going to do the key as input the user should input the still and now the user should make a submit so I'm going to say go to submit button go to home so extremely way of submitting St dot form sub submit button okay so when you hit on that you're going to send that we have to get the answer okay all right so far so good so the next one is to create a submit so if they submit in the user input so if we have this button in the user inputs and that is ready we want to create an output to be equal to now we are calling our conversation calling our conversation chat so we are calling this function right here that contains the query so we are passing it here in the user input right so that when the user inputs the question we are going to get the answer okay and within that submit button in the user button we want to also get the section state you won't get the station state of the past conversation from the user to the user side so that when the user input any question we'll get the previous input is the past state you also won't get a generated one we're going to say append output so we're going to append this user outputs and the the user input and the output into the past and the generator all right so the next thing to do now is so if another condition to say the generated output outputs now we are handling the output we want to create a container for that reply in I'm going to iterate through the lens of the section for the generated so if you want to create a message so we are using the stream let's over here we are using the screenplay chart message to get the message that was generated and we want to put that in I want to get that from the past and it generated okay so section state in the past all right so I'm going to walk you through that once again so you get it cleared understanding of that okay all right and is user screw so the user screw I'm going to say of the for Iron Range we want to get the iteration of the conversations for the user and the generator so key string K is equal to 3 of I okay with the user let's create an avatar Style okay I'm gonna do something like a thumbs but when you click in here you can see any of them so you can select whatever you want to do put there as a thumb or I'm going to picture stamps okay all right you want to get a message for the generated so I want to say I think I can just copy this change it copy that paste it if I want to change this one to generate it okay all right so now what I want to do now is we want to call the function so initialize to stay to run initialize the section state and we want to display so display .hp okay so display right this coding is done so what we've done is we've imported the packages for the um to run the application so extremely chat we import the message to create the streamlined conversation the conversational tribal chain to grade their memory for conversations to store the historical and we import the pi loader to and the directory loader to get the PDF documents loaded hanging phase to create embedding see transform it to load the models from hanging face this is the Llama 2 model and the recursive character splitter to also create the text into chance the phase to store the effect to do a vector store conversational perform memory to also implement the memory as well so we loaded the PDF documents using this code to write and restore them into documents and we do a split text into chunks so we did a chunk size you can vary this depending on your use cases so we create a text sound and split them and store them from here documents so we call them and then we store them into a text Chunk and now we create some buildings from the hanging face model right so we created a balance and we store that locally you know the CPU okay so using the sentence Transformers and also Vector store so we call We call we use the files right as the Facebook similarity search and we for each of the texts and embeddings they check the text songs and buildings we restore them in the vector store create the llms using the um number two model right and they're talking so you can vary these token sizes in a temperature as well we start with a memory using the conversational buffer memory okay and record the chart is free return messages through we want to return this history and we create a chain okay and for that so we pass in the llm that we created here say llm is equal to llm that's a chain and the chain stuff type stuff you can vary that also we have other ways you can also put that in the retrieval Vector store and the K is equal to 2 we want to talk to you um ranked house outcomes and the memory is equal to memory we are passing in the memory here okay so we create this Streamlight application we create a conversation chart we pass in the query and we want the results to be I'm using the chain so we want to get the question so that they want the user improve the question and chat history we want to get the chart history and also um so the chat is free is equal to the section of the of the history I think this is going to be state okay I think I have this state yeah this one is going to mistakes section state all right and so we get the history and we're gonna append the answers to the history and we want to also only get the answer all right and we initialize the section stage for them so we have history generated in the past right so if none of the history notes in the Sexes State we print we get a empty list if it is not generated it's not the sexy state that the answer was separated if it's on the station state which you see Hello ask me anything and if the past is not in the success State we should say hey something like that so you can put whatever you want to put there so depth we create a display chart history for that we create a container that is going to hold the chart history all right so this is how we want it to be this display we create a container called the St dot container okay we instantiated so we have a reply container and a container all right so with the container we pass in the form for the user to input a form so the user so if they open submit okay clear and submit so if the user input in the question issue clear for the next question so the user input we are passing the question and what we want the um what we want to ask in there all right so you can say ask about your mental health whatever you want to put there all right so if submit button if there is submit and the user button we want to get the compensation chart and the user input okay so this time we are passing the conversation chart for the query in here for the user to input a question and we also want to get the so open submit and user input when that is clicked you want to get the past the generated output so we want to get the of these conversations so if section dot state is generated if that is generated for the answers so with reply container it's going to iterate through the generated and it's going to give us the past and Legend written all right and it's going to come up with some emojis as well so we are now going to call these initialize section stated and display shade for application to run pretty good all right so let's save it and run it okay I think I'm just going to refresh it let's give it a few so we are getting an arrow on line 72 let's see this is the eight see what the error is uh okay so let me change this to on emoji all right save it around back to there's an arrow here let's correct that user right we don't have to put in the user there okay because you're already have it here okay so generated it's not from the user okay all right so let's save it and let's run it let's run bye all right so let's ask a question okay bye okay so as you can see we have our answer here so this is pretty much how it works okay so you can see it's giving us an answer there are many ways to improve your mental health including seeking professional help from therapists and all that yeah awesome so yeah I hope you like this video don't forget to subscribe and share so I will attach the link to the GitHub so you can get the code right thank you for watching see you in the next one

Channel: DataInsightEdge

Views: 7,448

Keywords: Medical, Chatbot, Open Source, Llama 2, CPU, Conversational, AI, Healthcare, Innovation, Tutorial, Step-by-Step, Sentence, Transformers, Faiss, Chainlit, Intelligent, Technology, Natural Language, Processing, Integration, Solutions, Advanced, Vector, Storage, Model, Interface, Development, Conversations, Implementation, Industry.

Id: XNmFIkViEBU

Length: 49min 53sec (2993 seconds)

Published: Sat Aug 05 2023