Build a CHATBOT with AWS Amazon Bedrock- Llama 2, Langchain and Streamlit [HANDS-ON]

welcome to this video on how to build a chatboard using Amazon Bedrock last year I introduced Chad GPT to my 8-year-old daughter Anisha and since then she's been pestering me to create a chatbot in this video I'm going to show you how to create a chatbot using Amazon Bedrock llama 2 Foundation model Lang chain and streamlit this video is also part of my Udi course on Amazon bedrock and generative AI which you can check from the link below also look out for some of the other use cases I have as part of this YouTube playlist now let's go ahead and check out what we're going to build as part of this use case in this section we're going to build a chatbot using Amazon Bedrock so before we get into the architecture for chatbot let's take a quick look what our end chatbot is going to look like so this is what our chatbot is going to look like and I'm running it locally on my machine so we have hi this is chatbot Amisha and then powered by bedrock and llama 2 so let me just pause a question I is your name and you can see on the top it's showing as a running it's giv me a fairly long answer hi my name is Lama and then is giving me some more details let me pose another question to it who's the best tennis player and let's see what's the response and on the top you can see it showing as running and you can see it's provided a fairly detailed answer which says I cannot provide a definitive answer to who is the best tenis player but it names some of the greatest ones Roger feder Nal and chage so the answer is fairly good now let's go and take a look at the architecture for this chatbot that we're going to build using Amazon Bedrock so for this chatboard for the front end we're going to build using stream lid so streamlet is a python Library that's being used for creating the front end for the back end we're going to use Lang chain and Amazon Bedrock where we will use Lama 2 Foundation model now this is super important if you don't have experience with streamlit or Lang chain then please go to the last lecture of this section where I'll will give you a high level overview of what is streamlit and what is langit and why we need to use use them when we're creating a chatbot with Amazon Bedrock now for this chatbot architecture as we discussed we're going to use Amazon bedrock and Lama 2 Foundation model then we're going to use Lang chain and streamlet now it's pretty clear why we going to use streamlet that's for developing the front end Amazon bedrock and llama we are going to use as a foundation model now the question is why do we need to use Lang chain now let's take an example say we are chatting with the foundation model and the first question we ask as we did previously who are the top three tennis players and say it comes up with a response of these three names and then the next question we POS is next three so what we are trying to highlight is who are the next three best players after these three names so if you pose the second question next three to the foundation model it will hallucinate and we will not be able to understand what is the context and the reason reason for that is foundation models are streets they do not have any memory so what that means is whatever you provide in the context window as a prompt it will be able to answer but once you provide the next prompt it will not be able to connect the two prompts so that's why we need to use Lang chain in this scenario langin has different modules and we're going to discuss three of them here the first one is the Lang chain memory so we have a module called as conversation buffer memory so what this does is once we post this question top three tennis players and it gives some answers it is going to store this conversation in the langin memory so then when the next question comes next three it's going to read that and then in combination with the first question is going to send the entire thing to the Lang chain prompt so for this prompt we use a lang chain method called predict or run it will combine this first question and answer then it will add the second question that has been asked and together it will create a prompt based on the foundation model that you have selected so you might know and we'll discuss in the later sections for each Foundation model there's a particular way of writing the prompt to get the best outcome so in this case it will format the past question and response plus the new question format it in the way llama 2 would like and then send this question to Amazon Bedrock Foundation model which is Lama 2 in this case then llama 2 would process all these details and answer the question being posed and then this answer is going to be sent to the front end which will be displayed to the user but then this response will also be written to this langin memory so when the user asks the third question it is going to combine the first question the second question and the third question that has been posed so we discussed in this architecture about Amazon Foundation model we discussed about streamlet and then we discussed about two modules in langin one is the langin memory and the second one is the Lang chain prompt now there's a third element as well called as Lang chains and in this case we're going to use conversation chain so what chains in Lang chain do is they are able to connect this Foundation model to various other components say external data sources internal data sources and in our case it's going to be able to connect with this memory then it's going to able to connect with this prompt so it can establish the connection between various external data sources and modules so this is the architecture we're going to build so the first we're going to build the back end and then we're going to build the front end now let's go ahead and build this handson hope this lecture was useful thank you okay now let's go ahead and build this chat board so the first thing we're going to do is we're going to build the back end using Lang chain and Amazon Bedrock so we're going to use l 2 Foundation model to build this Chad bot and then we're going to build the front end using stream l so the first thing I'm going to do is I'm going to open this Anaconda Navigator now I have the Anaconda Navigator open so what I'm going to do is I'm going to open the visual studio code editor using this Navigator and this is really important because if you don't do that then your streamlit code might not work later okay so here is the visual studio code editor so I'll just do a launch so now my visual studio code editor is open so I'll just click on file and open folder so I've just created this blank Bedrock chatbot folder let me just open this and you can see it's completely blank so there's nothing out here so now let me just go ahead and click on this file and new file and it's a python type and let me just do a save as so we'll say chatbot backend okay and just do save okay now what I have done is to make it easier for you on how to write this code code I just compiled a list of steps that we can use to write this code so let me just copy this so this should be available with the PDF that I'm going to share as part of this course okay let me just paste it here okay now let's look at the step one so here is a step one so we discussed in the architecture earlier we're going to use the Llama to as a foundation model then the second thing we're going to do is we're going to use Lang chain cu the foundation models are stateless so they don't have a memory so for that we're going to use this module from Lang chain called as conversation buffer memory then the second thing we're going to use from Lang chain is conversation chain so what this conversation chain does is it connects the foundation model to modules in Lang chain which can use prompt as well as this memory so these are the three things we're going to use Amazon Bedrock langin memory and langin conversation chain so those are the modules we have to import so first thing we're going to do is we're going to import OS so that should be pretty straightforward you can see here I just highlighted all the modules that we need to import so next thing we're going to do is we're going to say from Lang chain dot llms so you can see the Lang chain has a module called as llms which allows it to connect to any foundation model so that's what we're going to do and then I'm going to say dot and you will see that it shows all the foundation models that it can connect to so you have the coher you have the AI 21 and if you do open AI you'll see that it can also connect to open AI from Microsoft but for us we're going to stick to bedrock so we're going to say bedrock and then we're going to say import Bedrock okay next we're going to import is conversation buffer memory so from here okay so I will say from Lang chain dot memory and you can see it shows me memory and import so I say conversation and it will show me all the ones related to conversation buffer memory so conversation buffer memory conversation buffer window memory Etc and I'm going to just stick to conversation buffer memory okay now what we have to do is we have to import this conversation chain and let me just say from Lang chain do chains and you can see it gives me this option import and you can see conversation chain is here so that's what I'm going to import so I think now we're good so we imported the Bedrock service we imported the conversation buffer memory and we have imported the conversation chain okay now let's goad and write a function that will help us connect with the Bedrock service now let's go and write this function and let's give it a name so maybe I'll say demoore check and part okay now we have to establish a connection with the Bedrock service so for that let me say demo LM is equal to and the way you write in Lang chain when you want to connect to a particular llm or a foundation model service like betro so I'll say betro and you can see it's giving me bedrock and and you can see all the arguments that you can pass so here you have so you have the credentials profile name then you have the model ID then you have the model keyword arguments so those are the three we need to pass to be able to connect with the Bedrock service and Lama to Foundation model okay so the first thing I will say is profile okay here it gives me the credentials profile name and you can see here below the profile name that I'm going to use is going to be default and you can use anything you want and uh if you want to modify your credentials you can do edit credentials and you can see the credentials here so these are my admin credentials I'm going to delete it after this video but if you want to modify use something else you can go ahead and do that and we saw that it's a string so I say default okay so next we're going to use model ID which tells Bedrock which particular Foundation model to connect to and is a string so now let's do a little bit Google and see what is the model ID for Lama to so I just so I will just say Amazon Bedrock llama to Foundation model ID and I think I just go here and then um so here you can get the model ID for the one which you're trying to use so we'll say maybe we'll just use uh this one so let me just copy this and let me just go back and I can just paste it here next thing we have to do is we have to provide the model argument so we'll say model arguments and what it essentially means is we have to give all the inference parameters so if I just go back to this documentation page I can also click on this inference parameters and search for the foundation model that we're trying to use so in this case it's Lama 2 models and you can see these are all the inference parameters that have been provided so maybe I'll just copy this one that's here so you can see it has four things prompt and prompt we don't require because we are going to use the Lang chain prompt template so we can skip this but these three we can use so in earlier section we discussed what is temperature for a foundation model the temperature decides what is the randomness of the response that you're going to get from the found ation model and similarly top p is used in a similar fashion for defining the randomness and diversity of the response then you have the max generation length and this determines what would be the length of our response so here you can see it's 2048 tokens and you can say approximately every 100 tokens is equal to 75 words so here if we put maximum which is 204 tokens that is 75% of this which is approximately 1,500 words but we'll just go with the default values that are defined here so let me just copy this let's put it here and let me just move this to a little right okay so prompt we don't need prompt because we're going to use the Lang chain prom template and the temperature let's just see what is the default value temperary 0.5 0.9 and 512 so we'll say 0.9 and this was 0.5 0.5 and I think this was 512 so the limit we're setting is for 512 tokens okay let me move this a little up okay I think that's pretty much it and this should help us connect with the foundation model with the these inference parameters and then um we can just do a return and demoore llm okay so that's the second task we have accomplished now before we go ahead and create this conversation buffer memory and conversation chain let's go and test this out and see if this is going to give us a response okay so so to test out this llm we'll use something known as predict method in the Lang chain so Lang chain has two methods which is predict and run so what these methods allow us to do is so you can just pass in the input in any format that you want and then what Lang chain will do is it will convert that input user input into a prompt that is appropriate for that particular Foundation model so now let's go ahead and do that so so what we'll do is we'll say return and demoore LM and then we'll use the predict method so we'll say predict and this will need some argument so maybe we can say input uncore text okay and for now I'm just going to comment it it out because this return we're going to pass to this Lang chain template okay so I think it's ging an error I think we need the argument here as well so I'll just say input text okay now now let's go and run this particular function so I'll say a response is equal to demo chatbot and then we pass some input say I what is your name so we'll do something simple okay now let's just print it out and say response okay great so I think everything looks good we don't seem to have any error so I'll just do a save and let me do terminal new terminal okay now I'm just going to run this so it's taking time that means probably it's making a connection okay it's giving me an error saying that Bedrock object has no attribute predict um okay probably it should be in the lower case so let me just go ahead and change that so I just made it into lowercase let me just do a save and let's run it again okay awesome so it's giving me a response you can see here hello my name is Lama I'm a large language model trained by team of researchers meta how can I help you today so maybe we'll just give one more question and then we can proceed ahead what is the temperature in London then like and uh let me just do a clear here so we can see the response clearly so I just save this and let's do a run okay awesome after waiting for a few minutes I think it's giving me a response says temperature in London can vary throughout the year and it's giving me some details on that which seems pretty relevant okay awesome so I think this is good so what I'm going to do is I'm going to comment it out and I'm just going to leave it as it is so that if you follow along then it can just help you okay and I'm just going to uncomment this create okay now let's go ahead and proceed to step three which is creating a function for conversation buffer memory so if you go to the architecture this is what we're going to look to create a buffer memory which can store the past history of the chat okay so let me just go here okay before I do this I just want to show you one thing we can just remove this input text from chatboard because this is only for testing now we don't need to pass any arguments here we'll pass the arguments only in the conversation chain okay so I'm here so let's call it demo score maybe uh memory memory and now we just say a variable in memory or you can name it anything you want and we'll say uh conversation of our memory and then we'll just pass arguments and you can see it has multiple arguments but it just basically needs two key things so one Lang chain memory or conversation buffer memory needs a large language model or Foundation model like L 2 which can summarize all the past messages and the new messages so it needs to connect to the foundation model and that is Lama 2 in our case so we'll need to provide it details around which Foundation model it needs to connect to and the second thing it needs is the max token size so if you remember for foundation models we have a context window which says what is the maximum amount of tokens and if you convert the tokens into words what's the maximum amount of words that you can provide in one single shot the first thing we need to provide it is the details of LNM that we're going to use for the LNM we already have this particular function demo chatboard so that's what we're going to use so I will say here here so let me just say llm underscore data is equal to demo chatbot demo chatbot okay so now we can provide substitute this value here let's say lmore Theta okay and the next one we need to provide is the max broken limit and we can just keep it uh what we have provided as Max gen length here for Lama 2 500 12 and this will vary based on the foundation model you use uh in earlier in we saw that the max token limit for Lama 2 is 2048 uh if you just remember here it's 2048 but remember the more tokal limit you provide the higher is going to be your cost because the cost would depend on per th000 token okay so then we can just do so we can say return memory from here okay now I think we have written the function for conversation buffer memory now the next thing we need to do is write a function for conversation chain and we discussed earlier the conversation chain as for the architecture helps us connect with this prompt through predict method and also helps connect the foundation model in this case Lama to with the Lang chain memory okay so we'll say and let's call it as demoore chain okay now this particular function requires three things one is the details of the memory the details of the foundation model and then we also need it to print out its internal messages so now let's go ahead and do that so let me say okay so we need to a name uh let's call it lmore conversation that will be equal to now we have to look at the module that we had imported which is conversation chain so we say conversation chain and it's here now we need to pass the argument so you can see it has many different arguments so we're going to focus on just three and the first one is the llm that we're going to use and um let's add that here let's name it as maybe LM um chain data maybe and then basically what it provides is this particular function which will give us the details of the foundation model so maybe demo under score chatbot chatbot okay so now we can provide the llm which is this Lama model that we have here so we'll say LM chain data next thing we need to provide it is the details of this conversation buffer memory so what we're going to say is memory and here we're going to use memory from here so we say memory okay now the third thing we have to pass is something known as verus and it's a and you can see here it's a bullion so I'll just say true so basically what it does is it prints out some internal state of the Lang chain chains okay now it's showing me an error here and I think that's because we need to pass some arguments here so the first argument we need to pass is the input text that is going to come from user and the second one is it requires a memory so hopefully this should go away okay good this is gone now okay now this seems good uh but maybe I'll just change this name here from demo chain to something more meaningful so let me just say conversation okay now let's write the last piece of the puzzle so in this we're going to take the user input and then pass it to the Lang chain prompt template using the predict method as we had done earlier so let's say let's call it as chat reply and that will be equal to this llm conversation so llm underscore conversation do predict and then we just pass some argument and this is input to input score text and that's argument we had here and then we just say return chatore reply okay it's G me an error I think I just forgot the r here okay I think this looks pretty good so let me just do a save so we are done with creating the frontend and if you just briefly look at the steps so we did a whole bunch of imports then we created a function which allows us to create to Amazon Bedrock service and Lama 2 Foundation model then in step three we created memory for the application using Lang chain memory module and then we created the conversation chain again from again using Lang chain which allows us to connect to this memory and input text from the user and then finally we will pass this input text to the L B chain promt template now that we have built the back end now we need to build the front end if you are building an chatbot for an organization then probably would have a front end team which is going to build a front end for this application streamlet is a very simple python Library which allows you to build front end and it provides most the code out of box so let's just take a look so if I just go to streamlit documentation it says streamlit is an open-source python library that makes it easy to create and share beautiful custom apps for machine learning and data science in just few minutes you can build and deploy powerful data apps so this is what we're going to use to build our front end so you can see here it provides you a lot of code out of box to build basic llm chat apps from open AI as well as Amazon Bedrock so what I have done is I have taken this course so this is the source code provided by streamlit and AWS and I have have modified it based on my requirements so I'll just quickly walk you through what are the changes that you can make to customize the front end as well as add the functions that you have written here so let's take a look so I just want to highlight where I have put these two stars you can see all the places two stars you have to modify those lines you can modify it's either to update the functions that you have written here or you can modify the look and feel of your chatbot so the first thing you do is you import streamlet and the chatbot file that you have in the back end so here you can see import streamlet as STD and if I just show you here so so the first thing you can see here you import streamlet as STD and that's what we have done the next thing is we have to import this backend file here so import XY which is basically the name of your backend file so I'll say chatbot and you can see see here back end and import this file as and you need to give it some name maybe I'll just say demo and that's what I have highlighted here and you can see the two stars which shows that you have to modify this F the second one is you can modify the title and you can just go to the link to see how to modify this title so if I just go here and this is how you modify the title so it's st. title and then you just provide the title you want and that's what I've have done std. title and and this is hi this is chatbot Anisha and sunglasses and this is what I have written here and this is how it will show so the third thing is we need to provide langin memory to the session caching and you can just go to this link and it will tell you a little bit about session State and how we deal with session state in the in the streamlet but you don't need to modify anything you just let it be as it is and the only thing you need to modify and you can see here again you need need to provide the details of your memory function so if in our case if you see we imported the chatboard backend file as demo so you say demo and then dot whatever is the name of your memory function so if I go here and the name of my memory function is demoore memory so that's the value you substitute here okay and then similarly you add the UI chat history to the session cache and you don't need to make any change here you let it be as it is and then in the fifth one you render the chat history again you don't need to make any changes to this the next is you need to provide the input text box so if I go here so you can see chat with Udi course bot here so that's the input text window that you can provide and you can just modify here whatever you want so I've written chat with r's B Ro course and you can change it here and that's what I highlighted with two stars and then you need to let everything be as it is and then finally you have the chat response here what you need to modify it with is if I just go here and then you can replace this with the function you have for conversation chain so you can replace with demo conversation so if I go here again you can see two stars and here you add demo so like you have imported this file as demo and then you provide the name name of the conversation chain and as we saw it was demoore conversation in our case and that's pretty much it and then you don't need to make any change to this code so as I said this code is provided by AWS and streamlet and I have modified it as per the functions I have taken in the back end as well as to modify some UI components so if you want to add more data elements or modify anything else then you can use all these widgets that have been provided in this streamlet Library okay now I think everything looks good so now we can go ahead and run this chatbot and see if it works so I'll just go here and open new terminal so now to make this chatbot run rly this is the command that we're going to use so it is streamlit run the name of the front end file pile okay so now let me do that so stream lit and the name of the front end file is chatbot uncore front end. file and you can see it's giving me a URL and it's automatically opening this URL in my browser so you can see here now it's showing as running stop and deploy so it takes a little bit of time to come up okay awesome now I just waited a few minutes and now you can see this chat bot has come up and these are the details we had provided for user interface so if I just go here and you can see and you can see this is what we had as title and then if I just go down and the input text box said chat with rahul's bedrock would make course bought here and that's what is come up so now let's see if this works so I'll say hi what is your name and let's see see what is the response we get so you can see on the top it's showing as running stop and deploy okay finally it's giv me a response says hi my name is llama I'm a large language model trained by a team of researchers at meta and then it's giv me a really long answer maybe I'll ask it one more question maybe I'll ask it a couple more questions where is New York and it's giv me some answer New York is State located in the northeastern United States and gives some more details okay let me ask it a controversial question who is the best tennis player in the world and let's see the answer so it's G me a good answer it says it's difficult to say who's the best tennis player in the world and then some of the top players currently include Nook joit Nadal and Roger Fed so I think that's a fairly good answer so this is how we can build a chatbot now as a Next Step what you can do is you can also deploy this code on an ec2 instance and so that you don't have to run it locally hope this use case was useful thank you

Channel: AWS GenerativeAI and Amazon Bedrock with Rahul

Views: 7,741

Keywords: aws bedrock, aws bedrock tutorial, aws bedrock ai, aws bedrock workshop, aws generative ai, aws bedrock mini project, aws bedrock hands-on, aws bedrock chatbot, aws GenAI, amazon bedrock, amazon bedrock tutorial, aws, bedrock and langchain, llama2 and bedrock, chatbot using aws amazon bedrock, chatbot python, chatbot tutorial, aws chatbot, aws chatbot project, lambda apigateway, amazon bedrock ai, aws bedrock demo, aws lambda, aws bedrock llama, aws bedrock titan

Id: pWKaraDvVPg

Length: 36min 48sec (2208 seconds)

Published: Tue Jan 16 2024