Build an interactive RAG application with Gemini : A Step-by-Step Guide

hello everyone in today's video I will guide you through the process of creating a rag application using the Gemini model all with the free version of the API this tutorial is designed with two main sections in the first part we will explore a notebook I have created to demonstrate how to use chroma DB as a vector database this will help you store domain specific data sets and enable gini to provide answers based on relevant passages stored in the vector database moving on to the second part we will see how to build a cool looking interface this part is all about interacting with the model and by the end of this tutorial you will gain insights into extracting data from a Json or a PDF file storing the vector representation of the data into chroma DB using the embedding model to embed data chunks using length chain to efficiently split a large text into manageable chunks and finally creating a userfriendly interface using chain lit are you prepared to explore this insightful Journey well I certainly am hello and welcome back in this part of the tutorial I have created a very well documented notebook that shows you how to create a rag application a retrieval augmented generation app application using the Gemini API in this tutorial we will be using the free version of the API although it is limited but nevertheless for personal use cases it is fantastic right now I am at the last sale of this notebook because I want to show you a quick demo of what we will build here I have two questions sorry because they are in French initially this was a school project and later I decided to share what we did with the community so I hope you enjoy the video basically the project idea is as follows using the Gemini API to create a chat bot who will answer questions related to only EMS which stands for Environmental Management System the first question has no relation to this topic let me do the translation for you guys the question is how many planets are in the solar system when running this cell the model will output out of context as you can see here in the output this is intentional because in the prompt we told the model to say out of context if the question is not related to EMS on the other hand the second question is related to EMS and the model will give an answer using the data stored in the vector database as as you can see this is in French as I said but don't worry about it now that you have seen what we will be building in this video I hope that you are excited as I am and ready to dive in before starting I want to thank Google for giving us the developers the ability to interact with the model using the free version of the API and for sharing some useful notebooks this notebook was constructed based on the resources that Google shared freely okay let's get started first make sure to install the following libraries then you need to create an API key if you don't have one then click on this link and create one then copy the API key come back to the project and create a new file called Dov inside this file paste the API key between the quotation marks now come back and run this cell and hopefully you will not get any errors let's see the available embedding models we have the embedding 001 model this is the latest embedding model that we will use to embed the sentences let's move on to the topic specific data set here I will show you how to construct your Corpus from a Json or a PDF file we will start by the Json file first you need to read the Json file using the Json library in this example I have a Json file containing entries in the alpaka format this is a way to format your data set in order to fine tune your large language model what we need to do is to go through the entries one after the other and convert them into a single string because right now you can see that the first entry in this data set has three is is basically a dictionary so we need to convert this dictionary into a simple string then we will create a list that will hold these strings as you can see after running this cell now I have 398 entries in this document list and if I take a look at the first element in this list you can see now that it is basically a string so we concatenated the three keys uh the three values actually so let me show you if I uh if I print the type of this element you can see that it is a string it is no longer a dictionary this is what we need now let's see how to deal with a PDF file first you need to install the P pdf2 library then using the extract text from PDF method we can extract the text from the PDF file and store it in a variable when when printing the content of the text variable we immediately see that the text needs some pre-processing this is why I created the clean extracted text method this is not the most sophisticated method to process the text but at least it will remove some unwanted characters from the text remember that the quality of the data is important if the quality is good then the Gemini model will be able to provide accurate responses to your queries if the data is bad then you know what that means garbage in garbage out we have another problem the length of the text variable is huge right now after cleaning the text you can see that we have 800,000 characters which is a lot we cannot feed all this data to the model at once to fix this problem we will use a handy method that the length chain like library is providing the method is called create documents and it comes from the recursive character text splitter class this function which is called as I said the create create documents will allow us to split the text into chunks of fixed size in this example the size is set to a thousand characters and it will have a overlap of roughly 100 characters between the two chunks we are adding an overlap lab because we might have a problem in the beginning and the end of each chunk after running the cell the text variable I'm talking about this variable contains all the chunks that were extracted from the clean text variable let's see how many chunks do we have so let's print the length of this variable you can see that we have 898 chunks the last step step is to add those chunks into the documents list and we are going to use this for Loop to do this okay as you can see here I have printed uh let's just print the first element of this list and one more time this is a simple string okay good now we are ready to move on to the embedding phase the following table shows the tasks that the Gemini model can perform in our case we will use the retrieval document task we are going to create a custom function that will receive the documents list is this this one it will embed them and give us back the vector representation that we will store in the chroma database one note before continuing on for better results try to provide a title for each input if the Corpus is covering a lot of domain me in our case because I have data related to EMS here I have specified the title to be something related to this topic so I don't need to change it uh when adding the data to the effective database but if you want to have more than one domain then make sure to change the title for each one of those domains next we are ready to create the vector data database by using the create chroma DB method this method receives the documents list and the name of the database then it will save it in the provided path using this line of code after that we will either create the database if it doesn't exist in the provided path or it will load it so that we can append new data to it the next step is to Loop over the entries in the documents list and add the vector representation to the vector database the add function will take the text representation then it will call the embeding function we defined earlier I'm talking about this function and then store the vector representation in the database with a certain ID we have added a delay because because the free API has a rate limit of 60 requests per minute I will come back once this for Loop finishes by running this next cell we can see that the database is no longer empty and to see the vectors we will use the pandas library to create a data frame as you can see we have the ID that I was talking about before we have the vector representation of the documents and some other information that we didn't provide such as the me metadata and Uris the embedded model converts the document into a vector with 768 Dimensions now we can query the datab base to get the relevant documents that would help the model answer the question chroma provides a handy method called query it takes the question and it will give you end documents that are close to the question let's see this in action in this cell I have a question and I specified that I want five documents that are closed to this question and here is the result that we are getting passages contains five documents that are close to this question here I am showing you just the first one but to prove this let me print the length of passages you can see that we have five elements now let's see the different ways to prompt the Gemini model the first one is very straightforward basically we give the question directly to the model jam and I will answer every single question based on the training data that Google used to train it the second prompt is going to add some constraints so that the model will give the answer only to the questions that are related to the data that we have in the vector database the model will say out of context because we have specified this in the prompt if the question is not related to the data but we'll give the answer in the other case and here one more time I am sorry because this is in French but uh basically here I am saying first of all I'm giving the question I am providing the documents that I am that I got using the query method that chroma is providing so I am passing those documents here and here in this sentence I am saying that if the question is related to this information that I have provided then answer the question if not don't answer the question and just say out of context the third prompt is similar to the second one but here the model will answer the question even if it is not related to what is in the context because here I said to I said to the model to Output out of context but just answer the question regardless why why am I doing this because I am showing you that you can change the behavior behavior of the model just by changing the prompt finally the last prompt adds a description at the beginning before giving the question in this case I am telling the model that the question is about EMS why I showed all of these prompts because I wanted to convince you that a good prompt will give you good results right now I will stick with the second one I don't want the model to answer answer the question if it is not related to EMS let's move on you have seen that in the prompt we are given the model the relevant documents that we received from the vector database but these documents are stored in a list the passages list so we need to convert this list into a string this is what the convert passages to list method does now we are ready to give the prom to gini and receive the answer back you can see here before giving the prompt to the model that's I have used uh convert passages to to convert passages into a string and this is how it looks so basically this is the string that I got when converting the passages list and this is the prompts that will go to the model first we need to instantiate the model and then pass the prompt to it using the generate content method let's wait a bit and you can see now the model answered the question that I have where I have defined this down below so the model answered this question I think or let me let me see oh the model answered this question good finally I am going to take all the things that we have have seen combine them in order to make the following pipeline the pipeline consists of six steps the first one is the question we need to provide the model with the question the second one is search the chroma database for Relevant documents then we are going to convert the passages from a list to a string this is what we are calling the context or the useful information after that we are going to create the prompt we will give it to the model and then we are going to get the answer this is the last step and this is what I have showed you in the beginning of this video as I said I am using the second prompt in this case the model will refuse to answer the first question by saying out of context good let me switch to the second question and hopefully the model will be able to answer it based on the context text that we are giving it okay so this is good this was the last sell of this notebook I hope that I have succeeded in sharing the knowledge that I have with you guys in a simple way and I hope that this tutorial will help you build cool rag applications in the second part of this tutorial I am going to show you how to build a user interface using the chain L Library so that you can interact with the Gemini model without looking at the code see you in the next section in this section we are going to build this cool user interface that will allow us to interact with the Gemini model without looking at the code let's see how this works first let's test to see if the model will give us out of context if we give it a question that is not related to EMS so this is the question how many planets are are in the solar system as you can see the model has given out of context because yes this question is not related to EMS but now let's give it a question that is related to EMS for example what is ISO 14,1 uh again the question is in French but we are just testing okay so as you can see the model has given the answer good so everything is working as we saw in the notebook this user interface was generated using the chain L Library it has the following functionalities we can create a new chat by clicking on this button here we have the prompt history if you want to check the prompts that you have given to the model you can see here you have the date and you have the prompt so you can click on one and it will populate this text area we have the settings panel here you can change some settings so that you can configure your model you have the attach files button here you can attach a PDF file a CSV file and work with it if you want and finally we have the prompt area so here we can write our prompt now let's see how to build this going back to vs code the first thing we need to do is to create a new file called app.py inside this file import the same Library that we have used in the previous notebook the additional library that we are importing here is chain L after importing the libraries we will copy and paste some functions that we have in the notebook if you have watched the previous section then all these functions are going to be familiar to you for example this function will help us take the documents and convert them to a vector representation the get relevant Passage will give us the nend documents that are close to our question the make prompt is going to take the question as well as the relevant passages and give us the prompt and finally the convert passages to list here this is not correct we should name it convert passages to string it converts the passages from a list to a string because we need this in the prompt as I said these functions we were used in the previous section the next step is to use the start method this function will be called when we start the script by Chain L for example here I I will say starting okay starting chat after that I will open the terminal let me see where is this file located so it is located inside the populating in the vector database folder I need to go inside scripts okay so as you can see we have app.py here how to run this using chain lit it's easy if you have installed chain lit then you can use this the following command chainon app.py DW as you can see we have the interface it's up and running and here it said starting chat it said it twice because I have two tabs here if I try to refresh the page you can see now that I have starting chat printed to the console this shows that when the application starts the first method that will be called by Chain lit is the start method this is important because here we can initialize uh some things for example we can read the API key and get the vector database let's do this I will remove this line and replace it with these two methods the first one will help us set up the API key and the second one will help us load the vector database so let's the setup Google API method is very straightforward we are using load. EnV why is that because we have created the do EnV file if you remember in the previous section here we have the API key just for security reasons we don't want to show the API key so the Gemini API key is located inside the EnV file and we are using load. EnV because we want to read the API from this file okay I hope this is uh very easy to understand and as you can see we are getting the API key and setting the generative AI library or configuring it with the with this key let's see the load Vector database method again the load Vector database method is very straight forward remember that this path need to exist if you have followed the previous section then you have created your first Vector database and it depends on the name that you have given it for example here I have the the database and this is what I need to to choose but I need to configure the path correctly okay so this is the path and as you can see inside it we have the SQL database so after configuring the path here we are creating the client and after that we are using this client to look if the database has been created already or not if it was created then we are going to get it this name was used to create the database so make sure to give the same name after that we are using the Gemini embedding function because we want to use the embedding model to embed the sentences or the documents and after that we are storing the vector database in this DB fun uh variable after that we are storing the database using the user session user session in chit helps us store variables in the in something called user session so that we can get them or use them in other methods this is going to be very handy and we are going to see how we are going to use this later so don't worry about it now that we have loaded the API key and the vector database let's see how to add the sliders in the settings panel to give the user the ability to change those parameters first you need to import as you can see slider from input widget I have already done that now come back back to the start method and here we need to add something we need to add the settings variable here uh I'll write a wait cl. chat settings this class will help us add the sliders let's see how to do this for example this is the first slider that we have added inside the chat settings chat settings is a class as I said that comes from chain list and it will add the settings panel button here when we save the file here I have an error because I need to add a configuration to start with Okay so let's say that this is the default configuration when we load the application or when we load the model it we are going to use this parameters by default and we are given the initial temperature here we have an ID and a label and we can control the minimum and maximum values as well as the step for this slider now if I click if I make send because we are going to see why we are using send so let's save the file let's go back to the UI and as you can see we have the button and now we have just one slider you can see that if I change the max value to two instead of one now you can see that I can go from zero to two good and why I have the send method here let me show you because if I if I want to take this value and change the configuration dictionary I need to click on confirm when you click on confirm you are going you need to to add the send method because it is going to call another method that chainet will call and let me add it now the method is is called setup agent I can name it however I want I can name it setup model and here I am receiving the new settings because I have used the send method and now I need to call this method a wait setup model and I need to provide the new settings inside I will change the configuration with the new value that I have got when I when I clicked on confirm to confirm this I will print the configuration dictionary and I will print the temperature before and after the change okay this is looking good now I will save the file can see now by default it is set to 0.9 now let's change the value to let's say 0.1 hits confirm go back to the console and as you can see the before value was 0.9 but after the change the value was changed to 0.1 now I can add the other sliders I will do that quickly and I will come back okay so as you can see I have added the other sliders I have the slider for the top P value the top K and the maximum output tokens I will do the same thing here instead of printing the temperature I'll print the whole configur ation dictionary and here I will set the the keys with the new values like this so here make sure to convert this these values to float because uh this is how it should work here the maximum output tokens is an integer and the top K is an integer if you don't do this then you will get some errors okay let's save the file let's print the configuration dictionary before and after the change okay let's see if we have all the values good so I will set everything to zero just to see if this is working hit confirm go back to the console and as you can see the whole values have been set to zero this is actually working and this is great now I will remove these print statements because I don't need them now let's create the model and use the configuration dictionary to configure our model so this is how you do this you use the generative AI Library you call the model it's Gemini Pro and here you can pass a new argument called generation configuration and here you pass the configuration dictionary and if you want to know the parameters that you can change then go to Google's documentation and read about the configuration dictionary to see all the parameters that you can change now we need to do the same thing that we did here we need to save the model in the user session instead of DB I will call it model and I will pass model the model here because we are going to load it with this with these configuration we don't want to load the model each time we get a new message no we want to save it once and then retrieve it to give us the answer now let's see how to prom the model first we need to add a new function called main this function is called whenever we send the message that we have typed in the prompt area so when I type something here and click on the send button or I hit enter the main method will be called to see this let's print the message message. content this will give us the content of the message let's open the terminal let's save the file go back to the interface let's type in is this working let's see let's refresh the page I think because this is not this was not loaded correctly okay so we have a problem the app it's taking too much time to load I'll try to stop the process and I will run it one more time chainon appp DW okay so let's give the question is this working you can see now that is this working is has been printed because the main method was called when we set sent this message good now we are going to use this user session to load the model let's see how to do this so model equals cl. user session. getet model make sure to use the same the same key that you have used here so I have used the key uh model as a key that's why I have used this here and here I'm getting this variable and this will be stored in the model variable inside the main function now let's use the message to get the relevant passages you using this function that we have seen before get relevant passage passages should be called not passage okay let's change that to passages because we are getting a list of passages or at list of documents so first we need the question is message. content and the D database I'll pass them here so we have the question and the vector database so the question we are getting is from the message the database we are using user session again because remember we have stored the database when we have called the load Vector database in the start method if you remember and after that we are just calling the C relevant passages here the number of results is set to 10 but if you want you can add it as a parameter so that you can change it and here instead of 10 I will pass number of results now let's go back to get relevant and let's say that I want just Five results after that we can use the make prompt to convert passages to a string okay I think this is good and finally we can pass this prompt to the model like this so model do generate and and you give it the prompt and after that it will give you the answer now I can print the answer to the console let's do this let's go back to the UI let's give it this question so that the model can answer it let's wait for a for a while and as you can see here we got the answer we are not seeing the text because you need to access the text like this let's uh give the same same question and hopefully in the console we will see the answer instead of the this class as you can see the model has been able to answer the question correctly good but we don't want to get the answer here we want to show it to the user to do this we will use cl. message so we are taking the answer. text this is the answer and we are sending it to the user let's see if this is going to work or not okay let's give the question let's wait for a while and as you can see we got the answer let's give the how many planets are in the solar system question to see if the model will give us out of context and sure enough the model was was has been able to classify that this question was not related to what we have in the vector database good thanks to chainlet we were able to build this cool looking interface very quickly without spending a lot of time trying to design the UI sadly we have arrived at the end of this tutorial I hope that you have learned something new in this video and I hope that you get inspired and build something useful thank you one more time for sticking till the end of this video goodbye

Channel: 3CodeCamp

Views: 1,421

Keywords: Gemini model, Google Gemini, Google Gemini RAG application, Gemini RAG application, Gemini Langhcahin RAG, Gemini Chainlit, Gemini Google Chainlit, Build a RAG application using Gemini, Gemini free API RAG, Gemini interface, Gemini langchain User interface, Gemini pro guide, Gemini chroma db integration, Integrate Gemini model with langchain, Build retrieval generation with Gemini, Gemini 2024, Gemini tutorial, How to get API Key for Gemini model, Use Gemini in python

Id: ozq9fK9Pn-s

Length: 35min 21sec (2121 seconds)

Published: Thu Jan 18 2024