Chat with your PDF - Gen AI App - With Amazon Bedrock, RAG, S3, Langchain and Streamlit [Hands-On]

hi everyone my name is Girish and in this video we will be building a chat with your PDF application we will use Amazon badrock and use the Titans embedding model as well as anthropics cloudy model to build this application we will also use Technologies like rag which is retrieval augmented generation L chain stream L Docker S3 F Vector index Etc I'm super excited to build this application in EN handson video like this and I hope you like it let's take a look at quick demo of what we will be building and then we'll start doing the handson coding so I have downloaded this particular PDF from a arthritis website and I have created something called Vector embedding and which I'm storing in a S3 bucket and in my client application I want to query this PDF or chat with this PDF asking it a question and hopefully it can answer me uh with the contact that I provide based on the vector embedding that I created so let's go to my client application real quick and I'm just doing some um I'm downloading at this point I'm downloading the uh the index files and storing them in the temp folder and then from there I'm loading it to our um Vector store now U when I ask question to my large language model I'm assuming that large language model has never seen this document before or it could be your corporate document which is internal to your organization and this is not available to the large language model when it was trained so that's where uh the rag comes into picture let's ask it a question so let's say what is a joint and how does it work so So based on the context provided here's the answer a joint is where two or more bones meet etc etc and if I go back here let's see joint how does it work let's see so it got it from here and some other places in the document and using this as a context let's say ask another question what are different R types of arus okay so based on the context provided here are main types of arthes which is Osteo arthitis rheumatoid Etc and this is again very contextual let's ask it a question which is out of context for example uh who on the World Series last year okay so it says unfortunately I do not have enough contacts to answer this question which is what we expected because I want it to be very specific to the context that we have provided and use only the knowledge base that I am providing to it so this is our application so we will go back to our slides and see how I build it and what exactly we will be building so what will we build we will actually two applications imagine you have an admin site with which admin administrator have complete control like which upload the PDF this PDF document actually is getting split into various Chun why do we split into chunks we will see it when we actually do the coding but then each of these chunk get vectorized vectorized session is nothing but getting numerical representation of the textual content content and then once it's vectorized we store this embedding into FS Vector store which is basically we created the vector store locally and then upload it to S3 so it can be shared by the other uh Docker image so once my this is the admin side of the things where we will be splitting and storing the embedding creating the embedding and storing them in the vector store now comes the user side of the application when user ask a query like how we were asking first thing it does is it converts that query into Vector using the same Titan embedding model then it makes a or it does a similarity search what is similarity search is basically try to find documents which are very close in numerical space to what question that we asked once it finds those document documents it actually asks the llm the same question and give them the document that we retrieved as a context that's why it's called retrieval augmented so we are augmenting our question to the llm with the retrieved doc documents from a pre-indexed or pre vectorized embedding store stores content so once we got uh we send or make the make the query to llm and then llm respones return and we respond we actually show it to the user as we have seen in the demo so we'll be building these two application one is the admin side one is the user side I want to quickly touch based on a rag rag is basically retrieval augmented generation as we have seen uh when when when the llm was trained whichever llm you use when it was trained it did not have let's say the latest data or it will not or it did not have the your organization's contextual data so in that case we have two options number one is use user a and uh keep providing it with the context when the user does a query or fine-tune the M llm model to create your get the base llm and then fine-tune it with your your data set now fine tuning can be expensive so it depends on the use case but Mo in most use cases F fine tuning could be expensive and then but if you let's say using a lot of um tokens for every request uh when we build the contexts then and and and a lot of requests that you're making in that case it could be cost effective you have to do the math but in case of rag we actually as as we saw we actually have built an index or vectorized embedding store we get a similarity search to pull some similar documents and send those documents along with the user query as as it says LM uses knowledge based outs training data source so since llm was trained at a at some date in the past it does not have a latest data or it does not have your corporate data so we have to provide with the with the knowledge base so this is what we are doing in in in in this in this case so building admin site is super simple and then we will we'll do the hands on coding anyways so we'll create an S3 bucket we'll create requirements taxs create a python application build a Docker image access the application from browser upload the PDF and we'll see the PDF gets processed it gets converted into numerical vector embeddings and this embedding is stored in S3 on the user side again we will build up simple python application uh with and dockerize it and access the application from the browser ask a question and check the response again my name is Girish and I've have been a cloud consultant for several years and please do like share and subscribe to my YouTube channel it helps me stay motivated to create more videos like this with that let's start building our application okay so here I am in my visual studio code and this is where we'll be creating our application so let's let me create couple of directory let's call it admin and then also user so that we can create our code in here so let's go to admin let's we'll create the admin application first so first go to S3 bucket and we can actually use the same bucket or let's just create another bucket real quick and let's call it YouTube demo badrock chat PDF make sure to copy this uh file because or the name of the bucket because we'll be needing it uh when we do the coding and I'll keep all the defaults and just create the bucket hopefully the name is available and let's see so the name is available so we are good so I'm going to save this name somewhere let's go to let's just create a new file here and I'll just save it right here okay so let's start we'll start with some of the libraries that we need for this particular demo so what we'll do is we'll create uh a new file in here requirements or text and we need streamlit we'll also need P PDF we'll need length chain we will use f CPU model uh sorry CPU uh and boto 3 for accessing AWS let's quickly create a Docker file to build our Docker image and the docker file will build from python 11 Python 3.11 and we will expose for the admin site let's expose port 8803 and let's say work directory and what's the work directory let's make it SLA and we'll copy our requirements sorry requirements.txt to this location and then run pip install minus our requirements.txt then copy the rest of the files in in the in the image and then let's create the entry point entry point is stream lit run and what file we'll run let's make it admin.py for example and let's say what server Port we run on server. Port let me just increase the font size a little bit server. Port is let's say 808 3 we said right okay and then we can just do a server Dot address first z z okay so this will build our Docker image let's create a file called admin.py sorry and in this admin.py we'll write our code to do all the processing so we'll import boto 3 import stream lit as s import Quest I'll also import something called uu ID so when we save the file we'll save it locally with the uh unique ID so I just want to test it real quick so that what we let's do one thing let's just test it real quick whether our Docker image and everything is working and then we'll come back and let's call it equals to let's say sorry main okay let's run the main function and the main function let's just run let's just say the main function what we can say is let's say STD do right s this is admin site for chat with PDF demo okay let's just build this much and see if there are any hopefully there are no errors then let's just do this so what I'm going to do I'm going to build this and let's this Docker build minus f p PDF reader let's call it PDF soor not minus F minus t for tagging minus t PDF reader admin and with everything so it may take a little bit as it has to download uh actually I have downloaded most of the libraries so it took pretty it was went pretty fast but for you it might take some time now what I'm going to do let's just run this and I'm just going to S talker run minus p 8083 8883 in the interactive mode which doesn't have to be interactive but I'm just going to give interactive reader admin okay I think that looks good let's try to see what happens okay so it's running let's try to quickly access the application so so that we we know that there's no error okay so this is admin s for chat with PDF demo so that's what I expected to see let's just build the rest of the application okay so oops sorry let's just do some housekeeping and download or import some of the libraries that we will need and we will definitely need an S3 client let's say so S3 client let's call it as three client B 3. we will say S3 we will also pass our bucket name where we have to where we want to store this as an in an environment variable when we actually U run the docker container so let's say get EnV and let's call it a name environment variable okay now let's say what what else do we need we need some we need Bedrock related Imports also so what we'll do is we will import since we are doing going to do the embeddings we're going to import Bedrock embeddings from Lang chain Community Lang chain Community Dot embeddings import bed rock embeddings okay what else do we need now we will need a few more libraries from Lang chain let's say let's say here we need a text splitter which we'll be using to split the text into uh chunks so let's say from Lang chain do text splitter we'll import recursive character text splitter yeah what what is recursive character text splitter so real quick we want to when we split the Tex we don't want to split like this is one chunk this is another chunk this is another chunk what we want to do is we want to split in such a way that this is the first chunk then some part of the first chunk also goes in the second chunk so that it doesn't lose the context from one chunk to the next chunk and similarly third chunk may have let's say for example some part of the second chunk towards the end as well that's why we want a recursive character text splitter okay what else do we need uh we might need oh we need the PDF loader let's say PDF loader and then we'll say from from Lang chain Community has something called document loaders import by PDF loader okay so that looks good and then when we find some error we will import some more stuff so okay so let's see uh we will start our uh upload functionality so let's say we call it uploaded file equals to St do St has something called file uploader and here we can say choose a file or some text it's just a label and then we can give file type and here we are only ex uh accepting PDF type now if somebody has uploaded p what do we do we need to process it so if uploaded file is not none none then what should we do let's just process it so what I'm going to do here is I'm going to generate a request ID just for um just a unique ID just for traceability and let's say let's write stitute WR so we see what request ID it is and you don't have to print this but this is just just for my uh just uh troubleshoot in case we have to troubleshoot anything so with this request ID I will generate a file name I'll just say saved file name so I want to give it a unique ID with the PDF extension like that then what we'll do we'll open the file sorry with open saved file name and mode is right let's call it w and then what we'll do w dot w. right and we'll say whatever file we uploaded file. get value like that so this will actually save the file locally because when why do I have to save it locally because um P PDF loader from from L chain right here it actually takes the file name as a string not actually the file content file name and then it does its processing that's why we need to do it this way okay so once the file is saved what should we do let's just create an instance of loader which is pi PDF loader with the saved file name okay and let's just say Pages equals to loader dot load oh sorry it should be Pages equals to loader do load and split and let's just say we just write a quick um how many pages are there so let's say call it total pages in The PDF which is length of the pages let's just do till here and create our Docker image and just test it real quick I I like to do frequent testing so you know uh we are always sure that things are working at up to that point let's see let me just refresh the page okay so it's showing me the interface to uh upload the file let me just upload the same file let's see what happens I'm going to upload this what is IUS this documentation okay so now it says U ID is not callable yes that's correct that's a mistake and The UU ID should actually be generated uh with a different syntax so let's let's just create a simple definition of get unique ID and the get unique ID will give me string of The UU ID U ID 4 let's say okay return so my request ID should actually be let's replace it with this real quick I'm going to build it again and run it refresh the page okay dragging the file all right so basically I got uh okay this is so I I have to see why it's giving me the functions instead of actually a request ID but I saw that my PDF is loaded and it has total 17 pages and if I see it here well actually it has let me see if I have downloaded yeah it has two of 17 so it's it has 17 pages so which is good so let's see what's going on with the uu ID request ID is get unique ID sorry this is why it's showing me that way but anyways so let's let's keep moving forward okay so now once we have these Pages or S the PDF loaded what do we do we have to actually split the text from the PDF so let's just do let's just say split text split text now what we do is let's let's create a helper method let's do splited DOS equals to split text what do we pass it we pass it Pages now let's just say we want to have U chunks of size 100 charact thousand characters and with 200 overlaps as I was explaining earlier we need overlap so that next chunk doesn't lose the context of the previous chunk so let's do it this way and then we'll we'll just do this um std. write just splitted doc length and let's do length of the because there are 17 pages and they probably have more than thousand characters on each page so we should see number of split uh on the should be more than what uh the actual number of pages were let's just print couple of splits let's CST WR I'm just going to do this little bit of separator so that I can see what's on the pages as do right and what do we want to write here we want to write our splited doc let's just print doc zero and Doc one that's for fun okay you don't have to do this all but it's just just to test that splitting is working so now we have to create this function called split text let's just Define it here and in the split test text function sorry split text what we will do we will create recursive character recursive character text splitter so let's see what we need what we what are we getting here we are getting Pages we are also getting our chunk size and we are getting something called chunk overlap once again you test what chunk size and what chunk overlap uh gives the best results and depending upon your needs but for me I'm just testing with 1,000 as a chunk size and 200s chunk overlap so let's create this text splitter what do we create we creating recursive character text splitter with chunk size equals to chunk size and chunk overlap equals to chunk overlap okay then what we do is we use this text splitter instance to split our documents split documents these are my pages okay sorry and the splited pages we return return the docs let's just do here split the the pages or text on the pages into chunks so each of these docs will actually be one chunk so we will get some number of chunks which is definitely more than 17 if there are more than 1,000 characters per page let's run it till here and see what we have what happens okay so let's go back to our demo and refresh the page let's drag the file okay as we can see that uh we getting total 17 pages but we got split document length is 42 that means it has divided our 17 pages into 42 different text con uh different chunks well actually I just put zero in both the places so that's why showing zero but we could have probably just checked it here with what's the second split we can just do it when we do the next test okay all right now we got a splitter text what what do we do next so after the split text we have to create the vector store we are going to use f as our Vector store so let's say just give it a little note that hey writing the or creating the vector store okay then let's just create a function called create create vector store and we'll pass the request ID and the documents or the splitted documents that we just got so we'll just call splited docs let this function return a Boolean that's a result and if result is good what will do we say St do write let's just say rray PDF foret successfully else okay let's see SD do write error please check logs okay so here we'll be creating a vector store and let's define this function I'm just going to copy the whole thing from here and then Vector store okay I don't know if we imported FES uh when we imported let's see uh so no we did not so let's import that now and this comes from length chain Community dot Vector stores code F okay so let's create our Vector store let's see so here we have let's call it Vector store f equals to let's say f Dot from it has method called from documents what does it take it takes documents which is what we passed or actually in our case we passed splitter talks so let's just call it documents and it also takes embedding which we did not create yet so let's say bedrock embeddings okay so let's create the Bedrock embeddings real quick and Bedrock embedding we imported it right but we haven't used it so let's just create it right here and just say belrock and beding actually we have to create the client also so let's just create the client let's just first create the client bed rock Cent equals to boto 3 do div with the service name as Bedrock runtime okay now Bedrock embedding is actually bedrock embeddings and then you have to pass the model ID and please check my previous demo on how to find any of the model IDs but let me just quickly show you if you go to your console in a Amazon badrock if you go here and look at all the uh base models for example and here if you see Titan for Titan and if you go to Titan text embedding for example if you scroll further down you will see some examples and also the model ID so we going to use this model ID Ember text one V1 let's just copy it from here and it needs also client as our Bedrock client okay so Bedrock embeddings right here embeddings let's call embeddings okay so where am I uh so I got the vector store FS uh from documents what we are passing and using this embedding so it will split split the text it has already splited a text now for each of text it will create numerical representation of the text and using Amazon's Titan embedding model okay so once we have this let's see where do we want to save this so I'm just going to give it a let's say file path let's this is local path file path equals to that's where I was uh that's why I pass that um request ID because I want to let's say if you want to upload some more PDF then I we want to save it a different uh different request or different location so it will be unique every time let's call it folder folder path is for example I just say slash Temp and let's go we'll jum the index and save it locally so we just Vector store files do save local index name index name equals to actually let's call it file name this not a file part sorry file name index name is my file name and the folder is folder path is my folder path so at the end of this method if it successfully run it will save my request id.bin and it will save that or create that folder and save the indexes index related files into that particular folder in the temp file okay so it saved it locally now but we want to actually upload it to S3 so let's just do this upload to S3 because if it saved it locally it will only be on this Docker container I want it to create once and use it on the multiple client containers if you have more than one so let's upload this and then uh we'll see or verify this in S3 so S3 client it has sorry it has upload file function which takes a few parameters first name is file name which is my folder path folder path plus plus file name it creates F index create two different kind of files uh one it ends with f SS and the other one ends with do pkl so we'll upload both the files so this is the file I'm trying to upload where I'm uploading it so it takes something called bucket which is bucket name we already defined that that and what is the key to this file in the S3 bucket so let's just key let's just call it U my Fest do FS okay so this is the first file I'll just copy this and this will do the second file which is actually nothing but pkl file and just call it PK and if playay is no error let's just return true from here because we have a logic uh to display success message so let's just do let's just call it main method Okay cool so we are from the document that we splitted we are actually using Bedrock embedding or sorry Titan embedding model from Bedrock to create the vector index we save it locally first and once it's saved we actually upload these two files to our S3 bucket and if everything goes well we should uh we should be able to uh see this upload it to S3 okay so let's do one thing so now let's just build the uh Docker image first now this time I have to pass some environment variable as I said as I said uh bucket name is coming as an environment variable we also need to this Docker to be able to access our S3 bucket but how do we do that so what I'm going to do I'm going to mount my AWS credential into the temp directory of the docker image so that I don't have to copy all my access Keys Etc and it should be able to use that to run the application or access the S3 bucket so what I'm going to do this time I'm going to start it a little differently and if let's just do it here so instead of just running like this I'm going to pass in some more parameters for example minus E bucket name and bucket name we actually created a new bucket which is this bucket bucket name is this okay and we also going to mount my AWS folder which has my credentials to root AWS so my python file from there oh sorry my yeah my boto 3 can actually find the credential to credential which has enough permission to use the S3 bucket okay rest is fine let's run it see what happens okay so let's go to our application and then refresh the page okay okay unknown service Bedrock run time service names are maybe I did a m typo let me see let's see what happened here uh so it's right here border three client service name Bedrock Das run time okay let's try to fix it and the admin file let's go to where I'm creating this bed drop client service name oh oh sorry it took the whole thing as a string it should not be like that it should be like this okay so let's just rebuild the image real quick it's pretty fast because it only takes time when you build it first time after that super super fast okay so let's try to refresh the screen again and see what happens okay so far so good let's check the bucket real quick that we created it should not have anything at this point yep so let's let's go back to my let's go back to my uh application and try to upload the PDF okay so now it sees the this this this is the first uh chunk this the second chunk so or first page and the second page and now it says here is creating the vector store which takes a few seconds it's just it's not that big of a file so okay and it says H PDF process successfully which means our index was created and must have gotten uploaded to the S3 bucket let's just refresh it yes I have my FIS SS and PK files uploaded in S3 successful so now we can build our user user uh web application let's do that okay so let's build the user application and let's go to the folder user let me just open another uh bash terminal right here CD user I'm going to copy paste some code and some of the files so that this video is getting really long already so I'm in the user so so what I'm going to do I'm going to copy copy the requirements as well as Docker file so we can just quickly change it okay let's go to user and in the user requirements I don't need the P PDF because I don't need to read the PDF anymore so I'm going to remove this rest of the stuff we need and in the docker file I am going to run it on a different port and let's call this uh everything else looks good oh I'll call it let's say just app.py let's make it r84 okay so create a new file called app.py and I'm going to copy some stuff from the admin and let's see okay let's see this uh let me just copy the entire admin and then we'll remove what we don't need okay so I'm going to just say let's say uh just do the St do uh let's call it header and this is this is this is oh sorry let's say client site for chat with PDF demo using let's call it using using Bedrock rag Etc okay so this is what I want to just quickly test my uh uh my image building and other things so let's just do this and I'm going to remove other things as we come across but right now I just want to test my function real quick and or sorry my image real quick and then we'll see so let's just build this and then we will run it so okay so to run to build this application let's stay here and then let's do Docker build Docker build Min D let's call it what do you want to call it let's call it PDF reader or read client client application since it's a client application okay sorry I did a mistake so let's say Docker build and I have to give the current directory okay so it built the image then let's run it and then we'll see what we what happen so let's just run it Docker actually let me just copy that same command that we used to run admin so I'm going to just run it like this uh PDF actually did I put reader or read let me see real quick oh I want to make a reader client let's just create another image okay let's run it so the okay that's fine we'll upgrade it later but let's run this application real quick and see if it's loading for us Local Host 8084 oop sorry Local Host 8084 like that okay so it's actually loading the application just fine so let's just build the rest of it okay so let me go to my app py and then this is where we will be starting to build our application so first thing first is we have to load the index load index now where are we going to load the index from we going to load the index from S3 so let's call this load index and Define a function called load index so we'll use the S3 CL Cent I think we already created S3 client since we copy pasted it so let's just do this S3 client dot download file and what do you want to download it has to take some uh parameters let's say this is a bucket name then what's the key key is the key is actually what how we store it in here so this is the key this is a key for the first first file and then there will be key for the second file so I'm just going to give it a key like this and what is the local file name so local file name is let's just Define a file path or folder path somewhere here let's see PA equals to since we are going let's download in the temp temp directory I'm just going to say slmp okay so that's my folder path and let's let's go back here and this is my um folder path and what are we downloading here my FS do fiss file let's download the second file also which is nothing but the pkl file okay L file okay so we downloaded these two files in the folder path let's do one thing whether let's just show this real quick to see whether it's uh downloaded successfully or not so what I'm going to do is once we after download or load the index what I'm going to do is uh let's just list the directory using the OS os. list D and folder path okay and let's just write it uh files and directory in let's do do it this way in the folder path okay and we'll just list all the directories and the files in that s. write D list okay so if it's loaded successfully in our temp folder uh we should see the list of the files from there let's just quickly build it and see once again I want to test at every step so that it doesn't come at a surprise uh if something is not working later all right let's do this let's go back here refresh the page and once the page is refreshed oh yeah so my files and these files are downloaded and they are stored in the temp folder great now we are going to create our um index or like fire index loaded from these files and then we'll be able to query the index so let's do one thing from here let's just do create or let's say create index or it's not really index it's actually store but let's just call it index okay so we'll call it f index F dot uh what we going to do earlier we did uh save local now we're going to load local load from local load local what are we loading so index name is my underscore F then what's the folder path folder path is the folder path that we gave earlier now what are the embeddings we are going to use because we have to create the vector representation of the this data as well as the users query data so embedding says edings are bedro embeddings not not the this we actually created Bedrock embeddings this one yeah and let's see what happens okay so it created the index and then we can actually after this we can just say st. write let's say index is ready okay uh while this thing is doing it what we can do we can also test it real quick actually let's build the image and run it okay refresh the page okay so it's a serialization relies loading a pickle file so and so so okay what we're going to do is we're going to I mean it's probably not a good idea to do it in the production but we have to allow deserialization in this case so what I'm going to do pass another parameter called deserialization Equals to True like that okay so let's refresh this and see no actually we had to build it sorry you could do it without Docker also I just choose to do it in Docker it's uh easier that way so uh so that when you want to upload it to ECR or even Google it's like already available for you okay so now what the issue load local it says index name. fire do I not have an index name I am passing the index name so let's see what's going isue now um constant character at this field could not open file for reading okay okay I think I figured the problem it's uh the typo I think right here let's just quickly do this testing okay go back to this and let's see what happens okay now it's the index is ready so we are good let's just build our rest of the interface and the query the uh the index and get the response okay so what we going to do now is index is ready so what we are going to do is we are going to create a field where let's say if I'm not going to create a button on the and let's call it ask question if that button is clicked what do we do with st. spinner spinner we say okay or we just call this quering quering please wait so uh we'll get a llm uh let's get let's call it get llm we're going to create a method and then what we going to do once we have the llm we going to get the response so get response so how do we create this llm let's see let's create a method called get llm reusable method so let's call it Define get llm since we using Cloudy cloud model so we're going to say llm equals to bedrock I think I have the Bedrock imported if not we'll have to import it so let's see let's import the Bedrock uh we import the badrock from let's see right here from length chain. L I think dot llms dot Bedrock import Bedrock okay like that so let's do this uh we imported it and go back let's go back to our LM so we call it badrock and what do we actually which llm are we going to Lo load we are going to load the cloud llm cloudy or Cloud so we have to pass the model ID which is actually anthropic let me just copy it from the B drop website right here if I go back to base models and I think this is the one so if you scroll down this is the model we are going to use so this is the model ID Now what is my client client is bedrock client we already have an instance of this and then there's another thing called Model KW ARS and in this we will actually limit our model to generate maximum tokens like 512 or something Max tokens to sample let's call it uh let's keep it 51 so I'm actually creating a anthropic I'm getting instance of anthropic model and we will be using anthropic model to generate a user response so once this is done let's return the llm okay so we got the llm from here now we get the response how do we get the response so we let's create a reusable method again for getting the response get response okay so okay we will actually be creating prompt uh so I'm going to copy some code from uh so I don't have to type it and I'll just explain this uh here so okay so we have to uh import a few more things let's just do that first and then what I'm going to do is uh we have to import prompts and the chain now what is the prompt and chain let's see let's talk about that real quick so prompt is basically Al um how you send information to an llm for querying something and this is where we will be sending our user question as well as our context and chain is basically how do we want to change this uh this whole process so basically we are saying hey use this user query use this embedding model to embed or create the vector for the user query do a similarity search and then use all the documents retrieved using similarity search as a context to the llm so for that we'll use something called retrieval QA which is question and answer and let's go back here so llm is llm uh I have to pass the llm to this response method we will pass it from there and then what is the vector store here uh let's see real quick and I'm going to say okay actually we have created the vectors store um if you see right here this is my let me just make the method here it's easier that way so let's just say get response and I pass it llm we'll also pass it the vector store which is nothing but the F Index right there which is right there and what is my users question so actually we did not bring the question so let's just say sorry we'll create a text field st. text input and then let's ask uh please ask your question for example and then we'll pass this question also as uh the user query to the response and the response will actually give us some uh text back and then we'll just write that text back to uh the screen okay all right let's go back here uh what did I miss here oh I missed the okay so what what we doing here once our index is ready we are showing a text input for where the user can answer the uh uh sorry user can ask the question we'll use that question to [Music] to uh get the contextual documents from using similarity search from the index that we built and then we'll use that question as well as the context to query the large language model and and whatever response we get we are going to display it here I just say do success let's just call it done once our uh LM is able to answer properly let's go to get response here so here is the prompt template so all the models take different kind of prompt templates so you have to check the models documentation in this case for cloud we are actually sending human human is saying please use the given context only to provide concise answer if you do not know the answer just say I don't know don't try to make up the answer like how we ask it a question who won the World Series it could not answer here I'm passing the context and um and my question is actually I can close it doesn't really matter but yeah my question is here an assistant is here in the prompt template I'm using this prompt template and using these input variables to provide them with the context now retrieval so this is called Vector store so I have to get the vector store as well as the question okay so now using the prompt template I'm using something called retrieval QA which is a chain and this chain actually is U as I said it will do the similarity search to our um embedding or vector store and it will find five relevant documents or five relevant text chunks which we'll be using to send to the uh actual large language model once it has the result we'll return the result and we'll display to the user all right let's go build real quick and run the docker image okay let's go to our screen and let's refresh this okay so I'm I'm able to see the index is ready now I can ask my question so let's ask a question when should I see a doctor about arthritis let's see what it says it's quaring okay so based on the information or context we provided here are some of the things and it's totally relevant to document that we index let's call it let's ask another question who won the Cricket World Cup in 2023 let's see what it says okay so as expected it does not have enough contacts we are because we are only providing contacts from the arthritis PDF that we indexed uh which we create index for so that is it so our we we were able to build our admin site as well as our user site in this particular video I hope you really liked it and please please like share and subscribe to my channel it helps me stay motivated to create more content like this and again if you have any Cloud project coming up you can reach out to me if you need any Consulting help you can reach out to me at Girish joft docomo so much I hope it was helpful I hope you it will be helpful for some of your projects and uh please like share and subscribe to my YouTube channel thank you so much I will see you in the next video

Channel: My Cloud Tutorials

Views: 6,130

Keywords:

Id: KFibP7KnDVM

Length: 64min 42sec (3882 seconds)

Published: Fri May 03 2024