LangChain - Dynamic Routing - Retrieve data from different databases

one issue I frequently encounter with retrieval augmented generation is that people attempt to include all types of data into a vector store however not all data is well suited for Vector stores Vector stores excel at handling natural language data like books manuals emails and so on because embedding models capture the semantic meaning of texts but when it comes to data rich in numbers such as information found in tables or product prices it's not ideal for Vector stores in these cases the llm often ret Rees unnecessary information and may start to hallucinate so how can we address this we can Implement a router to retrieve data from different sources depending on the question we are the owner of a fictional Italian restaurant and host a chatboard where users can ask information about our restaurant like information about the owner the opening hours and so on this is text information which we want to store in a vector store and then we've got product information tabular data which is stored in a SQL database and we want a chatbot to be able able to handle both data sources okay I'm currently in we code and as you can see we've got multiple files here on the left we've got the code iPad notebook where we implement the routing then we've got the doer compos yaml where we set up our Vector database which also serves as normal SQL database with tabular data we've got the ingest dopy where we put our data inside our database here is the data we've got some products which we sell for different price here Bret lasagna and so on so this is for the fiction restaurant then we've got our requirements. txd which we have to install to make our code workk and then we've got our text.txt this is the text information about our restaurant like when it was established who is the founder what is the name of the restaurant and all that kind of information so let's first set up our Vector database with Docker compose up just a simple command to run everything and then we want to ingest our data inside our table so first step is to run pip install minus r requirements.txt to install the dependencies and after that we just insert these products in our database with python and then inest pi and now we can see that we've got these 10 products in or database and we want to query them later okay next step is to work with the jupyter notebook and what we're going to do here is we import some packages like PG Vector chat promp template open ey embeddings and so on everything we need for a reg application so first we create a connection to our database which is just running on Local Host we create an instance of OPI embeddings here we set up our PG vector vector store pass the collection name pass the database URL and also the embeddings function after that we load our text.txt which contains the information about our restaurant use the data loader and then split that document into multiple chunks and add the chunks to our Vector store after that we convert our Vector store in a retriever to have a standardized retrieval interface so that's pretty standard and after setting up the retriever we want to create a rag chain where we can now get the information from the retriever so we set up our prompt template where we've got two variables the context this is the information we retrieve from our Vector store and the question we set up our model to be GPT 3 to5 tobo to make it a little bit cheaper than gbd4 and then we use the L chain expression language to set up direct chain here we pass the context uh to the Retriever and this will retrieve the four most relevant documents and we let the question unchanged we pass that information to the prompt pass the information from the prompt to the model and pass the output from the model to an SDR output pass to just have text so let's try that out who is the owner of the restaurant this should be able to be answered yeah the owner of the restaurant is javanni matelli this is correct so this is how we work with our Vector store but what do we do with the tabular data to work with taba data we have to conver convert the normal language to a SQL query because yeah we've got a SQL database and SQL databases work with the language SQL so we create a new template with two variables the first one is schema which is our table schema and the second one is the question and base based on these two input variables the llm should write an SQL query that would answer the users's question so we create a new template from that and then we create a new instance of the database where we set up a connection to our database so we've got two other functions get schema which Returns the database schema and also a run query function which is able to run arbitrary SQL commands so now we have to construct our chain and this is how we construct our chain which Returns the SQL command so we have our runable pass through and assign this pass through our get schema function which we defined here so we pass that uh value to the prompt and pass the complete prompt to the model which will stop on that SQL result value and we pass the response of the model to the Str Str output passer so this is how it looks like let's run that and call the invoke method and here this is the SQL command we want to run to answer that question and now we want to pass that value to another chain a second chain and this is our template so again we pass the schema we pass the question but this time we pass also the SQL query and we also pass in the response from that query we create a chat prompt template from that template and then create a new SQL chain so here we've got multiple values we've got the query which we assigned to the runner the PA through we also assign the schema and a Lambda function here this Lambda function will execute this run query function where we pass in our initial query and save that in the response value this will be then passed to The Prompt response you can see schema question query and response these are the values needed we need um to create our final prompt which will be passed to the model and again we use the output parer to get a string as output and now we run the invoke method again with the question what's the most expensive dessert you offer so this should now be able to answer our question and the the most expensive dessert is terisu priced at 850 let's have a look at our ingest dopy and this is correct okay now we've got two chains which work fine independently but how can we combine them into a single chain this can be done with routing so the first step is that we need some kind of classification template and we want the llm to classify the question as database chat or of topic so if the question is about products of the restaurant or ordering food or something like that we want the question to be labeled as database otherwise if it's related to topics like opening hours then we want to classify it as chat and otherwise if it's off topic we want to label it as off topic so we use this classification template and create our chain like this so we have our template passed that to the model and passed that to the output pass so that's a very simple template and now we want to classify a question so how is the weather how would that be classified as you can see this is correctly classified as off topic and now to actually route it we can just Define a normal function which is called route we pass in a single argument which is the output of the llm and we want to check if that topic contains database or chat and if that's the case we want to Route this and otherwise we want to Route this and if it does not contain chat or database we just want to return a simple string because we don't have to make a request to openi and just provide I'm sorry I'm not allowed to answer questions about this topic so let's try that out currently we don't make a request this is in the code we commented out so first let's have a look if this actually works so this is where we put everything together we've got our dictionary here with the topic which is the output of the classification chain and we've got our question which remains unchanged and we pass it to this runnable Lambda function so we passed that complete dictionary to that function and now we want to invoke the question what's the most expensive dessert you offer so let's see how that's interpreted here this is the topic as you can see this is just printed here um so this is the the dictionary topic is database and this is the question as you can see this remains unchanged and this is the the answer so this has to be handled in a database so how will the weather be tomorrow let's see how that looks like I'm sorry I'm not allowed to answer about this topic and the third one is who is the owner of the restaurant and this has to be handled in the chat so let's not comment that out because this is just the code to check if our routing system works and if everything works we just comment that in and make use of our SQL chain or direct chain so let's run that code again and now create our full chain again and now we want to invoke the question what's the most expensive dessert and now we get a response from the llm so the most expensive dessert is the misiz which is correct this should return the same answer as before I'm sorry I'm not allowed to answer questions about this topic and now the chain will which should answer the question who is the owner of the restaurant which is information we've got in our Vector store so all of that works and this is how you can can combine multiple data sources like a vector store and a SQL database into a single llm call thanks for watching see you in the next video bye-bye

Channel: Coding Crash Courses

Views: 2,219

Keywords: langchain, sql-chain, rag-chain, retrieval augmented generation, rag

Id: nko60eGSYn4

Length: 10min 48sec (648 seconds)

Published: Thu Apr 11 2024