Building a RAG Pipeline with Anthropic Claude Sonnet 3.5

hey there guys so today we're going to be trying out the new um model from anthropic which is the clae Sonet 3.5 and one thing I want to do very with its coding abilities to see how it can create a rag pipeline from a diagram so I have this diagram that I created few months ago and this is just a a visualization of a rag pipeline you have the data loading process then you have the ingestion embedding and in ingestion into a vector database then you have the retrieval and augmenting the user query with the retrieve document and passing that into a large language model to give you better responses from Models such as GPT even Cloe itself and other large language model so what I'm going to do is we're going to take this diagram and we're going to pass it into CL sunet 3.5 and we're going to see if it can create a rag pipeline give us the code in Python and then we're going to run the code and see how well it's done so here is the image um I'm just going to upload it and I'm going to pass in a prompt that will basically tell it what I want it to do let me zoom in over here and let me move myself up here so I'm saying give me the python code for implementing a rag pipeline using mongodb as the operational and Vector um database I didn't even say database spell it uh operational and Vector database and using de line chain library and open AI apis so that is a poorly worded instruction but let's see how well it does so let's zoom out here and let's go so one thing um 3.5 does is it uses artifact so this is I guess um un we're going to see if this works we're just going to copy it and here is the copy button down here copy it and put it into a notebook and see if it worked so here is the notebook and we've just pasted the code one more thing there probably going to be a pip instore that we need to it's told us um where it is we need to install the required packages L chain open Ai and P mongle so let's do a pipin stall here so pipin stall it should have given us the P install um command P install uh Qui land chain and we need to install open AI uh yeah Pongo and open AI right and that's what it's told us to do so let's do what it told us to do and we'll take it from there so we will install this nice um then we can look through the code and this code looks relatively um it looks correct because we've imported all the modules we need which is good um we have the open AI key okay we have all the URI um requirements and we're using p to connect to a client nice and we're going to be using a text file interesting use the directory loader and convert the content of that into documents um and we're loading that into document and we're chunking uh and we using a chunk size of 1,000 and overlap of 200 okay and then we're creating a a object of our Vector store and using that as a retriever and then initializing our our llm and we create a a chain that has our llm specify the chain type the Retriever and return the source document from the query and then we process the user result so let's see this work so we've got all our installation cool um and let's see what we have to change here so we probably have to change the API key so I want to do one thing which is I have my API Keys stored under Google collab secrets and I can access it using this user data module so we're going to set our open the ID here so let's change that to um St user data dot let's say user data dot yes I think it the open cool so user data get and that will give us our API key in our environment uh if you're following this tutorial you can set it in in your local machine as well um so we need a DB connection so we need our URI URI string so you can get this when you create a free account on mongod DB um I'll put some instruction in the notebook when I upload this but we need a mongod DB URI and I'm just going to use the same process here um I think I called it you or I so using mongod DB as both the vector database and the operational database so we can start Vector embeddings we could Store The Meta metadata we could also store the operational data as well um so you don't need a separate Vector data B uh let's call this uh anthropic okay let's go uh collection name um let's go it read that I'm going to upload a Blog from anthropic as a PDF and we will load that in so we have the clients which is fine we get the client from the URI then we have the we get the database I've not created this database but DV doesn't see a database it will create One automatically and we have a collection and the collection will be called resarch cool so now we have a loader and one thing we want to do I want to change this I don't want to use a directory loader so this PDF is actually just something from anthropic where they they looked into I guess they call it mapping and Mind of a large language model I've not read it um but maybe I'll read it later it seem are interested but I just picked up one of the latest uh document converted the PDF and we can use that with our demo so we have that in there let's change the name of this we don't have to change anything it's just going to be a PDF we'll see how well this works so we're going to chunk it we're going to do the open a and but I think we're almost there we are almost there um so let me change this to a PDF loader instead that should be fine and then we can actually pass in the PDF directly which is located over uh what's the name named it uh this re right and we'll call it mapping llms which is cool uh we can call uh if we call this Mapp inm uhpd um and then we can yeah then we can use the load out in here and load it but one thing we have to do is actually bring in this loader and we'll bring it in from the document loader which should be over here so we're not using this anymore we're just going to use the loader from here we use the PDF loader cool so all right I think go back on track so let's see how far it goes so in previous situation we got to this line 925 we've made a little bit of a change but um let's see if we get further down no we're still stuck on this line let's see so error is uh yeah we're missing this PDF makes sense we're missing the pi PDF library and we can insert that again we can install that again so we're still stuck on this line 25 let's keep going we've not changed too much of the code so we we just keep going are we going to make it no we're stuck again why are we stuck here mapping llms not a valid file or URL my bad M this is still living within the folder let's take it out of the photo wow okay let's try that again are we going to make it past line 25 yes we did and we're stuck here in this line 33 let's see what the error is could not import Tik token okay um you know what I'm going I'm going to import the L chain um open AI Library so let's actually change this that should be good Library so this is just used the L this would just be the open AI integration within L chain um and that should bring in all the library need do minimal changes so we've so far we've had to install this new library and this fre new library essentially let's see do we get a error or a success okay it's wow worked okay so the capsu of Francis Paris sources no sources so there seems to be an issue I know what the issue is um but let me just say um one thing we need to check is if the data has been inserted into a mongodb database so remember the collection name is anthropic demo and the database name is anthropic demo The Collection name is search so we'll go into the database um here is my um mongodb database uh let's do a quick refresh of everything yes so anthropic demo is right here excellent so the code that clothe gave us actually created used the right modules to create the database and we have the collection which is excellent and we can see what's inside nice nice so we have text uh the chunk of the text and we have the embedding it's used um the default open air embedding that get from the L chain Library um I believe it's probably Adar or2 um we have the source and we have the page number as well okay this is good so we have metadata alongside the embedding so the data injection into a mongodb uh database as worked the embedding of the trunk as worked um I know what's wrong here and why am I not be using the source to get uh why am I not have any Source in is because we've not created our Vector search index so if you're following this tutorial the way you create a vector search index allow you to do I'll conduct a vector search you click on search index here I I'll walk you through this I'm going to move myself up here nice um it's like a motion uh so we go into the Json editor and we click next we call it Vector index which is fine remember to uh go into the exact collection you want and we will just create a vector search Index definition so the path is going to be embedded this is the path of where our bed this is the part of where our um Vector embedding for each chunk is located good Dimension is 1536 we saw that earlier on and the similarity function we're going to use is cosine and this should be good let's create that create search index excellent so now that's being created uh it it will do it relatively quickly so one thing we're doing is and you need to go through the step is create your ventor search index on mongodb as I've shown you very straightforward I'll put the link to the instructions on the notebook I share um with this for this video so we can go back this is probably going to be done soon and so far we have the we have the research we have the research collection we made a we made a connection to our our Mongol cross store and we ingest we loaded up the PDF converted into documents and chunk parts of the documents and embedded the chunks and passed it into a mongodb um Vector database and one thing we have to do right here is change it to um Vector index because that was the name of our index our Vector search index and we're using gpt3 uh GPT 3.5 turbo we're doing a simple similarity search and we're returning five document from the retriever that we've created from our Vector store and we create a chain which them specify the chain type and we have the Retriever and we're returning our source document which is fine and we're handling we have a function to process user query where we pass in the user query in the into the chain and we get the result next next thing we have to do is let's change this because this is irrelevant um what is a capital of front but let's change to something within the document so what is the docent let's see what's the document um okay so let's go and run this again and let's see what we get anticipation oh yes so it says the document is about a significant advance in understanding the inner workings of AI models specifically focusing on the interpret ation of features it's a large language model Claude Sonet 3.5 gave us the code to create a rag pipeline from AA diagram that I provided it right here so we provided it a diagram it gave us the code gave us instruction which is fine um told us we have to install our this packages and gave us instructions that we also had to get our connection URI and our open AI key and it gave us the code and if you see the initial code we've not changed much we changed how we're getting our keys fine we changed the strings here fine we meant to and I changed this part of the that exact PDF were loaded but after this I didn't really change much uh we changed the the index name which is also fine and this impressive and profit um take my money this is impressive and it works works we have the source that it's used um it used the PDF and it used the the exact uh it's shown the exact um uh extraction used to to to conduct Dan um that's impressive I'll put this code um on the Gen showcase repository um but let's touch on a few things right let's just see what um unpri is doing here with um the CLA 3.5 sunit so this was a Blog they released um yesterday 21st June today the 22 of June so here we see that they've launched clae 3.5 sonnet and this is first release in the fourth coming clae 3.5 model family so there's going to be more releases coming out from anthropic uh this year and to raised the bar for intelligence um and it's outperformed in a variety of evaluation we'll look at we'll look at it further down but clae 3.5 sunic cheaper and faster which is good it has 200k token Contex R which is interesting because they're not doing the whole 1 million context window um that we see with other mod providers doing and we see if we look at the previous benchmarks that they have and against the the intelligence against the cost we can see that sunet is a significant sunet 3.5 is a significant increase in intelligence and we're uh placed at relatively the same cost and I guess Opus is probably going to be somewhere up here if we're following this trajectory but one thing is I think they might release another um another model um because I think obus is good and I think as they moving more into a gentic systems they might release another model that is better at to calling um and other and other functionalities so they might call it let's say let's cuz Haiku is like a Japanese short poem it's sunet is a poem that is about 14 lines or so and Opus is like a collection of of composition so the next one to signify like the the scale could be called clae 3.5 Symphony or CL 4 Symphony he know he knows but Symphony like a like a Orchestra you have multiple instrument and playing and that could represent multiple agents or multiple applications within this single system so Symphony guys use that for your next model name but going back we can see that clo 3.5 is smarter and they've evaluated it on a bunch of um benchmarks and evaluation benchmarks used to see the reasoning capabilities of um language models and we can if we just go straight into this chart increase the screen it's quite impressive that it's performed uh quite well in relation to other um models that are currently top in the field right right now and cross ERS Benchmark it's done really well uh this is impressive results um if we scroll up and they've noted we can use cloes 3.5 for tasks such as customer support and mul workflows so very interesting and it's able to write and edit and execute code with sophisticated reasoning and troubleshooting capability So within the um agentic coding system you can use code 3.5 as your as your llm for your coding agents I might try this out um in another tutorial we will see but moving on uh they have a video here explaining the vision capabilities of son 3.5 so it's a multimodal um multimodal model um so it could process text and um uh images as well which is pretty cool and they have their own uh they have and their own um they have a vision um evaluation benchmarks and they ploted it against the rest of the models within the space but the interesting thing they had was the artifacts which we saw over the artifacts interesting I can see that being um moving forward being an interface where we can get connections to other applications within your within your system and you have the agent results um over I guess they're keeping humans in the leot within a gentic workflow so which is we'll see how this develops but this is interesting and it's it's not really new because I think uh with chat interface you can you can see it executing the code and the operation but I'm impressed the fact that it can just take a diagram and just give us an almost ready ready code for the R pipeline so they're currently going to be working on Enterprise version of um of of clae um which we can see that this whole Alat interface is to create a dynamic workspace collaborative work environment okay and entire essentially they want organizations to use clae to centralize their knowledge documents and and ongoing work in a Shar space and Claude said serving as an on demand teammate interesting so they they want to use clae essentially to augment um existing team members and Claude is going to be this agent that has um access and awareness of all the documents within your your your machine or within your drive and you can use it to perform actions which is very interesting I've already built I showed how can build in a previous video I showed how you can build an agent uh that can access Google Docs I guess they making CLA do something similar with documents and knowledges within your organization where you can basically just tell quote to create a document or review a document or maybe even send an email to you or an email to someone within your team with a link to a document who knows we can see that anthropic is going to continue developing on Cod and the code model family um they're going to be developing new modalities and features and focusing on business use cases for Enterprise um and integrative Enterprise application so what this means is we are probably going to see clae um be able to access applications such as slack um Zoom uh Google Docs maybe uh Word Microsoft Word and act as a agentic I guess co-pilot within within your whole system and I guess through the artifacts we can see the execution and interactions with all of the systems which is impressive um I've already shown done a video previously that shows how you can use an agent to access a Google doc but one thing is the team are exploring features like memory and I've spoken about the importance of memory within um agentic systems and how we improve agentic systems and why we need memory so um if you want to look at if you want a high level overview of what memory looks like within agent and when I say high level I mean not high level we'll talk some code as well you can look at this video um that I put that I did last week where you can see my hand movement but you can put this videos where I spoke about building AI agents with memory and I spoke about how mongodb within the agentic um system context is the memory provider um spoke about the different type of memories and in this video we implemented um a long-term memory which can store conversation history and a vector database and I also showed link to how you can create a semantic cache um implementation with Mongol DB so AI agents needs all of this various form of memory but more on that later in a future video but thank you for watching and see you in the next video don't forget to like And subscribe I always keep forgetting to say that but thank you for staying up all until the end and thank you for watching

Channel: Richmond Alake

Views: 3,588

Keywords:

Id: 3JAxRtD05zg

Length: 25min 41sec (1541 seconds)

Published: Sat Jun 22 2024