Python Advanced AI Agent Tutorial - LlamaIndex, Ollama and Multi-LLM!

if you're interested in AI development then you're in the right place today I'm going to be showing you how to build an advanced AI agent that uses multiple llms we're going to run our agent through a series of steps we're going to connect the two agents together we're going to parse the output and you're really going to see how to get a bit more advanced here and how to do some really cool stuff with llms running on your own computer we're going to be doing everything locally we're going to use something known as ol llama and llama index and you're really going to get a taste of what AI development is capable of in the short video that even beginner or intermediate programmers can follow along with so with that said let's get into a quick demo then I'll walk you through a step-by-step tutorial on how to build this project so I'm here inside of vs code and I'm going to give you a demo of how this works now the concept is that we're going to provide some data to the model this data will be a coding project that we've worked on we've kept it simple for this video but if you scaled this up you could provide it a lot more code files and it could handle all of those potentially at the same time so you can see that we have a readme.pdf file in our data directory and this is a simple kind of documentation for an API that we've written we then have a test. pi file and this is the implementation of that API in Python so the idea here is we want our agent to be able to read in this information and then generate some code based on the existing code that we already have so I've just run the agent here and I've given this a sample prompt that says read the contents of test.py and write me a simple unit test in Python for the API now the idea here is that we've provided this agent some different tools that can utilize and it will decide when it needs to utilize the tools and use them to get the correct information so in this case it will use a tool to read in the test.py file it will then use a tool to actually parse and get access to the information in the readme.pdf it's then going to generate some code for us and then what we'll do is use another model to parse that output and save it into a file so you can see that when I ran this it did exactly that now it doesn't always give us the best result because we are running these models locally and I don't have a supercomputer here so I can't run the best possible models but it's showing you what's possible and obviously all of this is free you don't need to pay for anything because we're running it locally using all open-source models so what this did here is generate this test API file you can see there's a few minor mistakes but if I fix these up by just adding the correct parentheses and removing the Escape characters here you see that we actually have a functioning unit test written for our flask API so this is pretty cool we might need to change this a little bit to make it work but it just outputed all of that for us based on the context of the files that we provided to it now if we go here and read through kind of the output we're getting from the model you can see that it's actually sharing with us its thought process so it says the current language of the user is English I need to use this tool to help me answer the question it's using the code reader tool it passes the file name equal to test.py it reads in the contents of test.py and then it says okay I can answer without using any more tools I'll use the uh users language to answer to write a simple unit test blah blah blah you do this it then writes all of this code now behind the scenes what actually happens is we parse the output of this using a secondary model and take just the code and then generate it into a file and that file name will be generated by a different llm to make sure it's appropriate for the type of code that we have so it is a multi-step process here it worked quite well well and you can see it also gave us a description of what the finished code was so that's what I'm going to be showing you how to build I know that it might seem simple but it's actually fairly complex and it uses a lot of different tools which are really interesting to learn about with that said let's get into the full tutorial here and I'll explain to you how we can build this out completely from scratch so let's get started by understanding the tools and technologies that we need to use to actually build out this project now what we need to do for this project is we need to load some data we need to pass that to our llm we then need to take the result of one llm pass it to another llm and then we need to actually use a tool and save this to a file there's a few different steps here and if we wanted to build this out completely from scratch that would take us a very long time so instead we're going to use a framework now of course we're using python but we're also going to use llama index now I've teamed up with them for this video but don't worry they are completely free and they provide an open source framework that can handle a lot of this heavy lifting and specifically is really good at loading in data and passing it to our different llms I'm on their website right now just because it explains it nicely but you can see llama index is the leading data framework for building llm applications it allows us to load in the data which you'll see right here index the data query it and then evaluate it and also gives us a bunch of tools to connect different llms together to parse outputs you're going to see a bunch of advanced features in this video now as well as using llama index we're going to use something called ol llama now ol llama allows us to run open-source llms locally on our computer that means we don't need to pay for chat GPT we don't have to have an open AI API key we can do all of this locally so the summary here is that we're using python llama Index o Lama we're also going to throw in another tool which is new from llama index called llama parse don't worry it's free as well and all of that is going to allow us to build out this Advanced AI agent that has rag capabilities rag meaning retrieval augmented generation whenever we're taking this extra information and passing it into the model that's really known as rag I have an entire video that discusses how rag works you can check that out here but for now let's get into the tutorial and let's see exactly how we can build this application so I'm back on my computer and we're going to start by installing all the different dependencies that we need we're then going to set up AMA and then we'll start writing all of this code again AMA is how we run the models locally we need to install that first before we can start utilizing it now there are some prerequisites here so what we're going to do is I have a GitHub repository that I'll link in the description and in that GitHub repository you'll find a data directory that contains a readme file and a test.py file now you don't need to use these specific pieces of code but what you should do is create a data directory in a directory in VSS code so I've just opened up a new one here in vs code made a new folder called data and then put these two files inside of here specifically we want some kind of PDF file and some kind of python file we could have multiple of them if you want but the concept is this is the data that we're going to use for rag the retrieval augmented generation so you need something inside of here so either take it from the GitHub repository or populate it with your own data now from the GitHub repository as well there is a requirements.txt file please copy the contents of that file and paste it into a requirements.txt file in your directory or simply download that file this is just going to save you a lot of headache because it has all of the specific versions of python line libraries that we need in order for this project to function properly so really again we want to get this data directory populated with some kind of PDF and some kind of python file we then want this requirements.txt file again you can find it from the link in the description I'll put a direct link to the requirements.txt so you can just copy the contents of the file or you can download the file directory uh directly sorry and put it inside of your vs code folder now that we have that what we're going to do is make a new python virtual environment that's where we're going to install all these different dependencies so that we have this isolated on our system so to do that we're going to type the command python 3- MV EnV and then I'm going to go with the name of AI you can name this anything that you want if you're on Mac or Linux this is the correct command if you're on Windows you can change this to Simply Be Python and this should make a new virtual environment for you in the current directory this is where we'll install all of the different python dependencies now once we've done that we need to activate the virtual envir the command will be different depending on your operating system if you are on Windows or sorry if you're on Mac or Linux so Mac or Linux here you can type Source the name of your virtual environment which in this case is AI slash bin and then slash activate and you'll know this is activated if you see the AI prefix you can ignore this base prefix for me it's just because I have kind of a separate installation of python but you should see this prefix in your terminal indicating that this is activated now if you on Windows what you're going to do is open up poersh and you should be able to type SL AI slash and then this I believe is scripts and then slash activate and that should activate the virtual environment for you otherwise you can just look up how to activate a virtual environment on Windows and again make sure you have this prefix once the virtual environment is activated if we want to deactivate it we can type deactivate we're not going to do that though and we can install the different packages that we need so what we can do is type pip 3 install Das R and then requirements.txt notice this is in the current directory where we are when I do that it's going to read through all of the different requirements and then install them in this python installation or in this virtual environment so we'll do that it's going to read through requirements.txt and install everything for you this is going to take a second for me it's already cached so it's going pretty quickly and that should be it so once this is finished I'll be right back and then we can move on to the next steps all right so all of that has been installed and the next thing we need to do is install olama olama again lets us run all of this locally so in order to install AMA I'm just going to clear in my terminal here I'm going to go to a new browser window and paste in this URL here I'm going to leave it in the description now this is the GitHub page for olama and it shows you the installation setup steps here so again this will be linked in the description so if you're on Mac or Windows you can see the download buttons right here Linux this will be the command I'm on Mac so I'll just click on download when I do that it's going to download the olama installation for me once that's done I'm going to unzip this and then I'm going to run the installer now I've already installed it but I will still run you through the steps and then on Windows same thing you're going to download this and then run through the installer and what this will do is download a kind of terminal tool for you uh that you'll be able to utilize to interact with AMA you can see it says AMA run and that's something like llama 2 and this will actually download the Llama 2 model for you and then allow you to utilize it and interact with it in our case we're actually actually going to use the mistal model but there's a bunch of different models here that you could install and there's a bunch of other ones as well these are just some examples of ones that you can use okay so what we'll do here is unzip this folder and once it's unzipped we're going to run the installer so you can see here that I can click on AMA 2 that's going to load the installation tool for me so I'm going to go ahead and click on open we're going to move it into applications and then we should be good to go with AMA so we'll go next and then it says in install the command line okay so we're going to go ahead and install it again I already have this installed so I'm not going to run through this tool but once you do that you should be able to run the oama command which we'll do in just one second all right so once you've gone through that installer what you can do is simply open up a terminal which we're going to do here and we can type let me just zoom in here so we can read this o Lama and just make sure that that command works so if we get some kind of output here we're good and then we can type AMA run and then we're going to run the mistal model okay so you can see here it shows you all the different models you can potentially run in this case this one is 7 uh billion parameters it's 4.1 GB there's a lot larger models here which obviously would perform better but they need some more intense Hardware to actually run properly so we're going to install mistal which is only 4 GB by doing AMA run mistol it's going to then download that model for you and then we can utilize it now in my case it's already downloaded so what I can do is start interacting with it by typing something like hello world and then it's going to give me some kind of output perfect so what I'm going to do now is I'm going to quit this so I think I can just type quit or something uh or contrl C contrl D okay let's get out of that contrl D I'm going to close this terminal and I'm going to show you now how we can run this from code so let it go through let it install it is going to take a second cuz that's to download all of this stuff and then we'll go back to VSS code and see how we interact with AMA from our code all right so I'm back inside of VSS code here and I'm going to continue here by creating a file now this file will be main.py and the idea here is just to initially test o Lama and make sure that it's working as an llm so I'm going to say from and this is going to be llama index. llms if we type this correctly dot o Lama like that we're going to import AMA and then we're going to say llm is equal to olama and inside of here we're going to say the model is equal to mistl because this is the one that we want to use and we can provide a request timeout equal to something like 30 seconds just so that it doesn't take too long now that we have the llm we should be able to do lm. run and then we can say something like hello world if we say the result is equal to this we should be able to print out the result so let's see if that's going to work I can type Python 3 and then main.py we'll give this a second and we'll see if that was a valid command or not or if we need to use a different one so actually my bad here guys rather than lm. run this is going to be lm. complete and then we can type in something like hello world and if we run this we should see that we get some kind of output give this a second it says hello here's a simple hello world program gives us the output and there we go so this was just a simple test to make sure that ama was running locally on our computer we also can run different types of local models for example in a second we're going to run a code generation one uh but now you can see that this is indeed working and we didn't need to have any API key use chat gbt Etc this is a local model running on our own computer so now what we want to do is set up a little bit of the rag application so we can load in some files pass that to the lolm and see how it can query based on that all right so let's go to the top of our program here and we're going to import a few things that we need in order to load our python file as well as to load our documentation we're going to start by looking at how we load in our PDF which is unstructured or semi-structured data depending on the way that it's set up and we're going to use something known as llama pars which can give us a much better parsing of this file so what I'm going to do is say from llore parse we are going to import and then with capital llama parse like that we'll talk about this in one second don't worry I'm then going to say from llore index. core and we're going to import the vector store IND index and the simple directory reader as well as the prompt template while we are here we're then going to say from llama index. core Dot and this is going to be embeddings and we're going to import the resolve embed model and I believe for now that is actually all that we need so let me break down what we're about to do here we need to load in our data now in this case we're loading in PDF documents but with llama index we could load in really any type of data we want and in the previous video I showed you how to load in for example CSV data but in this case we have a PDF now what we need to do is we need to parse the PDF into logical portions and chunks for example if the PDF had something like a chart we'd want to extract that because that's some structured data that we could be able to look at then once we have that we need to create a vector store index now a vector store index is like a database that allows us to really quickly find the information that we're looking for rather than having to load the entire PDF at once what's going to happen is our llm is going to utilize this database and extract just the information that it needs to answer a specific query or a prompt now the way that we'll build this Vector store index is by creating something known as vector embeddings vector embeddings take our textual data or whatever type of data it is and they embed it into multi-dimensional space which allows us to query for it based on all different types of factors based on the context based on the sentiment we don't really know exactly how it works it's handled by llms and some machine learning models in the background and I'm not quite qualified to talk about it in this short section of the video but the point is that rather than loading all of the data at once we're going to query this Vector store index which is like a really really fast database it's going to give us the information we need inject it into the llm and then the llm will use that information to answer the prompt so really all that means for us is we've got to create this index and I'm going to show you how to do that all right so to do this we're going to delete these two lines because these were really just for testing but we will leave this llm what we're going to do is start by setting up a parser now the parser is going to be a llama parse and then we can specify what we want the result type to be which in this case is markdown now llama pars is a relatively new product that's provided by llama index what this will do is actually take our documents and push them out to the cloud they'll then be parsed and then that parsing will be returned to us now the reason we use something like this is because it gives us significantly better results when we are trying to query pieces of data from something like a PDF which is typically unstructured I'll talk more about it in a second because we do need to make an account with llama parse but again it's totally free you don't need to pay for so we're going to make this parser and then what we're going to do is we're going to create a file extractor now the extractor is going to be a dictionary and we're going to specify a file extension which in this case ispdf and we're going to say whenever we find a PDF we want to use this parser which is llama parse to parse through the PDF and then give us back some results that we can then load next we're going to say documents is equal to and this is going to be the simple directory reader and inside of here we're going to specify the directory that we want to read from which is the data directory and then we're going to specify our file extractor here so file extractor is equal to the file extractor that we specified and then we're going to say load data okay so let's write that now if we hover over this you can see that what this is doing is loading data from the input directory so we're using llama index we have something called a simple directory reader what what this will do is go look in this directory grab all of the files that we need and then load them in and use the appropriate file extractor now that we've done that what we can do is we can pass these different documents which have been loaded to the vector store index and create some Vector embeddings for them so we're going to say the embed uncore model is equal to the resolve embed model and then this is going to look a little funky but we're going to type local colon and then b a a a i/b g- M3 now this is a local model that we can use because by default when we create a vector store index it's going to use the open AI model like something like chat GPT we don't want to do that we want to do this locally instead so what we're doing is we're getting access to a local model and this model will be able to create the different Vector embeddings for us before we inject this data into the vector store index so I know it seems a bit weird but we're just grabbing that model this is the name of it here and we're specifying we want it locally which means the first time we run this it's going to download that model for us and then use it okay we're then going to say the vector index is equal to the vector store index and then Dot from documents and then we're going to pass the documents that we've loaded here with the simple directory reader and we're going to specify manually the embed model is equal to the embed model that we got above which is our local embedding model now that we've done that we're going to wrap this in something known as a query engine so we can actually utilize it to get some results so we're going to say query engine is equal to the vector index. as query engine and the llm that we're going to use is going to be the olama llm now what this means is that I can now utilize this Vector index um as kind of like a question and answer bot so what I can do is I can ask it a question like what are the different roots that exist in the API and it will then go utilize the documents that we've loaded in which in this case are the PDF documents in this readme.pdf file and it will give me results back based on that context now in order to test that we can say query engine Dot and then we can actually send this a query and we can say what are some of the roots in in the API question mark and then it should give us back some kind of reasonable response now we do need to print that so we'll say result is equal to this and we will be able to run this code in 1 second once we get access to the API key for llama parse so let me show you how we do that so I am going to show you how to use llama pars here but I quickly want to break down what it actually is so on February 20th 2024 llama index released llama cloud and llama parse now this brings production context augmentation to your llm and rag applications now llama parse specifically is a proprietary parsing for complex documents that contain embedded objects such as tables and figures in the past when you were to do some kind of querying over this data you get really really bad results when you have those embedded objects so llama Parts is kind of the solution to that where it will do some parsing and actually break out these embedded objects into something that can be easily ingested and understood by your model this means you'll be able to answer complex questions that simply weren't possible previously as you can see rag is only as good as your data if the da data is not good if the vector index isn't good then we're not going to get good results so the first step here is that we parse out our documents into something that's more effective to pass into the vector index so that when we eventually start using it we get better results which drastically affect the accuracy uh in a good way so you can kind of read through here and you can see exactly what it does I'll leave this link in the description but just understand that what this does is give us much better results when we are parsing more complex documents specifically things like PDFs so what we're going to do here is create a new llama cloud account you can do that just by signing in with your GitHub I'll leave the link below and this will give us access to a free API key so we can use the Llama parse tool all right so once you've created that account or signed in you can simply just click on use llama parse it's pretty straight forward here and then what you can do is you can use this with a normal API or you can use it directly with llama index which is exactly what we're doing here so what we want to do is just get access to an API key here so we can click on API key and we can generate a new key I'm just going to call this tutorial I'm going to press on create new key you can read through the docs if you want but I'm just going to copy this key and we're going to go back into our python file and I'm going to make a newv file here and then I'm going to create an environment variable that stores this key uh that we'll have access to in our code so we're going to say that this is llore Cloud _ aior key this is going to be equal to and then I'm going to paste in that API key obviously make sure you don't leak this I'm just showing it to you for this video and I'll delete it afterwards then we're going to go into Main dopy and we're just going to load in that environment variable and it will automatically be detected by our parser so I know I went through that quickly but the basic idea is we're going to make a new account here on llac cloud we're then just going to use the free parser so we're going to go and generate an API key once we generate the API key we're going to paste that inside of an environment variable file with the variable llama Cloud API key we're going to close this then we're going to go to our python file and we're going to write the following code which will allow us to automatically load in this environment variable so we're going to say in lower cases from. EnV import load. EnV we need to spell these correctly though and then we're going to call this function and what the load. EnV function will do is look for the presence of a EnV file and then simply load in all of those variables which will give this line right here the parser access to that variable so we can use llama parse great so now that we've written this code we can test this out so what I'm going to do is type Python 3 and then main.py we're going to wait a second for it to install everything that we need and then we're going to see if we get some kind of output all right so this is finished running and you'll see that what happened here is it started parsing this file using llama parse it actually pushed that out to the cloud which means if we had hundreds of files thousands of files Etc we could actually handle all of those push them out to L par and then get the results back then what it did is gave us the result here after querying that PDF so it says the API supports several roots for performing various operations these include SL items SL items items ID items ID Etc so it used that context to actually answer the question for us so now that that we've created this this is going to be one of the tools that we provide to our AI agent the idea here is that we're going to have this and we're going to have a few other tools we're going to give it to the agent and the agent can use this Vector index and this query engine to get information about our PDF or about our uh API documentation and then using that information it can generate a new response and answer questions for us so the agent is going out there utilizing multiple different tools maybe it combines them together maybe it uses one maybe it uses two 3 Etc and then it Aggregates all of the results there and gives us some kind of output so now let's look at how we start building out the agent and we'll build out another tool that allows us to read in the python file because right now we're just loading in the PDF all right so we're going to go back to the top of our program here and we're going to say from llama index. core. tools and we are going to import the query engine tool and then the tool metadata then what we're going to do is we're going to take this query engine so I'm going to delete this right here and we're going to wrap it in a tool that we can provide to an AI agent so I'm going to say tools are equal to and then this is going to be query engine tool for the query engine this is going to be the query engine for our PDF or for our API documentation we're then going to say metadata is equal to and then this is going to be the tool metadata and here we're going to give this a name and a description now the name and the description will tell our agent when to use this tool so we want to be specific so I'm going to call this API documentation and then we want to give this a description so we're going to say the description is equal to and I'm just going to paste in the description to save us a little bit of typing here okay so let's paste it in and this says this gives documentation about code for an API use this for reading docs for the API okay so we're just giving some information about the query engine tool now we are going to write another tool in here in a second but for now I want to make the agent and then show you how we utilize the agent so we need to import another thing in order to use the agent here so we're going to go up to the top and we're going to say from llama index Dot and this is going to be core. agent and we're going to import the react agent okay we're now going to make an agent so we're going to say agent is equal to react agent. from tools and we're going to give it a list of tools that it can use okay so we're going to say tools and then we need to give it an llm which we're going to Define in one second we're going to say verbose equals true if you do this it will give us all of the output and kind of show us the thoughts of the agent if you don't want to see that you can make this false and then we're going to provide some context to this which for now will be empty but we'll fill in in 1 second okay so this is is great but what I want to do now is I want to make another llm that we can use for this agent because we want this to generate some code for us rather than just be a general kind of question answer bot so what I'm going to do here is I'm going to say my code llm is equal to O llama and we're going to use a different llm and this is going to be model equal to code llama now code llama is something that does uh code generation specifically so rather than using the normal m model which we had here we're just going to use the code llm because we want to do code generation so now I'm going to pass code llm here and you can see how easy it is to utilize multiple llms locally on your own computer again all of these are open source and when you do this it should automatically download it for you okay last thing we want to do is provide a bit of context to this model so just to clean up our code a bit we're going to make a new file we'll call this prompts dopy and inside of prompts I'm just going to paste in a prompt for the context for this model okay you can find this from the link in the description from the GitHub but it says purpose the primary role of this agent is to assist users by analyzing code it should be able to generate code and answer questions about code provided now you can change that if you want but that's really what it's doing right it's going to read code analyze it and then generate some code for us so that's what we're doing so now what I want to do is import that context so I'm going to go to the top I'm going to say from prompts import and then context and then down here I'm going to pass that context variable okay so now we have made an agent and we can actually test out the agent and see if it utilizes these tools so let's do a simple W Loop here and let me just make this a bit bigger so we can see it we're going to say while prompt colon equals to input and we're going to say enter a prompt and then we're going to say Q to quit so if you type in Q then we're going to quit and we're going to say well all of that does not equal Q okay so let's type this correctly then we are going to do the following which is result equal to agent. query and then we're just going to pass in the prompt and then print the result okay so you might be wondering what we just did well we simply wrote an inline variable here using something known as The Walrus operator in Python this just means it's only defined in the wall Loop and it will get redefined each time the wall Loop runs just to make things a little bit cleaner and we say okay let's get some prompt when it's not equal to Q then we'll simply take the prompt pass it to our agent the agent will then utilize any tools it needs to and then it will print out the result so let's test this out and see if it's working so let's clear and let's run and this time we're not just going to be using the API documentation Vector index we'll use an agent and it will decide when to utilize that tool I know it seems a bit weird because because we only have one tool right now but imagine we had 20 tools 30 tools 100 tools then the agent would pick between all of them and have the ability to do some really complex stuff all right so this is running now I'm going to give it a prompt I'm going to say something like send a post request to make a new item using the API in Python okay let's see what this is going to give us here and if this is going to work or not okay sweet so it looks like that worked if if we go here we can see that we get I need to use a tool to help me answer this question API documentation it's looking for post items okay the IPI documentation provides information on how to create an item using the post method I can answer the question to create a new item blah blah blah and then it generates the response and then it gives it to us here right to create a new item we do this import requests URL payload response okay that actually looks good and then come down here and it says this will create a new item and then we can ask it another question or hit Q to quit perfect so that is working however I want to add another tool to this agent that allows it to load in our python files so llap pars itself can't handle python files that's actually not what it's designed for but what we'll do is we'll write a different tool that can just read in the contents of any code file that we want and then give that into the llm so this way it can have access to the API documentation and it can also have access to the code itself so it can read both both of them so let's start doing that and the way we'll do that is by writing a new file here called Cod reader. piy so let's go inside of code reader and we're going to make a new tool that will then pass to our llm so we're going to say from llore index. core. tools Imports the function tool now this is really cool because what we can do is wrap any python function as a tool that we can pass to the llm so any python code that you'd want the model to be able to execute it can do that you just have to give it a description of the tool and it can actually call that python function with the correct parameters this to me is super cool and it really has a lot of potential and possibilities now I'm also going to import OS and then I'm going to define a function which will act as my tool so I'm going to say the code reader function and we're going to take in a file name okay now what we're going to do is say path is equal to os. path. joint in and we're going to join the data directory and the file name because we want to look just inside of data here perfect and then we're going to try to open this so we're going to say try we're going to say with open okay and this is going to be the path and then we're going to try to open this in read mode as F then we're going to say the content equals f. read and then we can simply return the file uncore content and this will be the content okay then we're going to have our accept exception as e we got to spell accept correctly and then what we're going to do here instead is we're going to return some kind of error and that error is going to be the string of e and that's it that's actually all that we need for this function now this is something that we can wrap in a tool and we can provide to the agent it can then call this function and get back either the file content or the error that occurred okay so we're going to say the code reader is equal to the function tool and this is going to be doore defaults and then we're going to say FN standing for function is equal to the code reader Funk and then what we need to do similar to before is we need to give this a name and we need to give this a description so the agent knows what to use or when to use this so we're going to call this the code reader and for the description I'm just going to paste in a description like I had before four let me see if I can move this onto separate lines okay let's just do it like this so that you guys can read it okay so it says this tool can read the contents of code files and return the results use this when you need to read the contents of a file perfect that looks good to me hopefully that's going to work for us and now we can go to main.py and we can import the code reader tool so we're going to say from code reader import code reader which is our tool our function tool and now we can just simply pass that in our list of tools so imagine right you could write any python function you want just wrap it like I said or I did here with the function tool and then just pass it in this list and now all of a sudden your agent has access to this and it can start manipulating things on your computer interacting with python functions this really makes the possibilities of Agents quite unlimited and that's what I really like about this okay so we have the code reader tool now and we also have the API documentation so now our agents should work exactly as before and we can simply read the contents of that file so let's try running this and see what result we get when we give it a prompt that asks it to say read that file okay so start parsing the file and then we'll write a prompt and we'll say something like read the contents of test.py and generate some code okay so read the contents of test.py and give me the exact same code back now remember we're running some local models that don't have a ton of parameters and aren't the best ones so we're not always going to get the best result but I hope this is going to work or it should give us at least some kind of result so you can see it says okay I need to use a tool so it says we're going to use the tool code reader file name test.py and then it gets the contents of the file and then what it says here is you provide a python script that contains an inmemory database for Simplicity which implements a list called items the script defines four endpoints one for creating new items blah BL blah blah and then gives us this whole result so it didn't give us the code that we wanted but it did actually give us a description of what was inside of that file which to me says this is indeed working and notice it only used this tool it didn't use the other tool because it didn't need that for that specific prompt okay so I think that's good that means that it's working and we're able to utilize both the tools now the next thing that we need to do is we need to take any code that this model is generating for us and we need to actually write that into a file now this is where we're going to use another llm so what we want to do is we want to get the result that we just saw there we want to determine if it's valid code and then we want to take that code and write it into a file now in order to do that we need an llm to analiz the result of this output so what we're really going to do right is we're going to take this result we're going to pass it to a different llm and that llm is going to have the responsibility of taking that result and formatting it into something that we can use to write the code into a file now I'm not sure if I'll be able to show this here because I think I cleared the console yeah I did but you would have seen before that it gives us some code output but the code is mixed with like descriptions and other information that we don't want to write into the file so the other lm's job is going to be to parse that output into a format where we can take it and we can write it into the file so let's start writing that this is where it gets a little bit more complicated but I also think it's where it gets quite cool so we're going to go to the top of our program here and we're going to start importing some things that can do some output parsing for us so we're going to say from pantic import the base model we're then going to say from llama index. core. output and I believe this is underscore parsers we are going to import the pantic output parser we're then going to say from llama index. core. query pipeline inut import the query pipeline which allows us to kind of combine multiple steps in one so now we're going to scroll all the way down and after we create our agent and our code llm we're going to start handling the output parsing so we're going to make a class here and we're going to say class code output and this is going to be a base model from pantic then what we're going to do is Define the type of information that we want our output to be parsed into now this is super cool because we can use l index and these output parsers to actually convert a result from an llm into a pantic object so we can specify the type that we want in the pantic object and then llama index and another llm can actually format the result to match this pantic object it's super cool so I'm going to say code and this is going to be type string I'm going to say description and I want this to be a string as well but we could make it other types but in this case we're just going to need strings and then I'm going to say file name is a string okay so I've just made a pantic object this is just a class that we're going to use to do our formatting and then we're going to write some things for our querying so we're going to say parser is equal to the pantic output parser if I can write this here and we're going to pass the code output which is specifying we want to uh use this pantic output parser and get our result and pass it into this code output object we're then going to have a Json prompt uncore string and this is going to be equal to a parser do format and we're going to format the code parser template which is a variable that I'm going to write in 1 second so now what we're going to do is go to prompts and I'm going to write in a prompt here I'll explain how this works but you just got to bear with me because there is a bit of code that we need to write okay so let me copy this in again you can find this from the GitHub or you can just write it out yourself and what this says is parse the response from a previous llm into a description and a string of Val valid code else will come up with a valid file name this could be saved ELO come up with a valid file name this could be saved as that doesn't contain special characters here is the response and then this is the response from the previous llm you should parse this into the following Json format okay so this seems weird but this is what I'm providing to my output parser to tell it how to take the result from this llm and parse it into the format that I want so let's import that and then we'll look at how this works so from here here we're going to do the code parser template and then I'm going to pass the code parser template here now with the parser do format will do is it will take this string and it will then inject at the end of that string the format from this pantic model so I've written my pantic output parser past the code output it's saying hey this is the format we want code string description string file name String what the output parser will do when I do parser do format is it will take this format find the Json representation of it and then pass it or inject it into the code parser template so then when I start using this template on the next step it knows the type of format we want the output to be in so now I say my Json prompt uncore template is equal to and this is a prompt template and I simply pass my Json prompt string now at this stage what we do is we write uh kind of a wrapper on the prompt template so we can actually inject inside of here the response so the response if we look here is this okay just bear with me this will make sense as we get there and then lastly we're going to make an output Pipeline and the pipeline is going to look like this it's a query Pipeline and the query pipeline is going to have a chain and the chain is going to go that we first need to get the Json prompt template where then going to get whatever we need in the template and then we're going to pass that to our llm and notice this time time I'm using my normal mistal llm which is this one right here I'm not using the code llm because I want a different llm for this task more general purpose one not one specifically for code okay so now we have our output pipeline so the idea here is that what we want to do is we want to take the output Pipeline and we want to pass this result to it and then we're going to get the result back which is going to be that formatted object that we want to look at so I know this is a bit complicated but that was kind of the point of this video is to make it a bit more advanced and you're going to see now how we do this so we have the result from agent. query prompt now let's take that result and pass it to our next agent so we're going to say next result is equal to the output pipeline. run and we're going to pass the response equal to the result so whatever the result was from the first agent that's now what we're passing is this variable right here in this code parser template prompt then we can print out the next result and we can see what we get now keep in mind this doesn't always work there is sometimes some errors based on how the parsing occurs but overall it's pretty good so let's go up here and let's run our agent again and let's get it to do a similar thing that it did before where it calls like a post endpoint or something okay so a similar prompt as before read the contents of test.py and write a python script that calls the post endpoint to make a new item let's type enter in here and let's see what we get all right so we can see that we get the kind of thought process here of the first llm which is that it needs to use the code reader tool which it does and then it generates this code and then what happens is we actually get output here from our second llm that says assistant and then it gives us this python object or this Json object really where we have the code which is all of the code that it generated which was this right and then it has the what else description use the request library in Python to do this and then it has a file name which is this so now that we have this kind of output what we want to do is take this output and load it as kind of valid Json um what do you call it uh data in Python I'm going to show you a fancy way to do that once we have that we can then access all the different fields like code description and file name and we can utilize those to save a new file on our computer so again we've gone through we've generated the code we've used our different tools from the rag Pipeline and then we've now parsed our open put into this format where we're able to utilize these different fields we just now need to load this in uh so that it's kind of valid for us to be able to view okay so now that we have that what we're going to do is the following we're going to say our cleaned Json is equal to and we're going to go up to the top of our program and we're going to import as now as is something that's going to allow us to actually load in Python code so what we'll do is we'll take the output from here and we'll load it in as a python diction AR so we're going to say. literal evaluation and then we're going to convert the next result into a string because it's actually a response object and we're going to replace the assistant which was kind of what was uh leading here with an empty string so all we're doing is removing that assistant that came before that valid python dictionary and then we're loading in the rest of this as a python dictionary object so now this is actually going to give us a python dictionary and what I can do is I can print code generated and then I can print what it is so I can say my clean Json and then access the code and then I can print my description so I'm going to go here and go back sln back sln description and this needs to be a back slash not a forward slash okay and then the description will be the cleaned Json of the description I can then say my file name is equal to the cleaned Json and this will be my file name okay so let's run this now and see if we get the correct output and then we're just going to add some error handling here and we're actually going to save the file because it is possible that some errors could occur so let's save and let's bring this up and let's quit and let's copy this prompt because this one worked and we will paste it again and we'll see if we're able to actually get all of those different fields and if we load in the the python object properly all right so you can see here that we're getting our result code generated which is this create item and then it has some Lambda function here and then we have the description and then we didn't print out the file name but we would have had the file name as well so it gave us a different result than we had last time um you can see this is indeed working and now we can quit and we can move to the next step which is a little bit of error handling and then actually saving the file so what we want to do now is just make sure that this works before we move forward because it's possible that we could get error so what we're going to do is retry this prompt or this sequence of steps a few times to just make sure it's hand or it's working properly sorry before we move on to the next step so we're going to say retries are equal to zero and we're going to say while retries are less than three so we'll just retry this three times then inside of here we're going to do this and we're going to say try and we're going to try the following we're then going to say accept and this is going to be exception as e we're going to say retries plus equal 1 here and then we're going to break if this happened successfully so what's going to happen now is we're going to retry this up to three times so every time we fail something happens here that's wrong we simply retry it and it will go and do this again with the same prompt that we typed now we can also do an error message here we can say print error occurred retry number and then we can make make this an F string and we can put in the number of retries and then we can print out what the error actually was okay so that should be our retry block I'm now going to come down here and I'm going to say okay if retries is greater than or equal to three which means this block actually failed we never successfully generated this cleaned Json then I'm simply going to say continue which means we're going to go back up here and ask for another prompt and I'm going to say print on able to process request try again okay now you've probably seen this sequence before but pretty much we'll try to do this if it doesn't work we'll just say hey you know that prompt didn't work for some reason okay give us another one because it's possible they ask us to do something we're not able to do or the output's not possible uh there's all kinds of errors that could occur here so we're just kind of handling that and cleaning it up a bit with this logic now if we do get down to this point here that means that we were able to actually generate this code so what we can do is we can save it to a file so we can write a little try here and we can say try with open and we can say open the file name which we have here and then we can say that we want to open this in W as F and we can say f. write and we can write the cleaned Json code into that file okay then we can say print saved file and we can just print out the file name and then we can have an accept here and we can say print error saving file okay now just to make sure that we're not going to override a file name that we already have what we can do is we can do an os. path. joyin and we can make an output uh folder here so we can say output and then file name now we need to import OS so we'll go to the top of our program and import OS sorry I know I'm jumping around all over the place here then we can go here and we can make a new folder called output so now all of the output will just go inside of this folder so we don't accidentally override any files that we already have okay so kind of final uh run here let's give this a shot and see if this works so let's bring up the code let's clear and let's run and then we'll enter our prompt and we'll see if it gives us that generated code okay so we're going to use the same prompt as before and now what we're looking for is that we actually get a generated file inside of this output directory okay so it seemed this did actually work you can see it has the code generated here and then if we go into our output we have this create item file now we do need to remove this because there was uh a few characters that I guess it left in here what it's doing is looking for an access token open up test.py okay f. read response request. poost access token uh response. status so it's not perfect there's a few things that it probably shouldn't be doing here uh but overall it gave us kind of some good starting code or at least kind of prove the point that hey we can generate some code it is attempting to call the API it is calling it in the correct way so yeah I mean I would call that a success obviously we can mess around with a bunch of different prompts we can see what ones it works for ones it doesn't work for remember we're using these local models which are quite small which don't have the same capabilities of something like chaty BT if we did this at an Enterprise level with the best hardware with the best models obviously we'd get some better results but for now I'm going to quit out of that model and I am going to wrap up the video here all of this code will be available to download from the link in the description a massive thank you to llama index for sponsoring this video I love working with them their framework is incredible and it really just opens my imagination and eyes to what's possible with these llms I mean look what we were able to create in about 30 or 45 minutes obviously I was walking through it step by step was going slower than I would normally code this out and we have an advanced AI agent that can do some code outputting and parsing we're using multiple different agents locally and we can continue to chain these and do some really cool things anyways if you guys want to see more videos like this definitely leave a comment down below like the video subscribe to the channel and I will see you in the next one [Music]

Channel: Tech With Tim

Views: 59,200

Keywords: tech with tim, ai agent tutorial, python rag tutorial, advanced ai programming, python machine learning, python ai techniques, python coding ai, rag model tutorial, python ai projects, python programming, artificial intelligence tutorial, advanced python ai, rag agent creation, rag ai model, ai coding guide, ai technology guide, ai development, tech with tim ai, tech with tim python, machine learning python

Id: JLmI0GJuGlY

Length: 53min 57sec (3237 seconds)

Published: Thu Apr 18 2024