AutoGen Tutorial: ANY LOCAL LLM Chatbot with Ollama and LM Studio

hey and welcome back today I'm want to show you how to set up your own chatbot with autogen and not just that we're going to do with of course open AI API but also LM studio and O Lama this means you can run it on a local open- source llm for no cost and you get to choose whatever model you want and we'll be creating with all the updates from the recent versions of Auto let's get started once you have your ID open which I use Pyar Community Edition it's free Link in the description go and create a new project let's just call this autogen chat click create and we'll come back whenever it's done okay now that it's done open up the terminal and we need to install one thing and that's pip install Pi autogen give that a minute and we'll be back all right the first file we want to create is this Json file that's going to hold our config properties so right click go to new and just create a new file now we have to name it oi config list. Json I'm go ahead and paste this in and what this is is just an array of config properties and the first one we'll go over is LM studio so I have the model as LM Studio because the model name doesn't actually matter when it's local and open source well at least for LM studio so I just have it here so we can filter in the main python file by the same property and then the API key it's local open source doesn't matter and then this is the base URL for LM Studio likewise the next one this is actually for oama the reason I the model set to F is because that's the model that I'm going to pull from oama to used for this chatbot and the API key again it doesn't matter cuz it's local source and then this is the base URL for o Lama and of course if you want to use open ai's API you can give the model you want to use with your API key okay so now the next thing is we need to create the main python file so go and right click your directory new python file and just type in main we only need two Imports here and it's going to be import autogen and import Json and now we're going to create a filter criteria variable which is going to let us know which config of the list that we had to choose from so we're going to type in filter criteria is equal to and then the we want the model property that's what we want to filter by and you just type in an array here and the model that we want is to use LM Studio first we want we just want to use local I put the openi API there so that you know uh so you have the config to use but let's just use local open source so LM studio and then we need to create the config list so config list and it's going to be equal to autogen do config list from Json because it's a Json file and then that Json file is called oi config list. Json and then the filter dictionary is going to be equal to the filter criteria okay so it's going to look over here in the Json file and say Here's LM Studio model we're going to retrieve the API key and the base URL and just so you can see this I'm going to just have a uh print config list here well the next thing we need to make is an assistant agent so we're going to say assistant is equal to autogen do assistant agent let's give them a name something unique like assistant agent and then we need to give them the llm config and this is the configuration properties basically for the llm that we are going to use so the first thing is we can set the cache seed because this can change by default this is set to 42 but let's say you run that once and then the next time you want it to be 43 and that means whenever you run through this llm again you're going to get different results because if you set the cach D to be the same and you rerun it again it'll just quickly print out what you already ran previously which is useful in certain cases and the next property is temperature this can allow you to have more deviation in the results so let's just set to 0.7 and then finally the one that you actually need because cash sheet and temperature you don't need but the one you need though is config list okay and we're just going to set this to the config list we created above that wasn't too bad now let's move on to the user agent so user is going to be equal to autogen do user proxy agent and this is actually going to be you or me okay this we're going to be the ones talking to the llm let's go over some of the properties here first off the name again something super unique called user and then the second property we're going to have is the human input mode okay there are three modes here always never and terminate so we're going to say always and what this means is that we are always going to have input every time the llm returns a response back to us first time we ask it to write a function it says okay here's the function and then we can say no maybe we don't like it or you can say we like it and then just go ahead and move on to the next part we're going to have the is termination message so we're going to say Lambda X x.g get content so we're going to get the content of the response WR strip and then ends with terminate okay so this is going to be the terminate message when we're just completely done then finally we're going to have the code execution config and what's going to happen is whenever the autogen assistant agent creates something for us like some python function that the user or you or me is we're going to test that and actually run the code so the code execution config has a couple properties that we can do something with that code that we test the first one is work directory so working directory this means we can create a directory uh that will store the python file that the assistant agent created after we tested it so let's just say coding and then the next property is called use Docker this in a recent update this came out to where by default they use Docker as the container to execute code it's okay if you don't know what that means but if you don't know what that means then you just want this to be false okay cuz otherwise if this is true they're going to be looking to see if you have Docker running on your machine and if you don't you're going to get an error and you won't be able to run it okay awesome now there's just a couple things left uh the next one I'm just going to copy and paste and what this is going to do is I want the llm to write three total functions for me in Python and you see here I have to save the code to disk now it's going to quickly say that aen seemed to have a little bit of a problem where it won't exactly save the python file that the llm created and when I looked into their FAQ section if you put this save the code to disk uh that's the first thing you say what it'll do it will it will add the python file name at the top of their code so that's how it knows to save it to your local directory and now we need some way to actually initiate the chat with the user and the llm or the assistant agent we do this by saying user. initiate chat and are some parameters that we need to have here the first one is the recipient so the user is going to initiate the chat with the assistant agent here so we'll just put in recipient recipient equals assistant you don't really need the recipient parameter here U it knows that that's the first one so if you just put assistant that also works as well then we have message is equal to message this is the message that we're sending and they have a couple new recent parameters the first one being silence we're going to set this to false if this was true then it wouldn't show uh all of the output right it would just perform this and then you would get just the response back at the end then summary prompt this basically has the llm based agent so in this case the assistant agent reflect back on the conversation and then create a summary but that's only when the next property will be reflection with llm just get to you in just a second and this is the default prompt which leads me to the last parameter summary method and there's two options either last or reflection with llm whenever it's just last that just gets the last response whereas when it's reflection with llm this will give you a more of a summary of uh the conversation and then finally one more line you can say json. dump user. chat messages we want the assist with the assistant agent and then we're going to say open conversations. Json right so that we can create the file then indent equals 2 okay awesome we're almost there let's just run LM studio and get this started okay if you haven't downloaded LM Studio already I have a link in the description you just literally go to the LM studio. website download the software and just run it it's pretty simple and once you run it you'll be greeted with this home screen here the first thing we need to do is just download a model choose anyone you want you can either search for in the search bar here for a model that of you're choosing like mistal or llama or if you just want to test this out if you scroll down they have some recent ones that they've added like coin 1.5 so then you can click this download button and whenever you download it you open this up here and it's downloading the model for you and then whenever it's done you'll come over here on the left hand side choose local server at the top here select a model to load just you'll choose a model I'm actually going to choose the stable Zephyr because my computer can handle that better so I'm going to choose that load the model and once that done there's this green button says start server just click it and we're ready to go here is the URL that I put in the config list now that we're back we're ready to run this so just open up the terminal and just type in Python 3 main.py and run it okay so it ran and then what we can see here the assistant agent to me uh it it created three functions here uh the pandrum count characters and square root calculation what you don't see here uh it just happened to not be in this example it didn't put the file name here what happened was I just came here and I hit I just hit enter so it would move on the conversation whenever you whenever you do that that means whenever you're ready to move on no human input received using auto reply and this means that it's going to execute the code up here see this is wrapped in three tick marks and then Python and more three tick marks that means it's going to execute that code and I did and it was successful there were no errors and then I hit it responded back with terminate which doesn't like mean anything I just hit enter and it's not really doing anything at this point now right but what it did create it however you see here that is a coding folder that it created but didn't actually put the python file in there it doesn't always work it could also be the model I'm just using a very simple model but we were able to run it locally now let's move over to olama okay so to get AMA working this is simple I first I'm going to have the download URL in my description so just go there download olama and just run it and if you're on a Mac you know you're running it because it'll be up here with the little llama symbol and on Windows it'll be somewhere on your toolbar and once that's running the next thing we need to do is now pull a model to use with AMA and in order to do that open up your terminal you're just going to type in ama pull and then the name of the model so for instance I'm going to type in F because that's what I want to use for this test and I already I already pulled it so it's already done but it'll probably take a few minutes depending on uh the size of the model you will open up your oi config list where I had this model for fi if you have a different model just put that here make sure you have the correct name of the model you just put that there and then you go to main.py and for the filter criteria just type in F and then we're ready to go and now all we need to do is just run the main python file so Python 3 main.py and we're ready to go and it's going to start chatting with the five model okay so it ran and it came back and it gave us the functions count characters is palindrome and square root it even gave us the import for this one now it's call this is calling me a liar uh but this actually isn't going to save because what we need is after this this triple tick mark with python it needs to have a hashtag with the file name but also the point is that it worked locally I'll also run this with open ai's API and see how that performs all I did was come back up to the filter criteria variable change the model so that it'll retrieve the API key as well and then when we hit enter it's going to run for open ai's API okay so this worked as well and you can see here there's a hashtag with a file name String functions. py and I had to be honest with you it didn't do it the first time I had to specifically tell it again during the conversation to add this and then it did it for me and which means that whenever it does that and then I come down here as the user you know instead of I just press enter to skip and it automatically uses the auto reply which means it executes the code block great you can import this module use these functions in other scripts blah blah I hit enter again and finally I hit terminate and in the is terminate message whenever you see terminate it just ends everything and now in our project directory there's a coding folder with the strings python file and this has all the functions that we asked it to create and then finally we generate the conversations. Json file and we have the save code to disk this is the basic message that as the user were giving to the assistant and then the assistant came back said sure here's the code for the functions giving it back to the user and then here's why I said can you save the code to dis by adding the string functions. py to the start of the Python code to the assistant and you know so forth and so forth Just so you can have uh a dump of the chat conversation thank you for sticking around and I hope you learned how to use open source llms with autogen LM studio and AMA are just two ways to do this there are other ways but these are common ways and they're easy to set up click here for more autogen videos if you have any questions leave them in the comments below or suggestions on something that you want to know how to do please let me know every Sunday at noon I send out a newsletter that's free there's a l in the description as well as the link to my GitHub thank you for watching and I'll see you next time

Channel: Tyler AI

Views: 4,846

Keywords: ai, ai agent, build autonomous agent with python, chat gpt, gpt 4, ai automation agency, autogen, auto gen, ai agents, autogen studio, chatgpt, gpt4, open ai, autogen tutorial, autogen update, autogen studio ui 2.0, microsoft autogen, microsoft ai, ollama, ollama tutorial

Id: QBpzGcp-23c

Length: 13min 33sec (813 seconds)

Published: Fri Feb 16 2024