AUTOGEN TUTORIAL - build AI agents with GPT-4o and Microsoft's AutoGen

autogen build your own Army of AI agents to complete any task in today's video I will show you how to install autogen it's got a whole lot easier since it's been released you do not need any coding background to get it up and running just follow the simple steps and you'll have it powered up and running within minutes I'll also show you how to make it run on open ey's latest model GPT 40 and we will test it by asking it to code up a bunch of video games for us spoiler alert it's really good at it as AI takes over make this your Mantra let the robots do the work subscribe to stay on top of AI news so let's talk about how to install autogen Studio since its release it got a lot easier if you forget here's their latest instructions AOG gen- studio.com and basically you have to just follow the instructions listed on this page you need python whatever version of python that they say you need a cond environment we will talk what cond environments in a different video so if you're not familiar with this it's very simple to set up and you have to do that first you will also need if you're using an open AI model you will need open AI key or Azure Microsoft's cloud service for AI models you'll need that to get this working or you can run it locally we'll come back to that but step one is to create a cond environment and they give you the command here to create that so we're going to search for Anaconda and here here's are Anaconda prompt so Anaconda is large kind of umbrella term for a lot of different stuff they have a lot of things under that umbrella that has to do with AI Etc we're using sort of one of their little useful tools that they provide which is this cond which creates an environment so environment you can think of it as like a little pocket Universe on your computer that's a little bit isolated and that allows you to install whatever you need within that environment and it's not going to interrupt anything outside of it it's not a conflict with anything outside of it so just it's a way to kind of isolate and control those things all right so we're going to take this command and we're going to paste it in there cond create so cond is the program we're using Create means create that environment dasn is we're going to name it we're going to name it autogen studio right and here's where we'll set the python version again check to make sure on this page that you're using the right python version now I already have this installed so I'm just going to put autogen Studio One after it just because I already have autogen Studio that name is taken up by the original install and I'll hit enter and it's going to start collecting the package data that it needs and it says we're going to install these packages so these are kind of just the Baseline packages that you're going to need for for anything right it has setup tools python some database packages it's just kind of like the basics that everyone will likely will need so hit yes and it's going to go ahead and uh execute that next you can use this command to to activate it they also provide it here on this page cond the activate autogen studio and as you can see here these little parentheses before it was saying Bas we were kind of in the base reality if you will in the Basse environment and now we're in the autogen studio environment and actually I'm supposed to put autogen Studio 1 I'll change to Auto gen Studio 1 because this is I've already installed that in here so you will say aigin Studio most likely if this is your first install all right so next we're going to set our API key or our Azure key a lot of developers that I talked to are saying using Azure for running open aai models is better they seem to be more stable so API keys will have a separate video explaining that so basically you need to have a account with open AI or with Microsoft Azure and you have to set up an API key that's going to allow you to use all of this so we're going to put this in the chat bot now now so we're going to put this into the terminal into the window now export that's a Mac OS command and I believe Linux as well on on Windows we're not barbarians we don't use that command we say set I'm kidding of course about Windows being better actually I am switching over to the Mac OS now it seems like potentially it's a lot easier to use for a lot of the stuff largely because a lot of developers are in mac and then what you do is you you paste in your API key there so kind of like that I cover up the last of it so I don't have to delete that quite yet you want to be a little bit careful not showing your API keys to people and once you do that you just press enter all right now that you used export or set to set your opening API key next we're going to pip install autogen studio and that's going to install autogen studio and next we're going to say autogen Studio UI and that's it this spins up the auto gen Studio UI as you can see here gives you this URL this is basically not an actual website this is running on your computer but this gives you a little user interface that's easier to use so usually you have to hit control and click on it to open it in the browser and then when you want to stop this application you Hitt contrl C so at the end you can hit contrl C but here we're going to control click on this and as you can see here this opens up in my browser window initially you're not going to have all this stuff it's probably going to be blank but this is where you get started all right so once you get into autogen Studio you have three main tabs here build playground and gallery playground is kind of where you actually run these agents and see what they're doing build is kind of the customization section and there you have four areas here on the side that you need to set up so we have our workflows so this is actually telling the agents what they're doing which agents will be using used in the workflow Etc then we have agents the actual AI agents that will be running and executing the things you tell them to do in the workflows you set up this will make a lot more sense once you see it in action than models so we have GPT 4 we have some other models as well as a potential to set up your own local models and we have skills so skills are basically where a lot of the functionality will come from and this is where a lot of the customization com comes from so for example let's say you wanted to generate images here's the skill for that and this is actually just python code it's pretty straightforward you import whatever packages you need here at the top by the way for people that don't code I know a lot of this might seem complicated but basically think of it this think of this as a recipe to cook something you give it to the agent and they execute it we're going to have a list of the various recipes or skills that are saved somewhere so you're not going to need to to write these up from scratch or even really understand what they mean it would be helpful to be able to kind of at a glance understand what this is kind of talking about but even that is not needed you just need to know that this block of code does something and then if you give it to the agent they will use it to get the result that you're looking for but basically what this is saying so most of these skills will start by defining a function so in this case we're talking to Dolly 3 that's the model that produces IM images and we're telling model this basically draw or illustrate images based on the user's query or request then we kind of describe what that looks like we're describing the Default Resolution Etc and here we're specifying the model that we're going to use right this is why you need your openai API key right because we're going to be using dolly3 and then we're saving them into this saved files folder here and this is checking if it was successful and then most of these will and with something like this this is say returned save files so basically how a function in Python works is we start by defining the function so this is a like a process that runs that does whatever you want it to do and at the end of it it returns something so if your function is add two numbers together right so you define that function of adding two numbers together here you describe how to add those two numbers together and the then you return the answer right so if you say 2 + 2 return four so that's a skill and again really in order to use a skill all you need to be able to do is copy it and then you can use that skill and so here we have three save cat ask key art then you have fetch profile so this is uh fetches the text content from a personal website right so again look it's importing the packages that we need defining this function in this function where we're saying given a URL of a person's personal profile we scrape the content of that page and and kind of take that take that text and so what comes out of this function we return we return the text that was scraped from that website so this is basically goes online finds something and Returns the text that found and this one is you know to produce these little pictures of kitties using ask key art and for all these you can click this green button here on the top right to add a new one so in skills you can add a new skill in models you can add a new model and agents you can add a new agent in workflows you can add a workflow adding open ai's Omni model AKA GPT 40 and of course we have to talk about how to add the brand new Omni model the GPT 4 and what we're going to do to add that model is we're going to click on new model and we just have to write it out here the openi API quick start provides a lot of great resources if you ever forget what it's called what the handle for it is you can just go here and everything will be listed here including pricing context length and the name names of the model so for example for gbt 40 there are two sort of handles for it this one is the one that we need we put that in there once we added the API key in here let's click test model model tested successfully we'll change this to latest Omni model and save it now as you'll note here they're saying changes made to your model do not automatically get updated in your workflow after creating or editing your model you must also readd it to your workflow all right so let's start a new workflow we'll call this the Omni workflow we'll leave everything as is under the user proxy We'll add GPT 40 whatever skills we might need for that particular task and the same thing for the primary assistant instead of the old model we're going to be using the Omni model and we'll click save okay now we have our Omni workflow let's test it out let's code up some video games create a new workflow and here going to write create an outline for a Tetris game and then coded with python click go and again we can also keep an eye on what's happens here in the background in the Anaconda terminal window and there it goes that was surprisingly fast which I guess not surprisingly since the Omni model is supposed to be twice as fast I believe but still interesting to see and so here's our code let's try it out shall we I'll paste it into Visual Studio code for some reason this is not code this is part of the description and that's interesting so when I click copy does that not just get the code let try that again yeah it gets a piece of the text in here but that's okay okay we'll just uh delete some of this and the rest looks pretty good and there's a little bit more code at the bottom let's delete that let's see if we can run it so the first line is import so python game so we need to install that package so we click terminal new terminal and P install py game all right let's see if that helped and make sure you're in the right folder where you have this file and then we're going to run Python and the name of the file which is here main.py and boom Oh file and it it CED up Tetris yes I apologize I know this is not professional but I got to say it CED up pretty well this is called game testing it's important to know that um it got it right I'm doing this for our benefit anyways that worked really well and I got to say that was really fast maybe four five or six seconds is the total amount that it took to code that up and output this whole entire thing let's try again with a Flappy Bird game all right so it gets to work and yeah maybe that was 8 seconds or so so here's the code copied and I just throw that in here same thing looks like for some reason it gets the description of the game as well but that's okay I'll just delete that and looks like we need the same package as py game and random let's see if we have random or not I'm just going to run main.py so it looks like it needs a couple images here bird background pipe and bass I'm actually curious if he can create those for us right within the same workflow I'm going to say create the following images for the game and I'll just paste the code in there as it wrote It's let's see so here's the output So it's talking about how to create those images and actually gave me a code that I can run that will generate those images in the same directory so it's saying save the above code in a file named create images. py so I just created something called create images. py I'll open it in here and we'll take this code that it gave me and put it into there so as you can see here here's how it's going to create the base image pipe image bird image all right so first we have to run that create images program which is going to create create those images within that file so there it is python create images. py so that ran and as you can see here in the folder it created these beautiful images of this for example is a bird as you can obviously tell it's a bird and base this is bass and of course pipe pipe and now let's run Python main.py and see where that takes us well that was fast but let's see okay so space bar to space bar to Flap My Wings and uh I mean that's looking pretty good there's no way to win this one here Chad GPT is teaching us about kambayashi Maru it's a no win scenario the solution to Kobayashi Maru is not to play but you know he's going to play it again anyway there he goes Kobayashi Maru strikes again so I got to say that's very impressive it's nailing those things I I would say on the first try yes we had to add some more functionality here to create these images but still I find that very very impressive all right so let's take this out for a spin we just have the basic skills models and agents that everybody has when they start and we're going to click on workflows we're on the build tab here and we're going to just start the general agent workflow all right so to do that we head over to the playground and you can start a new one all your previous ones are going to appear here so this is just a blank one so so let's start by asking to draw an asky cat for us so that was one of the skills that it had so we'll just just to test out that everything's working we'll we'll test it out we'll click enter so it gets to work now in the background you have your anaconda terminal window running and here you can actually see what's happening behind the scenes so as you can see here you know it's running it's calling the functions the skills that it needs this is the skill name askor cat. py so it's using that to create that output and it says okay we're done so if we go back to here here's the agent messages so this is the conversation that we had and here's the asky cat that printed looks like it had run into one issue but it was able to figure out how to fix it so let's give it something more complicated let's try stock price so if we click on that it says plot a chart of Nvidia and Tesla stock price for the year 2023 save the result to file name Nvidia tesla.in it shows you everything line by line would have executed the code that it ran to do that looks like it installed y Finance that's Yahoo finance and map plot lib so this is to create charts so every once while I was really fascinated to look through exactly what is doing behind the scenes to create those outputs because there's you know obviously a lot that's happening but here's the so here's the messages that the agents went back and forth and they talk to each other and actually you can see the outputs here I was installing the packages that it needed Etc and finally at the end of it here's the files that it has created with the chart of the prices of those two stocks of those two stocks very cool another skill it has is to generate images so let's add ask it we'll ask we'll say generate an image of a hot cup of coffee let's go all right and there it is so it used Dolly to create this beautiful cup of coffee excellent so as you can see this is pretty straightforward you just talk to your agents here they execute what you wanted to do the interface is very very simple but of course the whole point is to build out something custom to you some specific use case that you want to automate and that really happens here and the build tab we'll talk about how to use this in a different video but the point is once you have all those skills and Agents set up whatever you need to happen you just type your message in here and that workflow gets executed right here in this window with this thing running in the background that's pretty much it all right but let's say you wanted to create a new skill for your agents to use how do you do that well we're going to click new skill and so it has some stuff here I'll just go ahead and delete that we'll delete the name and we'll start from scratch now here's an example of a skill I have it's just a few blocks of code now we'll talk about how to get this in a in another video but basically once you have it you have this block of code that does something so in this case there's a function that saves a web page that you ask for as text and it saves that extracted text to a file so this function takes in a URL and returns a file name so I'm just going to copy this and we're just going to paste that in here and we're going to call it save web page as text so there it is save webpage as text we're going to head over to my workflow General workflow and we're going to head over to the playground we'll click new we'll do a general agent workflow and then let's say we take this Wikipedia entry about Juran Schmid Huber he invented everything and we're going to say take this URL and save it as a text file all right so it comes back with this so it use the correct scill as you can see here so save web page as text and here's the text file that it's saved and so here's that file you're going to Schmid H Wikipedia as you can see here so copied everything the whole HTML and saved it as a text file now of course we can change what it saves in that skill menu so if we go back here if you wanted to just save certain part parts of it certain like the text you can adjust the elements that you extract based on your needs next I'll show you a few more use cases for autogen things that will allow you to automate online research the creation of PDFs with various Graphics in them and how to have these AI agents automatically populate the PDF with the research that they're doing online if you're interested in learning more about stuff like this utilizing AI creating autonomous AI agents consider joining the natural 20 community our private network of AI enthusiasts they're all focused on learning AI using Ai and not getting left behind I'll leave the link down below find the latest five papers on archive about LMS use Dolly to create an image that goes along with the theme of the papers create a PDF with the image on page one and the text summaries on page two save the PDF to my desktop and we're going to hit play now this is what's running in the background it says to accomplish this task I will perform the following steps lists out all the steps and starts searching archive for those research papers finds several of them here they are creates a little summary creates an image using one of its skills we'll get to that in a second notice the PDF pops up on my desktop and then creates a PDF places the image places the summaries saves it to the desktop and reports back here's that PDF this is page one and here's page two with the summaries continuing into page three that took the AI about 45 seconds to complete it cost around 14 it would take a human assistant 30 to 60 Minutes to complete that task pull the stock price info for apple and Microsoft for the last 6 months create a PNG image that shows the chart comparison of the prices and save that PNG to my desktop so here it finds the libraries needs Imports matplot lib finds the apple and Microsoft ticker symbols the pricing for each and then saves it to my desktop there's that here's another one pull the total market cap stock info for apple and Microsoft for the last 6 months create a PNG image that shows the chart comparison of the prices save that PNG to my desktop so it installs Yahoo finance finds the market cap data for apple and Microsoft here plots the data the title is market cap comparison Apple versus Microsoft last 6 months labels the Y and xaxis of date at market cap and trillions here's that info researched charted and labeled if you found that helpful consider hitting the like button and subscribing all the cool kids are doing it

Channel: Wes Roth

Views: 42,671

Keywords:

Id: W9yap9AeVYU

Length: 21min 57sec (1317 seconds)

Published: Tue May 21 2024