AutoGen Tutorial 2.0 🔥 How to Build POWERFUL AI Agents with AutoGen Studio UI

autogen build your own Army of AI agents to complete any task the new autogen studio is here and you have to see it take a look find the latest five papers on archive about LMS use Dolly to create an image that goes along with the theme of the papers create a PDF with the image on page one and the text summaries on page two save the PDF to my desktop and we're going to hit play now this is what's running in the background it says to accomplish this task I will perform the following steps lists out all the steps and starts searching archive for those research papers finds several of them here they are creates a little summary creates an image using one of its skills we'll get to that in a second notice the PDF pops up on my desktop and then creates a PDF places the image places the summaries saves it to the desktop and reports back here's that PDF this is page one and here's page two with the summaries continuing into page three that took the AI about 45 seconds to complete it cost around 14 it would take a human assistant 30 to 60 Minutes to complete that task pull the stock price info for apple and Microsoft for the last 6 months create a PNG image that shows the chart comparison of the prices and save that PNG to my desktop so here it finds the libraries it needs Imports matplot lib finds the apple and Microsoft ticker symbols the pricing for each and then saves it to my desktop there's that here's another one pull the total market cap stock info for apple and Microsoft for the last 6 months create a PNG image that shows the chart comparison of the prices save that PNG to my desktop so it installs Yahoo finance finds the market cap data for apple and Microsoft here plots the data the title is market cap comparison Apple versus Microsoft last 6 months labels the Y and xaxis of date at market cap and trillions here's that info researched charted and labeled so you might have heard about autogen this AI agent framework that allows you to combine multiple agents to work on your behalf now when when it started it was a little bit difficult you had to use the terminal for it you still do to install it and get it to work but this new release makes it a lot easier in many ways it adds a lot of very different capabilities like skills you have to see it to really get a feel for how cool it is it can code it can do research it can write stuff and like they say here on the right this is just the beginning you're able to program by talking back in 45 I they execute your ideas they code them up as you're chatting with them it's funny this first comment says this video really undersells the power of autogen in this video I'll walk you through how to install this and start using this you don't need any background in programming or anything like that although it is going to help as you'll see the installation process is somewhat complicated if you've never done this before now about a third of the people watching this are god tier developers this first section might not be for you you can skip to the next chapter in the video but for the people that haven't done this before I think this is a really good time to start it can seem like rocket science at first but once you learn a few basic commands and a few basic tools it's actually very very simple it just looks kind of complicated the good news is that Chad GPT specifically GPT 4 can help you with a lot of this I'm going to ask it to give me a quick summary of what Anaconda is how to install it and how to use it to create environments so the first thing you have to know is that python is a programming language that is very often used when dealing with AI it's not the only one but it is one of the more popular ones you don't have to learn it but it is nice to have a very basic understanding of what it looks like what it can do Etc next we have anaconda anaconda is an open source distribution of python and it's used for a lot of computing data science and AI machine learning applications if you need to install your computer you go to the official anacon website gp4 will give it to you or just Google it once that's installed you will have the Anaconda prompt that looks a little bit like this it's basically what's called a terminal window just search for Anaconda and it will pop up right here here's what that looks like when you first launch it why this is important is because Anaconda allows you to create environments environments are kind of like their own separate Little Worlds that live on your computer that you can use to install a bunch of stuff mess around with it and later just delete that it's its own little pocket where you can run certain codes install certain software and have have it be separate from everything else so if you're starting from scratch you install python you install Anaconda then you open up the Anaconda prompt the terminal window next we create an anaconda environment where we'll install autogen if you ever forget how to do that just ask Chad GPT how to create an anaconda environment so you open up your anaconda prompt that's this thing right here and the basic command is cond like Anaconda create like we're creating an environment and then-- name is the flag that we used to say that we're going to name this environment and then we do space and we can say whatever we want whatever we want to name it if you're super cool you just do Dash and for name instead of writing out D- name and then we name the environment so in this case we'll be autog Gen 2 and now I hit enter it starts creating that environment says do you want to proceed yes and there it is so it created the environment says to activate this environment use and this it gives you this command cond activate autogen 2 which you can just copy and paste in there activating that environment you can think of as like going into that environment once you're in there everything you do in that environment stays in there so later if you mess something up or you want to do something else so you can just delete that thing and it will have no effect on the rest of your computer it's isolated all right so we hit enter as you can see now we changed from base to autogen 2 so we're now in that environment all right I'll put a link in the description that goes to my website natural 20.com that will have hopefully the updated installation instructions because this is where it gets a little bit difficult so if you're a new user this might seem complicated but there's a reason for why it happens right now due to all the different versions of python and AO agend that are out we have to make sure that we install the right ones so that they can work together correctly so what I showed you before will work most of the time if you're doing this right now as the video is being released at the beginning of January there's a few Hoops to jump through again I'll put a link down in the description with a link to my website that's going to show hopefully the updated directions to do that but right now I'll call the environment autogen 3 and as you can see here in ch pt4 so when it's saying con to create my environment and then it says python equals 3.8 this will create an environment named whatever you named it with python 3.8 this is the problem is we if we put this here right now in order to run autogen Studio it needs at least a 3.9 python 3.9 if you search what is the latest python version it'll give you the latest releases for Windows so it has to be at least python 3.9 and this is where it gets annoying it can't go as high as 3.12 it has to be less than 312 so I'll just do 3.9 and we'll hit enter it's going to ask us do you want to create that environment with the following things installed we say yes and then it says to activate this environment use this thing here we can just copy and paste it and activate here means kind of like going into it once you go into that environment everything you install and do in that environment is sort of its own isolated thing so later you can just delete it and it's not going to affect anything else on the computer so we hit enter and now we are in autogen 3 our new environment so again think of this as like we just set up a brand new workshop with all the things that we all the tools that we need in there and now we're inside that workshop and we're going to start working that Workshop is called autogen 3 and it's referred to as the Anaconda environment all right so the next thing to understand is PIP install pip install is just a way of installing new software on your computer in that environment so similar to how if you downloaded the setup file for a new software you double click on it and it starts installing it's the same thing here but we type pip install to install it so for example here in autogen 3 we say pip install py autogen and we hit enter and it downloads installs everything that we need and now it's complete and next we need to install origin studio so we type pip install aogen studio and we hit enter and it starts installing everything that we need for that we've installed autogen Studio we're almost ready to go in order to make everything work if we're using for example open AI models like GPT 4 we need an openai API key you're going to find that at platform. open.com ai- keys or just Google openai API keys this is kind of what that looks like you have to log into your account and you have to have billing set up so they can charge you for usage of whatever model you're using and we're going to click create new key Call It Whatever going to call it ag3 that's going to create the new secret key copy it here don't worry I'll delete this before I post the video and you should be careful of this key don't give it to people because then they can use your account and basically get you in trouble or run up your credit card bill so keep this safe and so basically we want to save our API key as a variable in here so that autogen can use it so the command for that on Windows is set and I believe on Mac it's export I'm going to say set open aore API key equal and I'll paste the key in there and I hit enter now that key is saved and everything will run next we'll type autogen Studio space UI then d-port space 8081 so what this does is it opens a local port on your computer what that means is like kind of like your computer opens a door that you can walk through and interact with the stuff inside it's local meaning that you can do it from your computer but no one else can do it from theirs then I'll push enter and it says run it on this URL now this URL is local to you so for example if I go to google.com I will go to Google servers if I go to this URL this basically is running on my own computer it's running locally so I paste that into the URL and autogen Studio pops up here it is you have now created an anacondra environment installed everything you need including python including autogen autogen studio and now this thing is pretty much ready to run there are two things that you might want to do to improve the experience so here's Microsoft kind of their documentation for best practices for running autogen locally so they're saying that we highly recommend using Docker for code execution so you also need to install this python package Docker so pip install Docker I'll copy that and you can use contrl C to shut down if if it's running in the background hit control C to shut it down and we're going to pip install stalker and that's going to install all the libraries that Microsoft is recommending that you have to run autogen successfully or effectively next when autogen runs sometimes you get this error message it seems that Powershell execution policy is still preventing the installation of the required packages so basically your computer security system prevents it from running too many scripts cuz you know it that could be a security issue now autogen is smart enough to basically run a python script to install the packages so you can disable those safeguards if you know what you're doing or just let it run as it is it will still figure it out all right so now once that done again you can run the autogen UI d-port 8081 to open that Port again it gives you the link so just copy that link paste it in there and here it is section created successfully now I know that might have been complicated the thing is once you've done this a few times it becomes like second nature there's only a few commands you need to know a few Concepts that you need to understand you don't need to take a course to pick this up you just need to know how to install python use the Aon environments and how to install the ne software within those environments and there's a few commands that do pretty much all of it all right now we are inside autogen Studio the user interface that kind of acts like a website like a web page where you can do all sorts of things so let's do a quick tour so there's three tabs up here build playground and gallery let's start with build we can create skills agents and workflows for building multi-agent capabilities it comes pre-programmed with three workflows for travel agent group chat workflow General agent workflow this is what I've been using and it does it's pretty effective for a lot of tasks and then the visualization agent workflow you can create new workflows like this name it Define it how you get summaries whether it's usually the last thing they say is kind of a summary of what what it did so that's pretty good you can also said none or an llm summary I haven't tried this one yet this might be good for troubleshooting stuff but by default It's At Last and then you have a sender and receiver so those are your Bots your agents we'll go into that in more detail later then you have your your agents right so you have your user proxy so this is the person this is your agent that you're talking to you're like hey I want this done and that user proxy then goes and starts handing out tasks so in this case we just have the primary assistant that writes plans and codes to solve those tasks but this can be much more complicated so for example you might recall Chad Dev we covered it in a different video where you have different departments and each of these little guys in G here are actual their own version of Chad gbt or whatever llm that you're using so you have the designing you have the coding the testing and the documenting and then you also have the chief technical officer I think that's this guy and so basically he goes here and starts by designing the process then the coders coded up then testing you know irons out all the bugs and then documenting writes out the manual for it so I forget how many of these guys there are but you can basically replicate something like this with autogen now if you've tried doing that in the past and failed because it was too complicated it got a lot easier we'll we'll take a look at that in just a second but basically here you can create new Agents Set various consecutive auto replies how often they ask for human input like every single time they have to they're like oh do you have anything else to say and then you can add your own stuff or never they keep going until it's completed and this is the system message basically the text to control agent Behavior this is where you tell it who it is you give it to the soul it's life's purpose something like you pass the butter for example now this AI guide that you've just created all it does is just passes the butter and then here's the model so you have GPT 4 so this is the 1106 preview that's the model do rolled out on November 6th and here you can add pretty much whatever you want as long as you have the API keys for it I haven't messed around with this yet but that's next on my list and add skills that are available to the agents what are skills here are the skills for example one of the skills is find papers on archive fetch profile fetches the text content from a personal website given a URL of a person's personal website this function scrapes the content of the page and Returns the text found within the body and this basically teaches the AI to do that thing or generating images so here we're using Dolly 3 and by the way it's very good at this it's very effective at this because this teaches it how to do all of it so when it needs to it grabs this skill and it executes it here's find papers on an archive and everything that is needed is in there to do that archive it's pronounced archive for people that are un familiar with it is just a place where there's a lot of scientific papers that are published you can just grab them as a PDF you don't need to log in or anything it's kind of an incredible thing that that we have actually and so here you can build your various skills for the agent so you can build a skill that for example goes to 11 labs and use the API to to create an AI voice for some text that you need so text to voice or pretty much anything else that you can do as code so here for example is an asky art that creates a little kitten and absolutely crucial skill for these AI agents to have next we have the playground so this is the sort of the main thing where you're um where you're going to be interacting with it back and forth and the gallery which right now I don't have anything in there and my previous ones haven't been saved oh because I reinstalled it so in my previous ones this probably shows the list of all the things that I've already done so do you want to take this baby out for a spin let's get started so we're going to hit new and you can view all the workflows there and let's do General agent workflow create and so they give you four kind of a example so stock price it's that one where it does Tesla versus Facebook let's try paint paint a picture of a glass of Ethiopian coffee freshly brewed in a tall glass cup on a table right in front of a lush green forest scenery so while this is processing if you go back to your anaconda prompt it will actually kind of show you the stuff that's happening behind the scenes now a lot of this will be on this web page here at the end of it but if you kind of want to follow it along live you know just kind of scroll down and see what is happening so as you can see here it I assume it used the skill of generating the images yes so as you can see here so it's saying from skills so from its own skills that it's created it Imports the skill called generate and save images right so that gives it the instructions that it needs to do that so it just follows those instructions and and then it's saved with the file name that's the file name and then terminate means it ended the process so it's like okay done so as you can see here you know everything that we saw happening is right here so this is the skill that it imported to generate images and then here is the file now I've been having a hard time opening these files inside this uh sort of web page that it kind of creates for us so I'm going to take a copy the the file name I say save this file to my desktop now normally that gets saved in the working directory which is going to be the directory that's created when you create that environment it's like 50 layers deep so you can just search for it and find that file or you can have just the directory open somewhere or you can actually just tell it hey at the end of every promt just say hey save that to this specific directory or to your desktop so you can see it Etc and so as you can see it here it's it thinks through what it has to do to save this file to the desktop I'll provide you with a python script that you can run and then it looks like it can't find it so it says I need to make a script to find that so does that it finds it so it had some issues finding it so it just kind of regenerated it and then here is that image now now there's a million things that you can do with this so I've been messing around with it all day and probably for the last 4 hours or so just doing a lot of random stuff and so it looks like it charged me $4.50 in GPT 4 Turbo credits and the image models the do the image making side of it that cost 24 cents and this is just the default General agent workflow with two agents and just these three predin skills the amount of stuff that we're going to be able to do with this just this by itself the aogen studio once we're able to create skills and various different agents for it will be kind of Limitless I am kind of Blown Away by how big this could be now you might be saying yes but isn't it difficult to create these agents and these skills don't you need to be a super smart developer and coder to be able to do this here's the thing someone created an autogen Builder so I got to give this person credit thinking abacus not familiar with this person/ compan but they've created a GPT like a custom GPT that is public that anyone can access that says I will transform your ideas into as many agents as required and so you know as a start you can you can ask whatever you want but what's interesting here is one of them is can I upload a screenshot of a flow which I was curious to see is it able to generate something like this so I saved this image and let's see if it's able to do it so I uploaded that image I'm going to say I'm going to say create a g development agency there will be four departments designing coding testing and documenting each will have one agent that runs that department the user proxy agent will be like the CTO the chief technical officer so the user proxy remember that's kind of like your guy your agent so you talk to them like your sergeant or whatever you're like hey you do this and it goes and it deals with everybody else on your behalf so the user proxy agent will be like the CTO and will deal with the others to make the final product so you know I think normally I would like expand on what designing coding testing documenting like specifically what they do let me just hit enter and see where it goes and then uh we'll see if we need more explanations or not all right so it's uh creating it and by the way I'm just using just one agent per Department obviously we can do more like we do here which in a lot of different AI research has been shown that multiple agents working together especially if they're like a little bit different if they're all kind of coordinating and working together they tend to be better at at their results than just one kind of working in isolation that's why autogen is such a big thing because it's it allows us to really tap into that power and so it writes the python script that you need to create those agents and it writes everything out so the user proxy agent the name is CTO the system message so it's that's it's you know when you tell it what it is you breathe life into it here's you know you pass the butter that's your mission so this is a human admin overseeing game development and again I mean I can just this is text so I can just go in there and change it before I use it so I can change whatever I want and then he creates the design agent and the system message is responsible for game design again uh I think chat dev has their system messages listed somewhere so it might at this point might even make sense to just take those and even just paste it in there but but my point is you can change this text to whatever you want here's the group chat with all the agents and the group chat manager now the thing is I think a lot of this you don't need it anymore because of the new UI because you have this you don't need it as much so when you're creating a new agent really you're just writing the system message in here and naming it Etc now wonder if I say help me create a skill so skills are the things that the agents can use to complete the task help me create a skill that let's say checks the weather all right so we'll hit enter let's see if it can give me code to create a skill that checks the weather so it gives us kind of like a basic generic thing but you have to you have to kind of set your own you got to tell what API you want to use and stuff like that what service you want to use I'm very tempted to create my own autogen builder GPT I got to say it's that Steve Jobs quote right good artist copy great artist steel I'd love to build one specifically that kind of walks people through how to build skills and agents here with all that said this was a basic overview for how to install and get up and running with this thing the installation is probably going to change over time in this particular moment since it's so new there's stuff that's a little clunky over time it's going to get easier but I you encourage you if you have an interest in this to try to figure out how to install if you haven't done it before I know it might seem a little bit daunting but it's this is a very like reusable skill you're going to be using the skill a lot if you're dealing with AI and all the sorts of stuff as you'll see in next video you don't have to learn how to code from scratch this can do a lot of that for you now obviously right now this is not going to be comparable to a team of Highly experienced Engineers there's still issues and there's still hallucinations there's still problems but it's advancing rapidly it's getting getting really good and I'm excited to see where it goes next I'm going to be doing more videos on this covering more specific topics like skills agents workflows if you're interested in that make sure you hit thumbs up subscribe comment share if you can the reason being is that helps me get this message out there it helps pump up the video and it helps me do more of this type of content and it really does help it really does help push this thing out there so if you made it this far thank you so much for watching I hope that was helpful stay tuned for more autogen content name is West rth and I'll see you next time

Channel: Wes Roth

Views: 96,674

Keywords:

Id: Cl19yWHhc2g

Length: 23min 30sec (1410 seconds)

Published: Mon Jan 15 2024