It is clear that in a few years, businesses
will hire agencies composed entirely of AI agents. For example, an AI lab cognition recently
released their first AI software engineer named Devin that outperforms anything else
that we have ever seen before SWE benchmark. It can train its own AI, learn unfamiliar
technologies, contribute to production repos, and even complete some side hustles on Upwork. But what many people don't realize is that
this comparison on their chart is made between standard large language models, like Claude
or gpt-4 while Devin has access to additional tools like a terminal, code editor, and even
its own browser. So, all it is, is literally just a cleverly
prompted LLM with a bunch of tools, and this lab has already gotten more than 20 millions
of dollars in funding. Personally, I don't believe they're heading
in the right direction and I'll explain exactly why later, but what this really shows is that
we're merely scratching the surface of what's possible here. -So In this video I will share with you my
entire experience developing custom AI agent systems over the last year developing custom
AI agent systems for companies of all sizes raging from small firms with 5 employees to
entire corporation with 30000+ people. In fact, by the end of this video you will
be able to build your own fully functional Social Media Marketing Agency that will generate
ad copy, create ad images with DALL-E 3, and reliably post them on Facebook. Here is the game plan. We'll start with an overview of new AI Agent
developer role and what it entails Next, we'll unravel what AI Agents truly are. After that we'll take a tour of the most popular
AI agent frameworks at your disposal. Then, I'll be pulling back the curtain on
my own framework, giving you an insider's perspective on how it works and how you can
leverage it in your own projects. And finally, we'll get hands-on as we build
a fully functional SMMA, ready to take on new clients and generate profits. This will be a comprehensive guide, highlighting
my entire process from start to finish. So make yourself comfortable and let's dive
in. First, let's me define this new AI Agent Developer
role and why I believe it will be one of the most in-demand skills in 2024. Well, numerous studies and industry experts
predict that we're headed towards full labor automation in the next decade. While I totally agree with this projection,
I don't think it will be a self-driven process. As AI models become increasingly intelligent,
they're certainly going to gain a broader understanding of the world. However, they will never know how a specific
company operates internally, simply because such data is rarely made public. As we saw in 2023, businesses don't just want
to incorporate standard LLMs into their processes. They want to customize them and at least enrich
them with their own personal data. The reason I believe why labs like cognition
will soon fail is because they lack customization. To fully automate a company like google you
need more than just a super intelligent AI developer. We need to make sure that this developer has
access to all the necessary tools, infrastructure and internal knowledge, before it can actually
perform any tasks. This is where AI Agent Developers come in. So, an AI Agent Developer is someone who fine-tunes
AI agents based on internal business processes. As an AI Agent Developer, my primary responsibility
is to equip AI with all the necessary resources and ensure it knows how and when to use them
in production. The primary skills required for an AI agent
developer role can significantly vary from project to project. This topic deserves its own separate video
by itself, so if you're interested, please let me know in the comments. Soon I'll walk you through exactly how to
accomplish all of this. But for now we need to understand what AI
agents truly are. A lot of people say that AI Agents are just
instructions knowledge and actions. And sure that's sort of true, but that's not
exactly what AI agents are, that's how we make AI agents. In fact, AI Agents are much more than that. Let me explain. To answer what AI agents truly are we need
to unpack the difference between standard 1.0 AI automations and more sophisticated
2.0 AI agent-based applications. Picture a straightforward customer support
automation where an LLM must label each incoming email and must respond to it, pulling some
additional context from a vector database. Does this sound like an agent or mere AI automation? You may have noticed that it doesn't quite
feel like an agent, right? But why? It has knowledge from your vector database,
and has instructions on how to respond, and performs an action of attaching a label. And the distinction lies in the fact that
I said that it must generate a label and must answer each email. You see, the fundamental difference between
automations and agents is that agents possess decision-making capabilities. So, In 1.0 AI automations, every single procedure,
like context retrieval, response generation, and labeling is hardcoded into the backend
logic. This means that it literally can not deviate
from this logic, no matter what. If the automation is tasked with responding
to emails, it cannot neglect to respond. And while all this rigidity works well for
certain use cases, it completely fails as soon as some unexpected circumstances arise. Imagine for example if your customer support
mailbox receives an inquiry about a potential partnership with your platform. If this scenario wasn't accounted for, a 1.0
AI automation would handle it like any other support inquiry, potentially causing a missed
opportunity. On the contrary, 2.0 AI Agent-Based applications
have a different approach. While they still equip the agent with the
necessary tools, context, and instructions, they grant the agent the autonomy on how to
utilize these tools by itself. Instead of feeding your context into the prompt
on every request, you empower the agent to retrieve it only when it's needed. This flexibility means that the agent can
adapt accordingly. So in our previous example, the agent would
recognize that it's dealing with an inquiry outside of its expertise and then it could
use other available tools if possible. For example, it could reach out to your human
support agent, or it could send a notification in slack. Overall, what AI agents truly are is a new
way of thinking about how to apply AI in various applications. It's a paradigm shift rather than a simple
technique. In my agency we all began with simple 1.0
AI automations, but as my clients saw the tangible benefits they offered, they yearned
for more-more advanced capabilities and automation of increasingly complex tasks. Over time, we reached a stage where I wouldn't
even call it as automation anymore. It was more akin to outsourcing, as some of
the processes we automated literally required multiple people to manually carry them out. And the performances were never the same. Now, having said all this, where do Agent
Swarms come in? To truly grasp the concept of agent swarms,
it's crucial to understand that all intelligence is environment-dependent. For instance, I might excel when it comes
to programming, but I'm utterly lost when it comes to cooking. I would not last a day as a cook even in McDonald's
for a day. I just basically eat meat and nothing else. This applies to both AI agents and your own
employees. You can't assign 10 different roles to even
the smartest person in the world Likewise, even we reach GPT-100, I would still
not recommend assigning so many different responsibilities to a single agent. Firstly, by removing all unnecessary information
for a given process simply saves you on tokens. And secondly, even if GPT-100 would not get
confused handling 10 different roles, the users of such a system certainly would. So, what agent swarms really allow you to
do is separate responsibilities for different environments, just like in real world organizations. This results in 3 main benefits:
First, it dramatically reduces hallucinations. I found that after you add 7 to 10 tools to
a single GPT4 agent, it starts to get confused. But when you split these tools into multiple
agents you almost completely eradicate this problem. Secondly, you can outsource much more complex
tasks. Because, The longer the sequence of your agents
is, the more tasks they can handle without direct supervision. And lastly, it makes the whole system much
easier to scale. You see, most of my clients don't stop on
a single AI Agent and often try to automate increasingly complex processes over time. So when the need arises instead of adjusting
your existing system, and than debugging it all over again, you can simply add another
agent and leave all the previous agents as they are. In fact this last problem of scaling is so
common among my clients that this week we are releasing the first of it's kind AI Agents
as a Service subscription. Basically, if you are a business owner you
can now pay us a fixed fee per month and we will develop as many AI agents as you need,
but we will work on them one at a time. Our goal is to provide a flexible and scalable
solution that grows with your needs. So if you are interested, you can apply right
now sing the link below at a temporarily discounted price. However, if you're inclined to take on this
journey by yourself, that's perfectly fine too, because next, I'm going to walk you through
my entire process from start to end. But before we get into the nitty-gritty, let's
start with a brief overview of all the multi-agent frameworks at your disposal. The first project is the one you've probably
heard of, called AutoGen by Microsoft. The main feature of AutoGen is multi-agent
chats. It was developed as a research experiment
and was quite groundbreaking at the time. However, the problem with AutoGen is that
it has extremely limited conversational patterns that are super hard to customize. If you look at its code, in AutoGen the next
speaker is determined with an extra call to the model that emulates role play between
the agents. Let me just read it to you "Read the above
conversation. Then select the next speaker from agent names;
only return the role." I mean, not only is this extremely inefficient,
but it also makes the whole system absolutely uncontrollable. A lot of people report that agents constantly
hallucinate because there is no clear separation of concerns when it comes to tool execution. 1 agent might write the code, but because
it needs to be executed by user proxy, or some other agent, it often results in hallucinations,
which is a huge problem in production. The next framework that has recently been
getting a ton of attention is called crew ai. CrewAI was developed as a side project and
it introduces the concept of "process" into agent communication. This provides some semblance of control over
the communication flow. However, just like in AutoGen, the conversation
flows are extremely limited, offering only sequential or hierarchical options. In the sequential process basically all your
agents communicate to each other one by one. And in the hierarchical there is a one manager
agent communicates to everyone else. Obviously, this is not how real organizations
are structured. For example, can you imagine Sundar Pichai
manually instructing a QA tester, who tested this amazing new sign in screen? Additionally, in Crew AI, the manager agent
is hardcoded for you, which for some reason people find cool. However, imagine if you want this agent to
first search the web for additional context before deciding who it should speak next to. Try doing that in Crew AI. The biggest problem with CrewAI, however,
is that it is built on top of Lang-chain, which was released before any function-calling
models. This means that there is no automatic type
checking or error correction when it comes to tool execution. The description for these tools are also extremely
limited. Recently CrewAI introduce way to overcome
this by extending a base tool class, however his process is definitely not straightforward
as it could have been. The goal, backstory, the role and the tasks,
that you need to define when you are creating your crew are simply prompt templates that
also take away control from you as a developer. Without these prompt templates the CrewAI
simply would not be able to function. The only advantage of CrewAI is that you can
use it with open source models. Now personally I would never utilize any of
these frameworks in production for my clients which is why I developed my own framework
called Agency Swarm. In this framework, there isn't a single hard-coded
prompt. It's easily customizable with uniform communication
flows, and it is extremely reliable in production because it provides automatic type checking
and validation for all tools with the instructor library. It is the thinnest possible wrapper around
OpenAI's Assistance API, meaning that you have full control over all your agents. So whether you add a manager agent, define
goals, processes, or not, whether you create a sequential or hierarchical flow or even
combine both with a communication tree that is 50 levels in depth, I don't care, it is
still going to work. Your agents will determine who to communicate
with next, based on their own descriptions and nothing else. But, you are probably wondering, why assistants
api for ai agent development? Well, that's a good question because if you
look at all the previous OpenAI endpoints, you'll find the Assistants API isn't significantly
different. However, it was a game-changer for me as agent
developer. And the reason for this is state management. You see, with the Assistants' API, you can
attach instructions, knowledge, and actions directly to each new agent. This not only allows you to separate various
responsibilities, but also to scale your system seamlessly without having to worrying about
any underlying data management or about your agents confusing each others tools like in
other frameworks. Agent state management is the primary reason
why Agency Swarm is fully based on the OpenAI Assistants API, and to answer your other question,
no we are not currently planning to support any open source models. If costs are a concern, simply use GPT-3 which
is much better than any LLM that you can run locally unless you have a 10000$ PC. If data privacy is a concern you can use my
framework with Azure OpenAI, which doesn't even share data with OpenAI itself. To get started creating your agent swarms
using my framework you need to understand 3 essential entities which are Agents, Tools
and Agencies. Agents, are essentially wrappers around assistants
in Assistants API. They include numerous methods that simplify
the agent creation process. For instance, instead of manually uploading
all your files and adding their IDs when creating an assistant, you can just specify the folder
path. The system will automatically attach all files
from that folder to your assistant. It also stores all your agent settings in
a settings.json file. Therefore, if your agent's configuration changes,
the system will automatically update your existing assistant on OpenAI the next time
you run it, rather than creating a new one. The most commonly used parameters when creating
an agent are name, description, instructions, model, and tools. These are all self-explanatory. There are no preset templates for goals, processes,
backstories, etc., so you simply include them all in the instructions. Additional parameters include files_folder,
schemas_folder, and tools_folder. As I said, all files from your files folder
will be automatically indexed and uploaded to OpenAI. All tools from your tools folder will be attached
to an assistant as well, and all openapi schemas from your schemas folder will automatically
be converted into tools, allowing your agents to easily call third party apis. Additional properties api_params and api_headers
are also available if your API requires authentication. However, I do recommend creating all your
tools from scratch using Instructor, as it gives you more control. I previously posted a detailed tutorial on
Instructor, which includes a brief conversation with its creator, Jason Liu. Check it out if you're interested. In essence, Instructor allows you to integrate
a data validation library, Pydantic, with function calls. This ensures that all agent inputs make sense
before any actions are executed, minimizing production errors. For instance, if you have a number division
tool, you can verify that the divisor is not zero. If it is, the agent will see the error and
automatically correct itself before executing any logic. To begin creating tools in Agency Swarm with
Instructor, create a class that extends a base tool, add your class properties, and
implement the run method. Remember, the agent uses the docstring and
all field descriptions to understand when and how to use your tool. For our number division tool, the docstring
should state that this tool divides two numbers and describe the parameters accordingly. Next, define your execution logic within the
run method. You can access all defined fields through
the self object. To make some fields optional, use the Optional
type from Pydantic. To define available values for your agent,
use a literal or enumerator type. There are also many tricks you can use. For instance, you can add a chain_of_thought
parameter inside the tool to save on token costs and latency, instead of using a chain
of thought prompt globally. To add your validation logic, use field or
model validators from Pydantic. In this division tool example, it makes sense
to add a field validator that checks if the divisor is not zero, returning an error if
it is. Because tools are arguably the most important
part of any AI Agent based system, I created this custom GPT to help you get started faster. For example, if I need a tool that searches
the web with Serp API, it instantly generates a BaseTool with parameters like query as a
string and num_results as an integer, including all relevant descriptions. You can find the link to this tool on our
Discord. The final component of the Agency Swarm framework
is the Agency itself, which is essentially a collection of agents that can communicate
with one another. When initializing your agency, you add an
Agency chart that establishes communication flows between your agents. In contrast to other frameworks, communication
flows in Agency Swarm are uniform, meaning they can be defined in any way you want. If you place any agents in the top-level list
inside the agency chart, these agents can communicate with the user. If you add agents together inside a second-level
list, these agents can communicate with one another. To create a basic sequential flow, add a CEO
agent to the top-level list, then create a second-level list with a CEO, developer, and
virtual assistant. In this flow, the user communicates with the
CEO, who then communicates with the developer and the virtual assistant. If you prefer a hierarchical flow, place the
agents in two separate second-level lists with the CEO. Remember, communication flows are directional. In the previous example, the CEO can initiate
communication with the developer, who can respond, but the developer cannot initiate
communication with the CEO, much like in real organizations. If you still want the developer to assign
tasks to the CEO, simply add another list with the developer first and the CEO second. I always recommend starting with as few agents
as possible and adding more only once they are working as expected. Advanced parameters inside the Agency class
like async mode, threads_callbacks, and settings_callbacks are useful when deploying your swarms on various
backends. Be sure to check our documentation for more
information. When it comes to running your agency, you
have 3 options: the Gradio interface with the demo_gradio command, the terminal version
with the run_demo method, or get_completion, which is similar to previous chat completions
APIs. Now, let's create our own social media marketing
agency together to demonstrate the entire process from start to finish. Alright, for those who are new here, please
install Agency Swarm using the command 'pip install agency swarm.' To get started quickly, I usually run the
'agency swarm genesis' command. This will activate the Genesis agency, which
will create all your agents for you. It doesn't get everything right just yet,
but it does speed up the process significantly. In my prompt, I'm going to specify that I
need a Facebook marketing agency that generates ad copy, creates images with Dalle 3, and
posts them on Facebook. As you can see, we now have our initial agency
structure with three agents: the ad copy agent, image creator agent, and Facebook manager
agent. I really like how the genesis agency has divided
these responsibilities among three different agent roles. However, I'd like to adjust the communication
flows a bit and adopt a sequential flow, so I will instruct the genesis CEO accordingly. Now we have a sequential agency structure
with three communication levels. We can tell it to proceed with the creation
of the agents. This process takes some time, so I'll skip
this part and return when we're ready to fine-tune our agents. After all our agents have been created, you
can see that the CEO tells me that I can run this agency with the python agency.py command. All the folders for my agents and tools are
displayed on the left. The next step is to test and fine-tune all
these tools. We'll start with the image generator agent. The Genesis Agency has created one tool for
this agent called ImageGenerator. It's impressive how close this tool is to
what I planned to implement myself. It uses OpenAI to generate an image with a
simple prompt, taking ad_copy, theme, and specific requirements and inserting them into
a prompt template. Yes, AI has learned to prompt itself. However, there's an issue: it uses an outdated
OpenAI package version with the Da Vinci Codex model, which is designed for code generation. Let's fix this now together. First, I'll load a new OpenAI client with
a convenience method from Agency Swarm Util. I'll also increase the timeout because image
generation can take some time. After that, I'll adjust the API call to use
the new Dalle 3 model, and then set the timeout back to the default. There's one more thing we have to do - we
have to ensure that other agents can use this image when posting the ad. So, I'm going to create a new 'save image'
method that will save this image locally. But here is the kicker - I don't want my agents
to pass this image path to each other because any hallucinations could cause issues. Instead, I'll save this path to a shared state. Essentially, shared state allows you to share
certain variables across all agents in any tool. Instead of having the agent manually pass
the image path to another agent, you can save it in one tool and access it in another. You can also use it to perform validation
logic across various agents, which I'll show you soon. Now we are ready to test this tool. You can do this by adding a simple 'if name
equals main' statement at the end, then initializing the tool with some example parameters. Then you can print the result of the run method. Don't forget to load the environment with
your OpenAI key by adding the 'load_dotenv' method at the top. As you can see, we have an image generated
and saved locally, as expected. This means we can now proceed with adjusting
the next tool, the AdCopyGenerator tool within the ad copy agent. This tool is also very similar to my personal
design. I'll adjust the prompt a bit and save the
results into the shared state. Moving on to the Facebook Manager agent, Genesis
Agency created two tools for us: the Ad Performance Monitor tool and the Ad Scheduler and Poster
tool. While these tools are quite close, creating
an ad on Facebook requires a few more steps. Specifically, we need to first create a campaign
and an ad set before we can post the ad. I will use a tool creator custom gpt to request
two additional tools, 'Ad Campaign Starter', and 'Ad Set Creator'. To run these tools, we first need to install
the Facebook Business SDK, which you can do with this PIP command. Next, we need to create our Facebook app. Go to the Facebook developer website, click
"Create App", select "Other" for the use case, then "Business" for the app type. Add your app name and click "Create App". Then click on "Add product" and add "marketing
API". Go to "App settings", copy your App ID, App
secret, and insert them into the environment file. Now we have to get our access token by visiting
the Facebook API Explorer website and adding the appropriate permissions. After that, copy it and put it into the env
file. Working with the Facebook API can be challenging
as it's known to be one of the more complex APIs out there. I won't delve into the details of how I fine-tuned
these tools. The process is the same: adjust, test, and
repeat until they work as expected. As you can see in the AdCreator tool, we're
actually utilizing the ad copy, ad headline, and image path from the shared state that
we saved earlier. I have also included a model validator that
checks the presence of all these necessary parameters. If one of the parameters is not defined, the
system throws a value error and instructs the agent on which tool needs to be used first. This approach significantly enhances the reliability
of the entire system, as it ensures that the Facebook ad manager agent cannot post any
ads until all the required steps like image generation have been completed. After successfully testing all our tools,
the final step is to refine the instructions. It is a good practice to include how specifically
your agents should communicate with each other. I would also recommend specifying an exact
step-by-step process for them to follow. Lastly, I decided to make a few adjustments
in our communication flows. I'd like to establish a direct line of communication
with our Facebook Manager agent, so I'll include it in the top-level list. Also, I'll allow our CEO agent to communicate
directly with both the Facebook Manager and the Image Generator agents. Now that we've made these adjustments, we're
ready to run our agency. It is as simple as running the python.agency.py
command and opening the provided Gradio interface link in your browser. Let's see how it works. I'll kindly ask for an advertisement to be
created for my AI development agency, Arsen AI. The CEO then instructs the ad copy agent,
which promptly provides a clear headline and ad copy for my agency, stating, "Revolutionize
your business with AI." Next, the CEO commands the image generator
agent to create an image for the ad copy, resulting in a futuristic visual for our campaign. Finally, the CEO directs the FacebookManager
Agent to commence the campaign using the campaign starter tool. It then creates an ad set and executes the
ad creation function, posting this ad on Facebook. You can now see this newly generated Facebook
ad, complete with ad copy, headline, and image, live on my Facebook account. Impressive, right? But what if you want to analyze your campaign's
performance? You can do this by directly messaging the
Facebook Manager agent, as it was included in the top-level list. It uses the AdPerformanceMonitor tool and
informs me that there is currently no data as it takes some time for an ad to reach its
audience. In conclusion, I'd like to briefly share my
roadmap for this framework. First, I plan to establish multi-agency communication. This feature will allow the integration of
multiple agencies for super complex use cases. Next, we'll focus on enhancing the Genesys
agency. With multi-agency communication, the Genesys
agency will be able to test other agencies during their creation. The goal is to reach a point where there's
almost no need to modify tools or instructions for simple agencies like the one we've just
created. And lastly, we will continue to regularly
update this framework to include the latest releases from the OpenAI Assistants API. With upcoming features like memory and web
browsing, the possibilities are exciting to say the least. So, stay connected with us on Discord. We're always on the lookout for new talent. If you're interested and you have previous
experience with this framework, you can apply through our job postings channel. Thank you for watching and don't forget to
like and subscribe.
