LangGraph: Multi-Agent Workflows

hey everyone this is Harrison from Lang chain here I wanted to do a video on Lang graph which is a new library we released um pretty recently and how you can use l graph to create some multi-agent workflows so for those of you who aren't caught up on L graph we have a whole series of videos going through it and it's basically a way to dynamically create agent-like workflows as graphs so what do I mean by that by an agent-like workflow I I generally mean running a language model in a loop in a variety of ways um and there's different structure and how you might want to run that language model in a loop and so L graph is useful for defining that structure and specifically it allows for the definition of Cycles because this is a loop after all um another way to think about this um is thinking about it as a way to define kind of like State machines um and and so a state machine machine and a labeled directed graph are pretty similar and that you have different nodes and if if you think of this as a state machine then a node would be a state and if you think of it um as as uh uh kind of like in this multi-agent workflow then that node state is an agent um and then you have these edges between them and so in a graph that's an edge in a state machine that is uh you know transition probabilities and when you think of these multi-agent workflows then those are basically how do these agents communicate to each other so we'll think about building multi-agent workflows using L graph the nodes are the agents the edges are how they communicate um if you haven't already checked out L graph again highly recommend that you check out the this the video that we have at a high level the syntax looks like this it's very similar to network X you define a graph that tracks some State over time you then add nodes into that graph you then Define edges between those nodes and then you can use it um like you would uh any other um L chain chain so today I want to talk about three different variants of multi-agent workflows that we're going to use laying graph to create so the first one is what we're calling multi-agent collaboration and so this is largely defined as you have multiple agents working on the same state of messages so when you think about multiple agents collaborating they can either kind of like share State between them or they can be kind of like independent and work on their own and then maybe pass like final responses to the other and so in a multi-agent collaboration they share the state and so specifically this this example that we've set up it has two different agents and these are basically prompts plus llms so there's a prompts plus an llm for a researcher and there's a prompt plus an llm for a chart generator they each have a few different tools that they can call but basically the way that it works is we first call the researcher uh node or the researcher agent we get a message and then from there there's three different ways that it can go one if the researcher says to call a function then we call that tool and then we go back to the researcher two if the researcher says a message that that says final answer so we in the prompt we say like if you're done say final answer so if the researcher says final answer then it returns to the user and then three if it just sends a normal message there's no tool calls and there's no final answer then we let the chart generator take a look at the state that's accumulated um and kind of uh respond after that so we're going to we're going to walk through what this looks like um we're first going to set up everything we've need I've already done this um importantly we're going to be using Lang Smith here to kind of like track the the multi-agents and see what's going on for for a really good debugging experience um Lang Smith right now is in private beta if you don't have access shoot me a DM on LinkedIn or Twitter and we can get you access so the first thing we're going to do do is we're going to Define this helper function that creates an agent so basically we're going to create uh these two kind of like agents up here um and these agents are parameterized by an llm the tools they have and a system message and so then we're taking kind of like uh we're basically creating this mini chain this very simple mini chain that is a prompt which has uh the system message in it um and then it's a it's a call to the llm and it has access to some tool so let's define this helper function we're then going to define the tools that we want the agents to be able to call so we'll have one Search tool we'll use tavil search for that and then we'll have one python tool that can run python code um now we can start to create our graph so first we're going to define the state of what we want to track so we want to track the messages and so again like each agent will add a message to this messages property and then we also want to track who the most recent sender is and the reason reason we need this info is because after we call this tool we're going to check who the sender was and we're going to return to them so this is just some some State that's useful to track over time um so let's define this state um we're now going to Define some agent nodes so here we've created a little helper function to create an agent node and so specifically this agent node is going to uh take in state agent name it's going to call the agent on the state and it's then going to look at the uh uh result we need to do a little bit of kind of like converting here so the agent is going to respond with an AI message but when we add that to the messages States that's accumulated over time we want to represent that as a human message actually um because we basically want the next AI that sees this to kind of like work with that and so we're going to if it's a function message then we're going to represent it as a function message because it will just say a function message otherwise we're going to cast it to a human message and we're going to give it a specific name we're going to give it the name of the agent so that it knows basically is this message that it sees again because we're keeping track of This Global State and so in the global State when it sees a human message is it from this researcher or is it from this charge generator um and so we're going to do that and we're going to create two nodes we're going to create this research agent node um by basically partialing out the agent node function um and then we're going to create this chart generator node we're now going to define the tool node this is the node that just runs the tools it basically looks at the most recent message it loads the tool arguments it kind of it will call it it will call the tool executor the tool executor is just a simple wrapper around tools that makes it easy to call them and then it will convert the response into a function message and append that um to the messages property and now we need the edge logic so this is the this is the uh big router logic here um which basically defines uh some of the logic of where to go so we look at the messages we look at the last message if there's a function call in the last message then we go to the call tool or then we return call tool and we'll see what that leads to later on if there's final answer in the last message then we return end um otherwise we return continue and so let's see how we use this in the graph so first we create this graph let me run this m first first we create this graph with the agent State then we add the three nodes that we have the research node the chart node and the tool node um we then add this conditional Edge so the researcher after we call the researcher um it uses the router logic if if the router returns continue then we go to chart generator so if the researcher returns basically something that doesn't have a tool and something that isn't final answer then we return continue in which case we go to the chart generator if the router returns call tool then we call the tool and if it returns end then we end similar with the chart generator um and then we add this conditional Edge after call tool so after we call the tool we basically look at who the sender is and if the sender is the researcher then we go to the researcher if the sender is the charge generator then we go to the chart generator and we set the entry point to be the researcher node now we can invoke it um we'll use the stream method so that we can see things being printed out um we'll pass in this human message asking it to to fetch the UK's GDP over the past 5 years then draw a line graph of it um and then after we code it finish it um and we'll run this it'll take a little bit so I'll probably pause the video here or I'll kick off the I'll kick it off pause it come back when it's finished and then we can see what it looks like in Lang Smith as well all right so it's finished so it was streaming out so we can take a look at the whole stream of things in here if I can figure out how to expand it so we can see the researcher um you know the first thing it does is it queries UK GDP data for the past 5 years and then calls a tool um and then it kind of keeps on going um at some point there's this chart so you know it PL plots it over the past uh three years it looks like so it didn't quite get all the all the data it needs so you can see uh yeah you can see it says that the data for years 2018 2019 is missing um and so it plots that and then this is just the final response so that's why it shows up so big if we want to see this in a little bit nicer view because this is a lot to digest we can go to Lang Smith um and so Lang Smith uh at the top of the notebook I sent I set it so that it would log to this specific project the multi-agent collaboration I can see that I've got one run can see some basic stats on it like it took over a minute it's got some tokens on it um if I click into it I can now see exactly what was going on under the hood so first it called the researcher and we can click into here and we can see the exact call that it made to open AI um and so there was this system prompt and then uh this was the user's first message and then output this it then called toil um which is the Search tool we can see the results there it then called the researcher again we can see that you know there's now more in it because we start to build up this message Bank we can see it calls toil again okay so now the researcher and it doesn't call toil okay interesting let's see what's going on here so at the bottom it basically says okay so now it's uh it's responding what the values are um and there's no tool call and there's no uh uh thing saying that it's finished so that means that it goes to the next one which is the chart generator so it calls the chart generator under the hood that calls open AI um we can see that down here we get a function call and this is running python code so it prints this out calls the python reppel um uh we can see that it runs this um the chart generator then goes um and the chart generator says uh you know here's the line graph see the chart above the researcher then goes and it says final answer so now that there's final answer it's finished and that's basically what happened um so that's the multi-agent collaboration bit the the kind of like specific thing about this that's interesting is it has This Global state of messages that each llm sees and a penss to so it keeps on adding to this over time and so on one hand this is good because it allows each agent to see kind of like exactly the other steps that the other one is doing and that's why we called it collaboration it's very collaborative the other version of this is that you have agents that have their own independent scratch pads um and so we'll take a look at the next multi-agent example that we have which is this agent supervisor so here we have the supervisor which basically routes between different independent agents so these agents will be Lang chain agents um so they'll basically be an llm that's run with the agent executor in a loop the llm decides what to do um the agent executive then calls the tool goes back to the llm calls another tool goes back to the llm and then then finishes and uh then there's basically the supervisor the supervisor has its own list of messages and essentially what it will do is it will call agent One agent one will go do its work but it will return only the final answer only that final answer is then added the supervisor only sees his final answer and then based on that it can call another agent or it can call another agent another way to think about this is that this is essentially um you know the supervisor an agent and it's got a bunch of tools and these tools themselves are agents so it's a slightly different framing than before it's not as collaborative they're not working on kind of like the same sections so let's get this started um again we're going to be uh I'm going to let me restart my kernel we're going to be using Lang Smith um so again uh hit me up if you don't have access to that um first thing we're going to do is we're going to create some tools we'll use the same two tools as before the tavil tool and the python tool we'll next Define some helpers again to create these agents um as the individual nodes so we have this create agent function similar to last time the difference is now that it's actually doing a little bit more work under the hood so it's not just a prompt plus a language model it's actually creating an open AI tool agent which is an agent class that we have in Lang chain from there it's creating an agent executor which is also in Lang chain and we're returning that that's the final agent so let's run that then what we're doing is we're also creating similar to last time this agent node thing which does this human message converion just like we did last time is taking the agent result um which is an AI message and converting it into human message with a name so we know kind of like where it's coming from now we create the agent supervisor so this is going to be the the you know system that's responsible for looking at the different agents that it has and deciding which one to to pass it on to so here you can see that we're defining a bunch of stuff we're defining system prompt your supervisor task with managing conversation a between the following workers and it's basically using this function definition to select the next Ro to send things to um and uh it's or it's selecting finish finish is another option we're creating this prompt template we're creating this llm and we're creating this supervisor chain um which basically has a prompt um it's got an llm with some functions that it can call specifically we're forcing it to call this route function where it selects the next best role and then we're pressing the the response in Json um and we're using that determine where to go to next now that we've got that we can create the graph so we've got this agent State um which is messages and then a next field um we've got this research agent so we create the agent we then create the node for it so we partial out agent node we then do the same for the code agent um and then we start creating the graph so we create the graph with the agent State and we add the three nodes we now connect the edges in the graph okay so for all the members um and again if we look at where members is defined members is defined up here so it's the researcher encoder it's two it's the two basically sub nodes or sub agents um so for those after we call those agents we go back to the supervisor um then we also have this conditional Edge so from the supervisor um after the supervisor um is run we look at what's in next and we look in the conditional map and the conditional map is basically a mapping between the the members so uh and and finish so so you know if if it says researcher then we go to research node if it says coder then we go to coder node if it says finish then we finish we set the entry point to the supervisor and then we run the graph now we can do the same thing we can invoke the team um we can uh see what happens um and see how it performs all right so it runs it um you can see that it calls the coder so here we're asking it to print out hello or to say hello world and print it to the terminal so it calls the coder and then it finishes here we can write a brief research report on pasas and it will do that while that's running um we can maybe look in Lang Smith um to see the previous ones so here we have the coding one um we can see that we first called the supervisor agent it's got this call to open AI um again it's got the system prompt and then the human um and then it decides what to do next and it's calling it's using this function call we can see it has this function definition of Route here based on that it goes to the coder so now there's the coder agent the coder agent does several things under the hood it calls open AI first um it does a function call it then calls the python reppel um and then it calls open AI again and so when we go back to the supervisor if we look at what the supervisor see says it just sees the response from the coder so there's a human message with the name coder and it gets this response it doesn't see any of these calls though or any of the calls to the python reppel and then we get a response and it says to finish so this is an example of where you know there's many calls but they're kind of hidden within one agent State we can also see that it finished writing the research report on pasas so if we go back here we can see that it's finished running and if we take a look it now calls the researcher agent under the hood and we can see here if you know this is interesting because this took a pretty long time 45 seconds we can quickly drill in and see that the the longest call was this call to open AI um inside the researcher agent where basically it uh uh has it has all these uh uh information from the internet and it writes this long research report so that's pretty much it for the agent supervisor now I want to talk about the third and final type of agent that we have an example of which is the hierarchical agent team so this is very similar to the previous concept it's just changing what's going on in the uh uh it's just changing what's going on in the sub things so now each agent node is itself a different kind of like supervisor agent setup and so what this means is that it's going to be a lot more kind of like a l graph um and a a a little bit less of L chain we can get started by setting up the environment similar to last time we now create the tools so there's two different sets of tools that we need to create one's for the research team it's the Searcher and web scraper the others for this document authoring team and it's these three tools here so we can do that we can load to villy we can create our scrape web pages tool and then we can start creating these uh other tools as well um and these are for the writer team so we create one that creates an outline read documents write documents edit documents um and then a python reppel tool as well we we're going to create a few helper utilities that make it easier to basically construct this pretty complicated graph so we have one that creates an agent we then have an agent node um and we then have this team supervisor idea and so these are kind of like taking a lot of the concepts that we had in the last notebook and extending them now we can Define these agent teams they're hierarchical teams so we have this research team This research team has a few different attributes on it the main things here are this messages um we create uh we initi in llm we create the search agent and then the search node the research agent and then the research node and now we have the supervisor agent as well we now put this all together um I'm not going to go over all the edges because it's very similar to some of the notebooks that we did before but the end result is that we have this uh uh graph that we create in order for this graph to be used in as a subgraph in another component we're going to add this enter chain function which basically just is just a converter to make it easier to use we can use this directly um so we can try out and again this is just one of the sub teams but we can try asking it a question um and we can see that it does a search it has a search to so it'll get a response from there and it will give us an answer we can see that we get back a response here and if we look at that in lsmith we can see that it's doing a pretty similar thing to before um it's got the supervisor agent it's calling search and then it responds with supervisor we're now going to put together another agent or another graph of Agents so it's this document writing team um so here we are defining a bunch of stuff we're defining the state that we're tracking it's largely these messages um there's then uh some logic that's run before each worker agent begins um so that's more aware of the current state of everything that exists this is basically with populating relevant context we then create a bunch of agents and a bunch of uh uh agent nodes um um and then we create the dock writing supervisor um now that that's done we can create the graph so we're adding a bunch of nodes we're adding a bunch of edges we've got some conditional routing edges um we set the entry point um and now that we have this we can do something simple like write an outline for a poem and write the poem to disk we can see that this is finished and if we look at this in Lang Smith we can see that it also looks fairly similar as to what we were doing before we've got the supervisor we do a bunch of note taking and then the supervisor finishes it doesn't see any of the internal things it just sees the final response and we can now put this together so we now have a layer on top of this we've got this team supervisor um and now we have this top level graph State it's just a list of messages we have some helper functions we start creating the graph we add the research team as its own node it's got a little bit of uh it's got a little bit of wrappers around the previous chain to like get the previous message and then p and then at the begin beginning pass it that into the chain and then call join graph to join it with the rest of the stuff same uh uh with the paper writing team we then add the supervisor we then add some nodes we compile it all right um and so now we can uh do a more complex thing where we call asket to write you know this brief research report on the North American sturgeon include a chart so let's kick this off and I'll come back when this is done all right so it's done took a while if we go to Lang Smith we can see that we have this most recent Trace took 400 seconds if we click inside it we can see that there is a lot going on so a lot of calls um down the hood it looks like there's this big call to the paper writing team at one point that took a while there was also this big call to the research team to start that took a while I can click into any of these I can see like what exactly is going on at any point in time what exactly was happening um so this is pretty helpful for understanding what's going on um yeah wow that's a lot of tokens um anyways yeah that's pretty much it for the multi-agent stuff hope this was helpful I think uh you know the the benefits of thinking about things as a multi-agent way are really that it I think it helps compar compartmentalize what you expect different functions to be happening inside the system so you know each agent basically has its own prompt its own tools even its own llm at times and so you can really focus it on just doing specific things as opposed to having one General autonomous agent with lots of tools and so even if you you know even if you have all those same tools and you just organize it a bit more hierarchically that can often be helpful for for doing kind of like complex tasks as well as just a good mental model to how to think about things so hopefully this has been a good introduction to that these are again probably only a few of the ways that you can create multi-agent workflows with L graph really excited to see what other ones emerge

Channel: LangChain

Views: 47,033

Keywords:

Id: hvAPnpSfSGo

Length: 24min 1sec (1441 seconds)

Published: Tue Jan 23 2024