Okay. So a lot of people have been asking
about how can we use Llama 3 with agents? and specifically, how can we use
it with something like CrewAI? So in this video, I'm going to
walk through how you can basically get CrewAI working with Llama 3. And we're actually going to do it on
the Groq platform, which you're going to see is really quick for doing this. And we're actually going to use the
Llama 3 70 billion model on there. So if you don't have an account
on Groq, you will need to come in here and basically go to
Groq cloud or console.Groq.com. set up an accounting here. And here you can basically
pick the various models. And the model that we're going to be
using is this Llama 3 70 B, and it's just over 8,000 token context window So you
need to come in here and make an API key. And currently this is all free. So you could actually use this and use
the 70B parameter llama 3 for free here. Okay, so let's jump into the code. Once you've got Groq set up, you
need to come in and you need to PIP install CrewAI, LangChain-groq. And in this case, I'm using
Duck duck go search in here. And I've just put this together pretty
quickly, so you can have a play with it. You can certainly make it a lot
better what I will do is I will walk through, the example here. And then in the next video, I'll make a
bit more in depth, one using LangGraph and showing you how you could actually
take some of these ideas and flesh them out a lot more with LangGraph
so that you've got more control over what's actually going on in here. All right. So the goal here is that we're
going to reply to a customer email. So we're going to have a
customer email come in. And what we would actually do is we
would have it set up so that we would have a script that would basically call
the email account, get emails maybe once every half an hour or something
like that, once every 10 minutes. bring that email in, then run it
through this agent, take the response out and then send that as a reply. So to simplify this here, I haven't
actually put the scripts for calling, the email server and stuff like that. Those are pretty basic. and you can find plenty of
examples of those out there. But what I have put down here is some of
the sort of agent flow that this would go through, when you're making this. So I put down some original ideas. I've simplified it a little bit in
here just to keep the time down. but the idea is that you get an email. You're going to have one agent that's
going to categorize the emails. So this is basically going to work
out is this the customer inquiry? Is it a pricing inquiry? It is a customer complaint? what kind of email is it? Is it something that's off topic? we're then going to basically
take the email and the category. And we're going to pass
that to a researcher. Now you can see here, I've got my,
I've taken my flow and I've worked out here a simplified version of this flow. I've worked out here, they're gonna have
three agents and three tasks in here. Now to do some of the checking and
stuff like that, you would add in extra tasks and extra agents in here. But the idea here is I'm going to have
one agent that does the categorization. One that basically takes
that and then does research. Now, if you're building something like
this for a bit more real world use. what you would want to do is actually,
rather than have it researched the web, like I'm doing here, you would actually
build an internal RAG system where it would basically look up things from your
internal RAG and then bring them back. And maybe have a fallback tool web
or something like that if the answer wasn't in your internal, sort of
database or FAQ, that kind of thing. once the research has got the
information it's then going to pass it to, the email writer. And they're going to write
the response back using the information that we've got here. All right. first up you need to basically
set up your Groq API key. And we're going to basically use
this to call the actual, Llama 3 70 billion model in here. Alright, I've got a little
utility function in here. I explained that in one of the
previous videos, I'm not going to go through that here now. And I don't really use it
that much in this one anyway. All right. Setting up the Groq LLM here. this is pretty simple because we've got
the LangChain package in here for Groq. So we can basically just
import a chat Groq model. I'm just going to call that Groq LLM. you can see here, I'm passing
in Llama 3 70 billion. because my actual key is
already in the environment. I don't need to do that again,
but just showing you there, if you want it to do something like that. All right. I'm going to basically
bring in the, tools. some of the key things from my previous
video about making CrewAI stuff in here. And next up, we need to set up our agents. So here I'm going to basically just
make a class with each of the agents this time and a class with the tasks. and that's more important for
the tasks, than the agents here. it's basically, what I'm doing here is,
going to instantiate these and for the tasks I'm going to pass in the email. so that it can use the
email in there as well. But can see here our first agent is
basically a email categorizer agent. And I tell it, okay, these are the
categories I want, price inquiry, customer complaint, product inquiry,
customer feedback, off topic. Now you could add a lot
more here, obviously. I've just done this quite simply. I've put in a backstory, You are a
master understanding what the customer wants when they write an email. And are able to categorize
it in a useful way in here. So the category, I think, just
helps to guide the LLM of, how to respond and how to respond back. And if you're building this for a
sort of production system, you'd want to log, exactly like what percentage
of your emails were complaints? What percentage of them were various
different tasks and stuff for this. All right. The research agents. So once we've got the email category,
that researcher agent's going to basically look at that and is going
to do some research on the web here. Now, if it doesn't think that search will
help it, we'll just put, no search needed. And if it doesn't find anything
useful, it will basically respond with no useful research found, in here. And obviously each of these is going
to correspond with the tasks for this. The first task is doing the categorization
where you can see we're passing in the email content for doing that. And then the second task is going
to be doing the research and we're also passing in the email content. And we're passing in the context
of the categorize email for this. Now for each of these ones, I'm
out putting a text file just so we can look at the end and see,
okay, what was the email category? What was the research info? and then obviously last off is
going to be writing the email. so here, I've put in a little bit
of info about, if the, customer emails off topic, then we should,
get some more questions for this. how to address the different
categories of things. Now you could actually, flesh this
out a lot more and make this part of the prompt a lot longer to get
very specific about certain things. I always wanted to sign off the
emails in an appropriate manner from Sarah, the resident manager. So I guess I was thinking of this as being
like a resort that, someone is writing about the resort, that kind of thing. And one of the things I could
have done is actually put in a lot more context about what the
business is or what the resort is. You'll see, later on when I ask it
some questions that are really off topic, because they're perhaps a
price inquiry, even though they're not a price inquiry about the
resort, it will try to address that. And it will try to answer that in here. All right. So the last task is
actually drafting the email. you see, we basically have, some more
things in there about, just that we want it to be simple, polite to the point. we were going to respond, et cetera. again, we're passing in the
categorization of the email and the research info there, for this. all right, so here, I've got the emails. I've got four emails in here. So let's start off with a really
simple email of, we've got, hi there! I'm emailing to say that I had a
wonderful stay at your resort last week. I really appreciate what your staff did. Thanks, Paul. So you'll notice the names of
the people doing the emails. Now, what are you gonna do is instantiate
the two classes for agents and tasks. And then I'm going to basically, you
know, instantiate my different agents. And I'm then going to instantiate
the tasks passing in this email that I just created up there for this. I then basically build the crew. I pass in the agents and tasks. same as we've looked at in one of the
previous videos about CrewAI in here. And we can now kick it off. and you'll see that it actually
operates very quickly in here. so it's gone through. It's, taking the email in. It's coming up with a category. it's decided that's customer feedback. is the final answer, you know, for that. it does some research. Now it is doing some errors
with duck duck go in here. And honestly, I would probably
rewrite the duck, duck go if I was trying to do something serious to
be more of a helpful tool for this. We can see that it's finally done
a search where it's got it going. it comes back with some answers about
responding to customer gratitudes. So it's definitely on the
right track with this. and then finally it comes down to
actually writing, the email out. And you can see that here we've
got this final answer out. Dear Paul, We're thrilled to
hear you had a wonderful stay at the resort last week, et cetera. All right now, if we go through
just, cat out these, text files. We can see that, you know what
actually we got along the way. So the final sort of draft email is here. That's done quite a nice job. We can see that, yes, it
got customer feedback. And you can see that the research
basically just got, respond with personalized, appreciative tone. and it gives us, some tips
for the email writer in there. Let's try it with a different email. Okay, so this is going
to be a complaint email. So the email here is, hi there, I'm
emailing to say that the resort weather was way too cloudy and overcast. I wanted to write a song called, here
comes the sun, but it never came. what should the weather
be in Arizona in April? so that's obviously hinting
towards the research part. I really hope you fix this next time. So let's run this through. And see what we get out here. And you'll see that because
we're using Groq, this is really flying through, at this. This is pretty amazing that we're using
the Llama 3 70 billion model and you see it's already finished, in here. It's just going through so quickly. so we can see that, sure enough
our research agent decided Arizona weather in April. it got some information back of what
the temperatures should be, et cetera. It then basically took all that
information and it wrote a final answer. Dear George, thank you for the time
to share your concerns about the weather during your recent thing. I understand the cloudy, it
might have been a damper on your plans and I apologize. So it's handled that quite nicely. thank you again. And I hope that you get to write
the song, here comes the sun soon. and again, we can see that it's
sticking to this Sarah resident manager as it goes through here. And sure enough, yeah, if we look
at the different things, we can see that was a customer complaint. So it categorized it very nicely in there. We got nice research. Let's just do one more to finish up. And this one is going to be an email
from Ringo . And the idea here, is, that this one is more off topic. Okay. So you can see that we basically
instantiated everything. we kicked off The crew. It's gone through. It has actually worked out
that this is off topic. the email by the way is
why can't I get to sing? and you can see it's already finished. So I didn't even get that
much time to talk through it. Now again, we aren't getting this
problem here around, the queries. So we could actually rewrite
the tool for doing that. That would be pretty easy to
fix in here once we see it. But it actually does work out eventually
that okay, it should just, pass things. Now in this case, it didn't, it
returned no use for research found There wasn't really useful research
for, why can't Ringo get to sing more? and we can see that, okay,
we've got, the response back. This has come through and look at
these and we can see that we've used a lot of tokens, right? We've used like a whole
bunch of tokens in here. going through these requests and
because we're doing it on Groq, it's just so quick, going through this. All right. Dear Ringo, thank you
for reaching out to us. I'm happy to help you with your concern. However, I'm not entirely sure what
you mean by, why can't I get to sing because you probably give more context. So again, it's following the
instructions nicely there. and that shows the quality of
the, Llama 3 70 billion model, that it is able to handle these
things, pretty nicely in this case. again, signed off nicely from
Sarah, the resident manager. The email category was off topic. And the research was no useful
research found in this one. So as always the co-lab. for this and the code for this
is basically in the description. have a play with it yourself and,
experiment round with changing different things with this. I would say, I do find, CrewAI often
hit and miss that it can be quite finicky to get certain things working. But when you've got a good model like
this, Llama 3 70 billion, we can see it's really up there in comparison to
Gemini pro the 1.0, perhaps not the 1.5. and one of the Mixtral MoE
models as well in here. certainly interesting
to sort of look at this. and have a play with it yourself. like I said, in the next video, what
I'll do is I'll take this idea and we'll flesh it out with LangGraph and
we'll add in some more checks for this. And perhaps I think there, I will
also do a version, with Ollama so just as you can run this with Groq, we
could have also done this with Ollama with perhaps the 8 billion model. Maybe we're not going to get as
good results out of the 8 billion model compared to the 70 billion. But you can still get some
good outputs out of this. So anyway, as always in the
comments, let me know what you thought, what works for you. I'm very interested, if anyone comes up
with some tips about how to do it better with this, that would be really cool. If you've got any questions,
put them in there. And if you found the video useful,
please click like and subscribe. And I will talk to you in the next video. Bye for now.
