Okay. So in this video, I want to go
through the five different types of LLM apps, that I'm seeing. And I'm proposing these as categories
to help people think about what they're actually building with LLMs, which of
these categories does it fall into. And then in the future, we can look
at some of the best practices around building each of these types of categories So of course there'll be some apps
that overlap from one to another. I find that generally, most of the apps
that I'm seeing people and companies build fall into one of these five
categories that we can look at here. All right. So the first category is
actually quite simple. This is like chat bots
and conversational agents. So people have been doing this for
a long, long time, over 10 . years,, But it's really only been with the
advancement of things like, the GPTs. and the better quality large
language models from the big providers and from open source. Okay, that has suddenly made these things
start to be viable and start to work here. So conversational agents can mean
everything from virtual friends through to customer support, through to anything that
you basically interact in a conversation as if it were a person in many ways. People are using these a lot for
things like customer support, like appointment setting. Also using them for outreach to people. Sometimes the conversation won't
be just like a simple sort of chat style, it'll be something
like emails going back and forth. But the idea here is that the
conversation is being driven by the agent in response to a human here. So a lot of the ideas here have been
promised for sort of 10, 15 years, but in the past, they've been very
limited to Only being able to work in very closed domains for very specific. kinds of conversations. They were very much heuristic
and rules-based approaches that didn't have a lot of flexibility. So this changed, obviously,
with two main models. So the, probably one of the
first ones that it changed with this was a model called Lambda,
which was a Google internal only. and also obviously the big one
that it changed with was ChatGPT, which was available to the public. Both of these models suddenly
had the ability to have wide open domain conversations. In a way that people felt like
they were actually interacting. if not with another human, then with
something that, had the ability to hold a coherent conversation and advanced
that conversation and return back. Interesting answers through this. So this has really opened up a whole bunch
of different opportunities for people to build apps around better customer
service, sales, virtual friends, virtual dating partners, coaches, A whole bunch
of things around this kind of topic. If you are looking to build an app
like this, some of the key things that you want to think about things
like the personality of the bot. What the bot will actually engage
in conversation in and what it won't engage in conversation? What is the outcome of the conversation? So if you're doing a customer service
bot, you're ideally trying to help the person get the result that
they want as quickly as possible. And then end the conversation. If you're building a virtual friend, you
probably want to keep it going on, and then you want to think about, okay, how
are you going to store a sort of memory? that when humans have conversations
together, they're able to store different memories over time and to be able to
bring that back up and follow up and ask questions and stuff like that. Conversational agents are very
interesting kind of application for large language models. it's certainly one of the ones that has. Kind of amazed people a lot. the whole sort of idea for a long
time of the Turing test of being able to have a conversation with a bot. that you couldn't tell if it
was human or if it was a bot. In many ways we're there, you know,
if you know how to treat these things, you certainly can trick them. But for a lot of people, they're
having conversations with these things and treating them as if
they were another human being. All right. The second kind of large language
model app that we're seeing a lot or the whole sort of Copilots and duets. So I've taken the names from Microsoft
and Google who really sort of pushed these ideas of basically having a Copilot
or a duet, an assistant that is like there to help you as you go through this. So the idea of a Copilot or a Duet
is that it's able to make use of your current data and your knowledge
around a customer to help customers achieve goals easier and faster. So there are two main types of Copilots. the first one is the ones that we're
seeing a lot that are being embedded into different kinds of software,
different kinds of SAS applications, different kinds of online tools. And what these are doing are they
basically using a lot of current data that the company has about the customer. So things like customer knowledge
analytics, that kind of stuff. to be able to help customers
achieve their goals much easier. So these kinds of Copilots are
ones that can do a set of very specific tasks that are generally
going to be limited to a product. These kind of apps and these
kinds of products, favor incumbents and big companies,
much more than smaller startups. A much harder play for a startup to do
something like this, just because the big companies have so much data about people
using their tools, about people using their software and they can work out,
okay, what do people want to do with this? and then they can set up sort of like
interactive guides and interactive points of, discovery to help the
customer out, with the particular task that is that they're doing. The other kind of Copilot that we're
seeing, which is very interesting. So this is kind of like a subgroup
second subgroup in co-pilots and this is the whole ideas around
curation and guidance usually for things like education learning apps. So here where you may not have a
specific set of tasks around a piece of software, but you've got a specific
set of tasks around learning a subject or learning a particular topic. And these education apps will be able to
guide your journey through that learning. And curate what they're actually showing
you, exercises that they're giving you a whole bunch of different things like this. One of the great examples of
this has been the Khan academy, Khanmigo app where they're basically
using it to teach children. And the whole idea is that the
app can adapt to the level of the learner in this sort of case. I think this is one of the areas
that startups can get involved with. this is something that we're going
to see a lot more in the future for these kinds of apps going forward. Okay. The third category of these LLM apps are
the whole sort of chat with data or RAG. So RAG stands for retrieval
augmented generation here. these apps are becoming very popular. And we've seen a lot of people do
things like, chat with PDF, chat with your SQL database chat with,
all different kinds of things here. The idea here is that we've got some
sort of information retrieval system. The user is then able to use a
large language model to access the information that they're most interested
in that system, and to be able to bring it back, answer questions. One of the key things, with this
is that it allows for natural language interaction with large data. This can be used for a
whole bunch of things. Mostly it's going to be things
around question and answering stuff. so allowing users to access data quickly
to be able to do this kind of thing. startups are definitely, trying
to do these kinds of apps. Startups finding two main
issues with these kind of apps. So the first one is that if you're
basically just putting together a very simple RAG system, you're not much
more than a wrapper around an API. And you have very little moat. So, OpenAI introduced their
GPTs a month or so ago. And at that stage, you know,
people realize that all these chat with PDF apps we're kind of
like, Out of business overnight. Because really they weren't bringing
anything to the table Apart from a very simple as system for doing
retrieval augmented generation. The second point with these apps is
that doing really good high quality retriever augmentation is very difficult. And it's something that often needs
to be highly customized to the specific topic that you're working on. So this is not something where
you can just bang out a little bit of code and then expect it
to work for lots of different scenarios in different situations. So these chat with data apps suffer
usually from two things, either one there's no moat at all for
defending them in that they can just be reproduced in an hour or so. Or the second one that they're just
okay or good at things that are very surface level in that topic. And they're not fine tuned for
the actual end use that people are going to be using it for. So the fourth and fifth type of
apps not things that consumers necessarily notice as much. These are things where the large
language model is being used a lot more in the background than just the
interaction with a consumer, with a user, with something like that. So the fourth type, are, basically
all your traditional NLP tasks. So for a long time, people developed
models where they had it, you know, individual models for doing named
entity recognition and individual model for doing sentiment, and individual
model for doing data extraction. And more and more people are finding
that these large language models are just better at all of these tasks. So you're getting to the point now
where you can use a large language model to do something like co reference
resolution and work out in a sentence when somebody says, you know it, or
they, or he or she, what actually are they referring to that came before this. when they're looking at sentiment
to try and work out, okay, is this neutral sentiment? Is this positive? Is this a positive review? Is this a negative review? How much should we prioritize
our response to this based on how annoyed or angry the customer is? All of these things are things
that large language models are proving to be very good at. another one that's huge area is the
whole thing of, data extraction. So this is both getting data out of
things and also in many ways, converting unstructured data to be structured data. So these sort of NLP tasks were things
that, like I mentioned in the past, people would develop a whole model around it. They would highly specialize in one
particular task, but more and more we're seeing that large language models have
the ability, especially if fine tuned, to be able to do lots of these tasks. and achieve better and state-of-the-art
results for these particular tasks. The fifth type of LLM app,
that I'm going to talk about. It's probably my favorites. The thing that I've been working on
most for the past six, eight months. and these are autonomous agents. So the whole idea here is that rather
than, just have a conversation with a. Large language model or with an app
that's using a large language model, you're actually getting these apps to
Autonomously do different tasks So the big thing here is automation, right? This is the key idea is that you're
trying to basically automate some result by using large language models
to do reasoning and decisions here. So the whole idea is that sort of
humans are not needed these agents are apps that have the ability to
use reasoning and decision-making. they usually going to be very modular. so currently with the state of language
models that we have, you don't want to make one agent do too many things. You tend to want to make the modular
and make lots of mini agents. and then you will want to basically
give them ability to do things like having self-reflection, recursive
improvement for checking their own work. And it surprisingly, it turns out that
these large language models are often much better at looking at something and
then critiquing that and improving that then just being able to come up with it. So as humans, we would think of
that, okay, if I ask the model to do some kind of reasoning, it's going
to give me the best that it could. more often than not, you would often
find that by having another agent using the exact same large language model
check that reasoning and work out what could be improved in that reasoning. You end up getting much better results. So these autonomous agents are still new. people are trying to put
them into production for a variety of different things. One of the challenges that they
have is that You do need to have state-of-the-art models for the
reasoning and decision-making. This is one of the tasks that
unfortunately, A lot of the open source models are still not good enough at doing. so it requires things like the latest
Gemini models, like the latest GPT 4 models and stuff for doing this. Or it requires highly fine tuned
models for very specific types of reasoning and decision-making. I do think that this is the area
that startups and that, you know, we're going to see the most amount of
growth over the next year or two as people start building these things. certainly, like I mentioned earlier,
certainly the area that I find most interesting with large language models. And it's one of the areas that the
most amount of research is actually going into for these things. So very interesting times. anyway, just to recap quickly,
the five types of LLM Apps, are, your conversational chat bots. Your co-pilots and duets with there
being two types, one sort of helping people with specific tasks and software. The other ones being more for
education, curation guidance. the third type being chat with data apps. Fourth type being the
traditional NLP tasks apps. and then the last one being the
autonomous agents with these. going forward in the future, I do want to
spend a lot more time making some videos of how you would structure some of these
particular apps and some of the thinking about how you actually go about building
some of these apps for this kind of thing. Anyway, as always, if you've got
any comments or questions, I love to engage with you in the comments. So please leave a comment below. If you found the video useful,
please click like and subscribe. And I will talk to you in the next video. Bye for now.
