Dear Fellow Scholars, this is Two Minute
Papers with Dr. Károly Zsolnai-Fehér. Today we are going to see how DeepMind’s
incredible AI teaches these virtual characters to move, play football/soccer
and so much more. Now when reading this paper I expected that they would learn the
basics, but what happened afterwards, I did not expect at all. They learned some absolutely
amazing tricks and even more surprising things, and yes, we are going to look at all of
them, but first, the humble beginnings. Now, this is a physics-based game, so to move,
these AI agents need to use their joints and produce just the right kind of forces and torque
on them to create sophisticated movements and amazing plays. So, let’s see. Oof. Unfortunately,
we are seeing none of that here. Why? Well, of course, because they haven’t
had enough time to learn yet. So, are these just the humble beginnings, or
is this it? Is this the limit of their AI? Not quite! But first, they need to learn to move.
How? Well, scientists at DeepMind say by imitating motion capture data. In other words, this tries
to copy how real humans move. This is not bad, at least it can now perform some basic movements, so
that’s great. However, that is not nearly enough, look. It cannot control the ball yet. Not even
close. But once again, let’s come back 5 days later and now, can it pull it off? Now, hold
on to your papers, and…oh yes! Yes it can, and with flying colors! It can now follow a
moving target, dribble, kick, and so much more. Now, get this, we will put these agents
into a training camp for no less than 5 years and see how they do. We started out
like this, and now. Holy mother of papers, these little AIs can play! And they can do
so much more than just moving around. Look, they can do these little tricks where the
ball bounces back from an invisible wall, and not only that, but this player tries to pass
to the other AI this way, and will they score? Oh yes! Fantastic. I have to say, these are
now pretty good players. Good job, little AI! However, wait a second, what about the 5 years
part? Did scientists at DeepMind have to wait for 5 years for this paper and hope that something
good comes out of it? Well, not quite! You see, in real life, one second passes exactly in
one second. That is not new information to anyone. However, if we have a quick computer,
in a simulation, one second in the game can be simulated in a matter of milliseconds in
real life, thereby speeding up time itself. Not our time, of course, but the time that the
AI lives in. So this 5-year training camp only took 3 days in real life. That’s not that much.
Hm. Are you thinking what I am thinking? Yes, if it’s so quick, let’s run it
for even longer. See what happens. They ran it for 50 days, and let’s see the
level of plays these can now do. And when I saw these results, I almost fell off the
chair. They have become even better. Let’s see what they have learned. Now, they not
only can think a few steps ahead, but they are even anticipating the behavior of their
teammates and position themselves accordingly. They also learned to use body to
body contact to their advantage. High kicks are also being performed to catch the
defenders off-guard, and I absolutely loved how they developed these quick turns to get away
with the ball. This is incredible. I love it. And they can even play their way out of
really difficult situations. Look at this one. Oh boy, the defending red player would have to
clear this ball without scoring an own goal, so it would have to kick the ball in this
direction, and the margin of error is razor thin. So far, good thinking, but can it
pull this off? Wow. Good job, little AI! The saves they are capable of are
also outstanding. And as we don’t really have a referee around, these
AIs haven’t exactly been encouraged to be gentle with each other,
so this can happen too. Ouch. And here is one of my favorite moments, after
a little altercation, the red player falls, and it almost seems like throwing a
little tantrum. This is super fun, especially that he was the one who
started it. I am sure some of you Fellow Scholars will point out that this
is not unlike some real football players. And of course, this little AI appears so injured
it cannot get up, but later when it is really needed to be there to defend, look! A miracle
happened! And of course, once again, this guy just cannot help it. This is one of the most
fun papers I’ve read in a while. So cool. However, we are experienced
Fellow Scholars over here, so we would like to know so much
more than what we’ve seen here. So, let’s look under the hood and see some
more about what they have really learned. These are the curves that I am looking
for, yes, and I see that their Elo ratings increase over time. This means that
as they learn more, they get better. That is very reassuring, however, I still haven’t seen
what my heart desires yet, so let’s continue. Now we’re talking! More detailed data. This is
so cool to see, look. Over time, they can get up quicker, on average, they use their joints better
to run faster, and yes! There we go. Division of labor. Yes yes yes! Wow. That is what my heart
desires. This is where we struck gold. This is the graph that shows that these two players
learned not only to move properly, but how to work together as a team. This is also reflected in
their passing frequency, which is increasing over time. But the division of labor, that is where the
real magic happens. You see, this is one of those elusive qualities that is quite difficult to learn
properly, often even for humans, and it really shows here too. How? Well, look, this is one of
the very few cases in the paper that does not show steady linear growth. At times, it even worsens
over time. After one day of training, the agents know just enough to score a goal alone, and thus
they become quite selfish. We all know that guy, don’t we? But, over time, they realize that as a
team, they can perform even better. Wow. What an incredible piece of work. And this is the moment
where I got goosebumps when reading this paper. I love moments like this and I am so happy to share
it with you. Remember, this is where we started, and this is where we ended up, and all this
through the power of modern learning algorithms. Yes, this is Two Minute Papers, the corner
of the internet where we look at a blue curve and make happy noises. So good. If
you enjoyed this, make sure to subscribe and perhaps even hit the bell icon if you
wish to see more amazing papers like this. So, what do you think? What would you use
this for? Let me know in the comments below! Thanks for watching and for your generous
support, and I'll see you next time!
