Dear Fellow Scholars, this is Two Minute Papers
with Dr. KƔroly Zsolnai-FehƩr. Today we are going to see how an AI can learn
crazy stuntsā¦from just one video clip. And if even thatās not enough, it can even
do more. This agent is embedded in a physics simulation,
and first, it looks at a piece of reference motion, like this one. And then, after looking, it can reproduce
it. That is already pretty cool, but it doesnāt
stop there. I think you know whatās comingā¦yes! Not only learning, but improving the original
motion. Look, it can refine this motion a bitā¦and
then, a bit moreā¦and then, a bit more. And this just keeps on going, untilā¦wait
a second. Hold on to your papersā¦because this looks
impossible! Are you trying to tell me that itās improved
the move so much, that it can jump through this? Yes, yes it does. Here is the first reproduction of the jump
motion, and the improved version side by side. Whoa. The difference speaks for itself. Absolutely amazing. We can also give it this reference clip to
teach it to jump from one box to another. This isnāt quite difficult. And now comes one of my favorites from the
paper! And that is testing how much it can improve
upon this technique. Letās give it a try! It also learned how to perform a shorter jump,
a longer jumpā¦and now, oh yes, the final boss. Wow, it could even pull off this super long
jump. It seems that this super bot can do absolutely
anything! Wellā¦almost. And, it can not only learn these amazing moves,
but it can also weave them together so well, that we can build a cool little playground,
and it gets through it with easeā¦ well, most of it anyway. So at this point, I was wondering how general
the knowledge is that it learns from these example clips? A good sign of an intelligent actor is that
things can change a little and it can adapt to that. Now, it clearly can deal with a changing environment,
that is fantastic, but do you know what else it can deal with? And now, if you have been holding on to your
papers, squeeze that paper, because it can also deal with changing body proportions. Yes, really. We can put it in a different body, and it
will still work. This chap is cursed with this crazy configuration,
and can still pull off a cartwheel. If you havenāt been exercising lately, whatās
your excuse now? We can also ask it to perform the same task
with more or less energy, or to even apply just a tiny bit of force for a punch, or to
go full Mike Tyson on the opponent. So how is all this wizardry possible? Well, one of the key contributions of this
work is that the authors devised a method to search this space of motions efficiently. Since it does it in a continuous reinforcement
learning environment, this is super challenging. At the risk of simplifying the solution, their
method solves this by running both an exploration phase to find new ways of pulling off a move,
and, with blue you see that when it found something that seems to work, it also keeps
refining it. Similar endeavors are also referred to as
the exploration-exploitation problem, and the authors proposed a really cool new way
of handling it. Now, there are plenty more contributions in
the paper, so make sure to have a look at it in the video description. Especially given that this is a fantastic
paper, and a presentation is second to none. I am sure that the authors could have worked
half as much on this project and this paper would still have been accepted, but they still
decided to put in that extra mile. And I am honored to be able to celebrate their
amazing work together with you Fellow Scholars. And, for now, an AI agent can look at a single
clip of a motion, and can not only perform it, but it can make it better, pull it off
in different environments, and it can even be put in a different body and still do it
well. What a time to be alive! Thanks for watching and for your generous
support, and I'll see you next time!
