AI Learns To Play Table Tennis (Part 1)

in the last two videos we use this algorithm called PPO and today we're going to keep that streak going but this time around we will use it to create a table tennis AI now you might be thinking oh this is easy we can just build a table tennis Arena throw in a couple of ragdolls and let PPO figure out the rest just like the other videos and technically you would be right but you would also be dead because that'll take about a hundred years table tennis is a lot harder to solve than bowling or being Spider-Man so unless you're a really gifted and healthy toddler this won't work for you instead we are going to break this problem down into a four-point plan here's how this is going to go step one create a walking AI this AI has two goals move towards a Target and don't fall over Step 2 create a table tennis Aimbot using a few physics equations we can predict where the ball will go and use this to plan our shots in advance step 3 extend the walking AI by giving it a secondary task keep your right hand in the Box this box will move around randomly so the walking AI will have to gain full control over its right hand to succeed step 4 combine our AI with the Aimbot we will tell the AI to move its hand wherever the Aimbot goes while also telling it to stay on its side of the table this will allow the AI to execute the same shots that Aimbot does effectively making it a fully functional table tennis player this is a great plan and nothing will go wrong as I'm the greatest programmer alive so let's get straight into the walking AI the first thing we need to create a walking AI is a ragdoll luckily I've already built the perfect physical specimen in a previous video this little beauty is six feet tall and weighs in at about 85 kilos or if you're American about 773 cheeseburgers it also features 12 joints 13 bones and spherical feet we will take this Ragdoll and hook it up to a large neural network the structure of this neural network will be the same as before each joint in the Ragdoll will send information about its current state and the network will tell the joint which way to rotate our ai's job this time around is to figure out how to move towards a Target so we will give our AI a sense of direction by telling it which way the target is this is where things get tricky walking is a complicated task so we'll need to give the AI a well-crafted reward function for it to succeed to start with we will give the AA a large reward for every single time step that it stays upright this is essentially the AI equivalent of force feeding which will aggressively guide it away from falling over once the AI can stay on its feet we will give it a small reward for moving towards the target this reward will scale of how fast the AI moves and will also double as a punishment if the AI moves away from the target finally we will give the AI a punishment for moving its limbs too much you can think of this as a taser that will go off when it detects sudden movements this will give the AI a sense of energy encouraging it to move as efficiently as possible these three factors will guide our AI towards a walking technique that'll be fast reliable and efficient so our work here is done the only thing left to do now is run the learning algorithm or in other words we wait see you soon foreign [Music] [Music] [Music] our first learning session has gone really well and we now have a ragdoll capable of walking you can see that the AI relies on Fast tiny footsteps to move around which are kept in motion by the most active pelvis since Wilt Chamberlain you may have also noticed the odd head movement which it does despite being punished for it I actually have no idea why it insists on this my best guess is that the AI is smuggling extra information into a snap which it uses in the next time step to help it walk better ironically the only part of this walking technique that resembles a human is the arms which occasionally shift to help the Ragdoll maintain balance nevertheless it gets the job done so we are free to move on to the next step which is the paddle Aimbot so let's get to it the goal here is to hit the perfect table tennis shot but if we want to do that then we need to predict how the ball will move this is easier said than done the ball is very light so it'll be greatly affected by things like drag or spin or the Magnus effect luckily we can ignore all of this because this equation will give us a good approximation the ball's motion this doesn't account for everything though our ball can bounce so we will need to figure out when that happens let's run through an example we know that our table has a constant height and we can use that number to figure out how long it will take before the ball falls into it with a little bit of rearranging we arrive at this quadratic equation which can be solved for time we can then plug this time back into the approximation which will give us the position of the bounce since our table is not an infinite plane we will also need to check the x and z axes to make sure that the ball actually hits the table in addition to this we will reuse the calculated time to figure out the velocity of the ball at the Bounce from there we can simulate the Bounce by flipping the velocity's y-axis and applying some decay this routine of detecting bounces can be completed as many times as necessary eventually the ball will take its final bounce off the table at this point we can start calculating exactly when to hit it first we will Define where the strike is going to happen then we can rearrange the approximation again to find out exactly what the ball will be doing when it gets there this is where things get interesting we need to hit our ball to the other side of the table so we will need to give it enough velocity to get there and we will also need to figure out how to do this via a paddle strike the first part of this problem is quite easy to solve essentially I'm going to steal my own code this will calculate the required velocity for the hit but getting that velocity through a paddle strike is a little bit trickier let's look at this from a more visual perspective we know how fast the ball is coming into the strike and how fast it needs to come out what we need to figure out is how fast the paddle should be moving in maths terms this will consist of a normal and a velocity vector for the normal we can simply use the average of the incoming and outgoing velocity vectors using this direction the ball will bounce off the paddle like this now you might be thinking wait this doesn't work at all there's clearly a difference between the velocity we got and the velocity we wanted this is why we add velocity to the paddle the reason we chose this direction for the paddle is because it's parallel with the difference so if we set the paddle's velocity to this difference then we can be sure that all of that speed will be transferred perfectly into the ball and hence make up for the difference the only thing we need to look out for now is the energy that's lost in the bounce which we can easily add back by moving the paddle faster but other than that we've solved this puzzle we now have a fully functional table tennis aimbler and while it isn't perfect it's good enough to hit all sorts of shots precisely and it should serve as a great guide for our AI on how to hit the ball but what good is a God if we have no way to use it so with that in mind we can now move on to step three giving our walking AI Control over its hand surprisingly this is the easiest part of this whole thing and it's relatively straightforward to have our AI learn this new skill our AI will be given a new Target represented by this box this box will move around the Ragdoll periodically and our ai's new job is to move its right hand towards it normally we would have to train a new AI with extra inputs but luckily I already added the extra inputs into the network before in anticipation you can think of me as an AI trying endless amounts of random and occasionally learning something useful as well as giving our AI a new Target we will need to give it incentive to move its hand towards it it's important that we don't change the original rewards and punishments that allowed our AI to walk since it may unlearn this skill if they aren't present what we will do instead is add any reward on top of this system this new reward will scale with how close the right hand is to the box which will encourage AI to minimize the distance between them surprisingly this is all we need to add so we only have one thing left to do now which is wait see you soon foreign [Music] [Music] [Music] our second learning session has gone well and we have successfully expanded our AI skill tree and given a control over its right hand which it picked up quite early due to the random teleporting that the target did the AI can pretty much do anything with this right hand it can wave it can draw circles or it can even pretend it's using a Shake Weight the walking technique is basically the same as before but there is a small difference here that I thought was pretty interesting since our AI is no longer T posing there is more weight pulling it down to the left it compensated for this by leaning slightly to the right this is disturbing the human-like I mean seriously try holding your arms out like that you'll feel your body do the same thing so our AI has done its job beautifully and it's now ready to become a fully functional table tennis player that unfortunately we've run out of time this concludes part one in part two we will combine our Ragdoll and Neymar together and then have a player match against itself special thanks to K gene they could have chosen any of these Cosmetics but they chose that if you think you can do better join the Banana Fish cult on my patreon you get to make your own fish and you'll appear in my future videos foreign

Channel: b2studios

Views: 116,770

Keywords: b2studios, ai, b2studios ai, tabletennis, tabletenis, ping pong, b2studios ping pong, ping ping, b2studios tabletenis

Id: 9JW41BNH9CM

Length: 11min 20sec (680 seconds)

Published: Fri Aug 11 2023