Automatic pool stick vs. strangers

come on it's okay you're still a winner you're the best loser that i know instead of being a loser that's somehow a winner let's actually win i don't have skill but i do have technology so let's be honest robots are better than us they're more precise great at computation and they work long hours for no pay i'm going to apply these strains to pool to make a robot that will find and take shots that humans can only dream of i'll also be using this robot for something that you might not expect i'm going to use it to play pool online physically which raises some pretty interesting ethical questions it definitely doesn't seem like it's working this was kind of a random idea but turned out to be really fun this just might be the star of the whole show playing pool takes skill for now at least it's also a game with many variants i'm going to be focusing on 8 ball pool two players take turns trying to sink all their balls by striking a white cue ball with a stick the gameplay boils down to two steps figure out what shot you want to take and then hit the balls very precisely to execute it i want the robot to do both of these things figuring out what shot to take henceforth referred to as the big brain algorithm is going to take some explaining and i'll get to that in a minute the idea behind the robot is a bit more straightforward say i'm trying to make this shot if i had a robot in the tip of this queue it could move the tip so that it hits in just the right way so that the ball goes in here's what we're gonna have to build a camera system to watch the game and figure out what's going on a complicated algorithm to interpret this and figure out the best shot and a pool cue with a magic robotic tip that can take the shot for us this is probably the most complex project that i've done the robot q is obviously going to have to move and rotate laterally but pull is not 2d where you hit the ball vertically matters a lot off center hits put spin on the ball which gives you control over where it goes i want the robot to have control that means moving and rotating vertically as well fitting a mechanism that can do all these motions into the tip of a pull cue is going to be very hard it's going to get fat but my goal is for it to at least remain a stick i have an initial design but i don't have a good feel for it i need to hold it i haven't really talked about my glowforge but it's really useful for making quick prototypes like this i use it kind of like my plasma cutter where i can very quickly make structures out of flat patterns except for in this case out of plastic and wood rather than metal i'm glad i quickly prototyped this because there's no way this is going to work it's just too bulky and there's no way i'm going to be able to get this stiff enough deep down i knew that a steward platform was probably the right way to go here but i've done everything i can to suppress that thought because i know it's going to be very painful to make the idea is that you arrange six telescoping rods in a very special arrangement you can completely control how this platform moves around by extending and contracting these rods i can stick a tiny one of these in the queue to move it wherever i want it's very stiff so it won't deflect when i take a shot and it's also really compact i wanted to avoid a stewart platform because there's a ton of finicky little parts instead of telescopic rods i have the linkages on these cranks which push them up and pull them down but it's essentially the same the defining feature of this stewart platform is that it's driven remotely through tension cables the servos are on one side of this broomstick and the moving end is on the other the electronics are pretty straightforward they're mostly off the shelf microcontrollers and other components man that end got thick not really the best look there we go these platforms take a bit of math to move them around but it's really not too bad say i wanted this platform to move over and rotate like this one of the actuators gets shorter two get longer and it results in it being this position if i want to calculate how long these actuators need to be i have the solution right here in front of me this is how long they need to be for it to go here conceptually all i have to do is move the platform where i want it and then measure the distance between these sets of points to tell me how long each actuator needs to be i solved it by drawing a picture but it's also very easy to solve this mathematically using linear algebra that is exactly what my program is doing this is a case where the results make it look much harder than it actually is if i did everything right the platform should move wherever i want now [Music] wow working on the first try is not normal it's pretty cool how the motions all combine isn't it this air cylinder on the tip of the queue will be used to fire the shot it's connected to pressurized air when i open a valve that shoots out really fast but not all shots are done at maximum power we need a way to vary the power from gentle touch to battering ram to do this i built the power pack this tank is normally intended for car horns the tank has a small wimpy computer attached to it which people call a microcontroller the microcontroller measures the pressure with this pressure gauge if the pressure is too low it'll open this valve which will connect it to my shop air and pressurize the tank if it's too high it'll open this valve and release the pressure and to fire i open this valve it's really simple and works well this cue is really cool but it's completely useless without some more information if i'm holding it here how does the computer know that it needs to move it over here so the shot will go in i have to know where everything is to be able to figure that out i have this camera mounted on the ceiling looking straight down it can see everything going on on the pool table it has a big problem though although this table was the cheapest one that money can buy it is not this out of square this image makes the balls look like they're over here but in reality they're over here this is caused by the camera being out of square with the table if i took the position of the balls from this image then i'm going to calculate the wrong thing programmers call this garbage in garbage out i've heard this also applies to raising children i glued these funny looking tags to all the corners of the table and they're called fiducials they're basically a reference point that's very easy for a computer to pick out of an image kind of like a qr code i can find these four points and then use them to straighten the image up i know where the holes and bumpers are relative to the fiducials so i know where they are now too and then if i find the balls i know where they are and the last thing to find is the queue which is why it has these two fiducials one fiducial tells me where it is two fiducials tells me what direction it's pointing there is still one little problem the camera only tells me where things are laterally because it's a 2d image i don't know how high the cue is above the table or how it's tilted normally you rest the q on your hand but the height of my hand isn't controlled there's an easy fix for this it's just like my hand except plastic screwed to the queue and most importantly known height i still don't know how it's tilted so i added this imu to the end of the queue and imu is what you give someone when you owe them money but you're broke imus can do all kinds of things but in my case it tells me what direction is down so now i can compensate for tilt i still need to make the big brain algorithm to figure out the best shot for now all it can do is a single ball into the hole i show the shot in this really simple interface that i made and if i want to hit this shot it tells me that i need to hold the cue in this area the problem is that the ui is on the computer and i'm at the table i guess i could look at the computer screen but that's awkward i want to be unencumbered i want to enjoy the game of pool the way that it was meant to be played i've got an idea i'm going to project the user interface onto the pool table oh no one little tiny hole took out my furnace my lights my security system and my internet this must be the most important cable in my entire house all right let's see if this thing works i have an annoying problem the projector is just projecting on the table it has no sense of where anything is if i just display my ui on the projector it's totally wrong cue ball is here not here if i project a grid of fiducials with the projector i can figure out where the projector can project relative to the table from there it's easy to make the ui line up with anything all right we have everything we need to try it let's see what this thing can do at this point it looks like i succeeded in building exactly the opposite of what i wanted which is a pool cue that always misses even if the shot is really easy this would make a really good wife mode for some reason the robot is moving to the wrong position sometimes so what could cause this i have a camera watching the table a program which modifies the image to correct for camera distortion and then another program which extracts the positions of everything this is plugged into a hacky physics simulation which does a lot of math to decide the angle of the tip which is then converted to six actuator positions which are then realized by an untested robot so why is it going to the wrong position i have no idea i present to you a week of pain in 20 seconds alright i think i figured it out no these plastic pulleys sometimes slip let's redesign them that is the wrong ball i found some math issues which must be causing problems you know what math is useless i made a system that records where the cue goes versus where i want and then corrects for it this has to work at this point i'm just systematically testing everything this is painful but eventually you run out of stuff that can be wrong i finally found the problem these six little servo motors are what control the end of the platform they were sold to me with specs of how far they can rotate but two of them don't do that which means they're rotating a different amount than i think which makes the end either sort of wrong or really wrong depending on where i'm trying to move it in my defense you sort of assume that the thing you bought does what it says that it does all right let me show you what this thing can do have you ever woken up from a nightmare but you're actually still asleep and you're in another nightmare yeah it turns out there's more than one big problem so something somewhere is broken i know that but i have no idea what it is i feel like i went through everything trying to solve that servo problem i flailed around helplessly for quite a while we're gonna just fast forward through all that oh steps up this is the good part now back a little bit it looks like things are in a slightly different position than my software thinks and remember garbage in garbage out it's not a huge amount but it's enough to mess up most shots i think this is due to some kind of optical distortion if you've ever used a wide-angle camera lens you'll have some idea of what i'm talking about if i take a wide-angle view of the table and square it up these points are square but look at this edge it's supposed to be straight this is a distortion caused by the wide-angle lens my camera already has a calibration that should correct for this but i'm guessing something somewhere is wrong day 4 of flailing and i'm trying to decide if this is what the pit of despair really feels like everything i do that should make the system better is making it worse because there are so many places that this can be wrong i'm doing a hail mary by developing my own calibration system the rough idea is i take a bunch of pictures of something that i know the shape and size of i can solve a series of equations that'll take a location in the image and directly calculate the corresponding location in the real world since this goes directly from the image to a predicted location it will skip over anything that might be broken this picture is giving me an idea of the distortion on the pool table i don't see any physical way this could be an actual optical distortion which implies that i am an idiot you know how occasionally a surgeon will accidentally leave their scalpel inside of you and then you're trying to figure out why you're sick all the time but you never thought to look for the scalpel inside your belly the same thing happened to me and i guess by me i mean my code way back in the beginning when i was developing this code i was doing a little test and i accidentally left that code dormant deep in the bowels of my system this code does a camera distortion correction and this correction is for a completely different camera than what i'm using this was subtly messing up my images and it could be applied in a couple of different ways and in one case it was applied twice it made it very confusing i'm exaggerating the effect so you can see it it was a lot more subtle five to ten millimeters of position error so this is roughly equivalent to punching myself in the face for five days straight all right moment of truth all right one two yes finally they went in and it wasn't random a neat thing i realized you can do is use this to guide normal pool shots too this thing is just really fun although if i'm going to go against my wife i needed to do more than just hit a single ball i'm going to get to the big brain algorithm soon i promise but there's one thing i want to try first so it should be possible to play pool with someone over the internet if i send a video feed of the pool table to someone and then i give them some controls to select a shot to take the computer can tell me where to hold the stick and then it will take their shot for them physically so it's like online pool but real which is weird i went ahead and built it we're just going to skip over that park so it's kind of boring all right let's test it with a game with the wife first when it's her turn she chooses the shot on her computer it tells me where to hold it the trigger tells the computer i'm ready and then she hits fire cool i'm even gonna beat you from upstairs i think you've just been lucky that was pretty close it doesn't seem like it went where i aimed the queue not your best shot ever this is definitely broken it seems like it's doing exactly what it's supposed to do we should just play when you fix it all right it's working perfectly hey how's it going this concept is so weird i basically built it on a whim but it is surprisingly fun boom i really thought this would be basically the same as online pool maybe a little crappier but it really is different i was so engrossed i didn't even notice that i lost the game wait where'd they ball go yeah i've actually been looking for it for a while wait well that's that i'm playing with some people who support me on patreon which helps me a lot so i'm trying to understand why this is so fun i think it has something to do with it being a real game in fact i challenge you to find an online full game with a more realistic physics engine than this one it's almost like two people holding the stick at the same time which is strange oh who is this redneck yeah man you just call someone up and play a real game of pool with them nice through the internet through the internet maybe this isn't just a goofy tangent it feels like a legitimate way to connect and have fun with others oh you sunk you moved it huh nothing remember when they connected pokemon to twitch what if i did the same thing with this pool table basically me versus a crowd do you think that'd be worth doing why didn't you go for the green i because i call the shots literally i call the shots you're just you're just the meat servo to align my servo all right finally we get to the fun part the big brain algorithm the idea is to make a physics simulation based on what the camera sees this should look at the entire table and calculate the best shot for you so what is the best shot this shot may be the easiest to sink this shot is a bit harder but it leaves the ball right next to another ball which will be easy to sync what i want is a list of all the possible shots that i could take and then i can select the shot i want from that list based on risk and probability and whatever other criteria matter imagine i have a table with three balls on it the simplest way to figure out all the possibilities is to just brute force it i can make a physics simulation where i virtually take hundreds of thousands of different shots and see what happens there's a lot of reasons doing it this way is bad the biggest reason for me is that this is slow so here's what i'm doing i look at each ball and i calculate all the ways that it can go into holes bounce off other balls and anything else that would advance the game it makes a big fat mess i do the same thing for all the other balls now i have all the possible things that each ball could do individually the question now is how do these things relate here are a few of the many things that the red ball could do could any of these things be caused by something that another ball could do one of the possibilities for this cue ball is that it directly hits this red ball in reality this cue ball can hit this red ball anywhere in this range if we hit it right here that would send the red ball along this trajectory the white ball could also bounce off the bumper into the red ball which could also cause this shot to happen or it could make the red ball hit this blue ball that hit could make the blue ball go into this pocket if you do this systematically for every ball you end up with a dense network of all the possible outcomes this structure is called a graph specifically a digraph because it's directional since everything begins at the white ball all the paths through this represent a potential shot that i could take this is a very simplified version that i've drawn the real version is just a gigantic mess if you've heard of ray tracing before what i did here is kind of a second cousin of ray tracing it looks incomprehensible but this graph is very easy for the computer to process and it has all the information we need to rank the shots for example how long is the shot or how precisely do i have to hit the ball i can ask the graph for the shortest path the longest which is almost certainly impossible the robot currently chooses shots based on a blend of distance precision and estimated speed i'm feeling good about my odds of winning at pool for once a lot of people will tell me that it would have been more efficient just to practice rather than building this whole system i agree but here we are although before i go up against my wife i want to talk about this video's sponsor so you know how this works the sponsor is what basically enables me to spend so much time developing insane pool cues and other crazy things if you like this video or you like me taking a minute to check out the sponsor is one of the most helpful things you can do to support me and these these projects this video sponsors micro center and you need to know about them when they asked me if i was interested in doing a sponsorship i just said where do i sign the amount of hours and dollars i've spent at micro center is just i don't even want to know how much money have i spent at micro center um lots i need a jumbo calculator for that one if you are remotely interested in technology you have to go check out one of these stores there's just nothing else like it they have everything it's the only physical store i can go to and buy parts for my robots their selection of computer parts is just fantastic and somehow their prices are also really good my computer is built entirely from parts from micro center i don't know if i'm allowed to say this but they employ a bunch of nerds which is amazing when you're trying to figure out what components you should get for building a pc all that i'm really asking is for you to go and visit a micro center there's 25 of them in the u.s and more people should know about them and before you hop in your car and peel out if you take just a second to click on the link in the description and enter your email address this lets microcenter know that stuff made here sent you this is how they decide if these videos are worth sponsoring which helps me out a ton but it also helps you out because when you do this they'll give you a coupon code which you can take to the store and they'll give you a free micro sd card and flash drive so thank you for hearing me out and thank you microcenter for supporting this video q versus wife who will win oh those things perfect there's some initial back and forth but then the robot really hit its stride [Music] i can't hold it still enough i have no shots now yep my little stripeys are blocking you and then the eight ball nice you really didn't win though the robot one it was my brain encoded in the software sounds a little creepy let me put it this way if your kid wins do you win yeah then i'm a winner the robot is not your kid we've talked about this getting this thing done was a bad as hard as labor oh please so hard all right i think this pool cue is awesome although there are still a lot of things that i just didn't have time to get to i'd like to take the pool move solver to the next level and have it look ahead multiple moves like a chess engine this is where ball spin becomes really useful and so far i pretty much just ignored it i think this is a good proof of concept and hopefully you liked it if you have any thoughts or ideas let me know i'd love to hear them thanks if you like this video please consider subscribing it helps me out a lot and keeps you in the loop you

Channel: Stuff Made Here

Views: 10,316,383

Rating: 4.9606438 out of 5

Keywords: billiards, pool, automatic pool, engineering, stuff made here, robotics, computer vision, robot path planning

Id: vsTTXYxydOE

Length: 21min 18sec (1278 seconds)

Published: Mon Feb 15 2021

Reddit Comments

[u/neetoday]

I am constantly astonished at this guy's capabilities in ME, EE, fabrication, programming, physics, troubleshooting, video editing, explaining complex topics/storytelling, and pure entertainment.

[u/djadry]

He just gained a new subscribed. Really an impressive project.

[u/Nishse]

He is the next Mark Rober I feel, just with 10x more technical knowledge.

[u/Beli_Mawrr]

One day, if I get even decently wealthy, I want a workshop like what he has.

[u/PossibleRussian]

I love this guy's channel. Thanks for posting. So I guess since this video isn't a collab with LockPickingLawyer we don't get any resolution on who won that challange?

[u/1wife2dogs0kids]

Can someone please get this guy, Mark Rober, The mythbusters, smarter every day, and NDT and put them in the same room like the mythbusters studio, lock them in, and see how many of the worlds problems they can fix? Actual problems. Like making flip flops that don’t flop when they flip.

[u/RobertGracie]

Stuff Made Here has yet again made a bar room game rather interesting and exciting, I will admit to chuckling that he has a "wife mode" on his builds to really mess around with his wife, but sooner or later she will have her own revenge on him! that day is coming!

[u/sketch_fest]

Imagine having money

[u/luozang]

did he called that guy red neck?!😆