This is jigsaw. He's a friendly little robot that's
really, really good at only one thing: Putting together any jigsaw puzzle,
no matter how complicated. Really, really fast. It's taken us three years
to get to this point, but according to our initial tests,
we have hopes he might be 200 times faster than the fastest competitive
jigsaw puzzler in the world. So today, we're going to walk through
what it took to get ready for the ultimate robot
versus human face off. And along the way,
we may just discover some tricks you might find helpful as a mere human
jigsaw puzzler yourself. But before we unpack how jigsaw does
what he does I first want to give us humans
some well-deserved credit because while this seems pretty straightforward of me to be able to pick up and arrange these 12 pieces of a puzzle, I'm actually
doing four very complicated tasks. The first is just picking up a piece. Have you ever stopped to think
just how amazing our hands are? Hiding beneath
that skin are 27 bones and 34 muscles, which makes them flexible and strong,
but they're also incredibly precise and dexterous thanks to the high concentration
of nerves in our fingertips for sensing pressure
textures, and temperature. We've also evolved
to have opposable thumbs, which makes it way easier to hold tools and to manipulate and pick things up than if it was just
five fingers all side by side. Step two is rotating the piece to the correct orientation,
which again is pretty straightforward when you have all the abilities
I just mentioned. For step three, we need to move the piece into position,
and that requires our whole arm. As I’ve mentioned before all mammals have the same
basic arm bone configuration from a human to a bat,
to a chicken to a turtle to a dolphin. But when you include the hand,
our configuration is the most technically capable arm of any living thing
to have ever existed on this planet. If you imagine a large cube
in front of me, it's wild that us humans can move this puzzle piece to any position
and orientation within that cube. That's really hard for a robot or pretty much any other species
to do for that matter. For step four, we have to decide where
this piece should go and for us humans, it's hard it’s hard to explain how but when we look at this,
it just very quickly feels super obvious this piece should go here. What's actually happening,
though, is our eyes communicate visual perception to our brains,
which then subconsciously synthesize a complicated combination of pattern
recognition, spatial reasoning, visual memory, and executive function,
and as a result, in fractions of a second? The answer just feels obvious. And not to brag. But this is once again where us humans
are the undisputed champions. Our complex brains are
what make us special. Physically, we're kind of unremarkable
in the animal kingdom. We're not faster than cheetahs
or stronger than bears. We can't swim as well as dolphins or fly
like an eagle. It's our brains
that make up 2% of our body weight, yet they consume
20% of our energy every day. And that ratio is higher than any other
living thing ever. And it's the reason
we're the ultimate apex predators because it allows for the huge survival
advantages that come from tool use planning, problem solving, language, and large scale
cooperation with other humans. It's also what has possibly, up
until this moment in history,
made us the best... at jigsaw puzzles. So if we wanted to make a robot
that was as good or better than us at puzzles,
our daunting challenge was to take the 200 million years of evolution
that enabled those four steps, and figure out
how to translate it into things a robot can do For number one,
to pick up a piece in lieu of an opposable thumb
and 27 hand bones we used a tiny, specialized suction cup that's often used to manipulate
small objects on assembly lines. This solenoid here can cut off
and then connect to this vacuum pump, which means we could turn on the suction
exactly when we want to pick it up, and then turn it off exactly when we want to let it go For number two we attached the suction cup
grabber of jigsaw to this very fine
tuned donut motor. That's precise to an incredible
0.005 degrees. That means if you attached an infinitely
sharp knife to the end of the motor, it could slice your circular birthday cake
into over 65,000 pieces. For number three to move a piece around,
we modified our avid CNC router we're constantly using around here
for our large builds by upgrading the motors
to ClearPath industrial servo motors. These are the same motors
we used on the Dominator. Our autonomous Domino robot. Once we did this, we saw it could accurately place a puzzle piece
down to .0005 inches That's one tenth
the width of a human hair, which means, as you can see here, jigsaw can take the lead
out of a mechanical pencil and move all around the table, and then
come back and put the lead right back in. So now that jigsaw could pick up rotate, and move any piece
with terrifying precision. The only thing he lacked was step four,
knowing exactly where to place the pieces. And as you might guess,
this was by far the hardest one to solve because all the subconscious work
performed by neural pathways in our brain that handle pattern recognition
and spatial reasoning, that makes finding that makes finding the right piece
feel so obvious to us is a really, really hard problem to solve using just computer logic and code To make matters worse just as we were really struggling to come up
with a good solution to this problem, the most devastating thing that can happen
as a YouTuber actually happened. My friend Shane from the YouTube
channel Stuff Made Here released a video about, you guessed it,
a machine that solves jigsaw puzzles. And what you should know about Shane
is he's probably the most technically capable engineer
I know. If there are like 15 categories
that make you a good engineer, my knowledge goes pretty deep in maybe
four, and I'm a generalist in the rest But then I have people like Ian and others on my team
who help fill in the gaps. Shane, however, is somehow
an expert in all 15 which is why you should subscribe
to his channel. When I told him
he beat me to the finish line, he told me he personally wasn't satisfied
with where he landed the project and encouraged me to keep trying.
So I persevered. And that's when our fortunes improved,
because while filming a video in Rwanda covering the work done at Zipline,
where they deliver life saving blood using autonomous drones,
I got to spend some time with Ryan, their co-founder, who's responsible for
all their complicated software algorithms. And after hearing all of our challenges by the time the plane landed on the way back
home, he'd coded up a solution that ended up being the backbone
that unlocked everything for us. Basically, instead of trying to
compete with our brains very complicated
pattern-matching neural network we took a much more simple approach by completely ignoring
what was printed on the puzzle and just looked at the edges so while painting it white makes it more impressive
and way harder to solve for us humans. It would make no difference to jigsaw. So to do this edge analysis,
we first need a set of eyes. but not those eyes. Those are just decorative
because it's no secret I love a good pair of googly eyes. And these were our most advanced
set of googly eyes ever. Because the googly is controlled
by two servo motors jigsaws real eyes were just a cell phone camera. The idea was to run a serpentine pattern
over all the pieces and take a picture of each one, and
then isolate them against the background then convert the edges of every piece
into four splines. So if we wanted to find a match,
say for this edge, we would just need to take that spline
and then find the puzzle piece that had a corresponding edge spline
that matched up perfectly. Because matching pieces
share identical edges. Now, even for a small, simple puzzle
like this, that's over 200 edge comparisons that need to be made. So the first step is just to look
at the overall length of each spline and disregard any edges
that were significantly longer or shorter, and that typically reduced
the solution space by about 50%. Then for the remaining candidates,
we would overlay the splines and then quantify the mismatch
by calculating the area between the two edge splines. So two pieces that were obviously not a
fit would have a lot of overlapping area, whereas two pieces
that were a perfect match the overlap area would essentially be zero. From there, we looked at all the possible
spline matchups and ranked the pieces as potential matches with each other
from least overlap area to most. Then jigsaw would start with one of the four corner pieces
and map out the potential solution space, which you could picture
visually like this. So the correct solution is the only one that finds a good fit
for all 12 pieces of the puzzle. But as you can see,
there's lots of forks and dead ends. And that's because sometimes it was
really obvious what the matching piece was. But sometimes there were 3
or 4 close contenders because the edges are so similar,
and our pictures weren't always perfect, so jigsaw would always pick what seemed like the best match
for a set of two pieces. But at some point
the puzzle would only be halfway done and there would be no more good candidates
to match the edges to continue solving the puzzle. And that's a dead giveaway he chose the wrong piece somewhere
previously up the chain, so he would work his way back to the closest fork
and then choose the next best option, and he would keep doing this over and over until he found a path
that finally connected all 12 pieces with essentially zero spline
mismatches at every connection. And then at that point, because he took pictures
of all the pieces, he knows exactly where each of them
is starting from over here. And he can simply assemble
the puzzle using the precision he demonstrated back in steps
one through three. So it was working on a 12 piece puzzle,
but now for the real challenge. Could we scale that up to a 1000 piece,
all-white puzzle? Since our ultimate goal was to dominate
the world's fastest human puzzle solver. We figured maxing out the number of pieces
would give the advantage to the non-overwhelmable robot. So after a couple more long weeks
filled with plenty of late nights working through all the challenges
that come from scaling up a simple prototype
100 times over... we landed here. It's 3 a.m. after three years. We're at this moment about
to fully solve it for the first time. What's our confidence level? I'd say 100%. Wow! The code always works
on the first try And so with that
very heavy dose of sarcasm, jigsaw got to work starting first
with taking all the pictures. And that took about 1.5 hours. And after we had the pictures, and we did
the prep work on them I mentioned earlier it was time for jigsaw to actually try
and solve the 1000 piece puzzle. And astoundingly, while running on
just a simple laptop the actual time to figure out
the correct placement of all 1000 pieces was less than a minute. And I love how you can actually see him working his way around the puzzle
from the top corner solving, and then going back
each time he hits a dead end to try and find a different fork
in the solution space, until he finds the only combination
that makes them all fit perfectly. So at this point, all that was left for jigsaw to do was get to work
placing the pieces, since he now knew where every single one needed to go and things were looking really good. And just as we made the mistake
of having some kind of glimmer of hope, a single piece
didn't quite snap into place. And then a bunch more followed
right behind. And upon troubleshooting,
what we discovered was there were a few
very small sources of error that tended to compound on each other. The more the puzzle was assembled. So, for example,
there's always a little slop between any two puzzle pieces,
which adds up over a lot of pieces. Or another source of error is
the whole puzzle itself shifting slightly when certain pieces were laid down. And this led us the realized
jigsaw was actually still missing one final important
feature us impressive humans possess Because if you think about it, when we assemble a puzzle we first just approximately place
the piece where it needs to go and then we rely on those
highly sensitive nerves in our fingertips to form a feedback loop with the brain to make really tiny adjustments
until we feel the piece fall into place and then a double tap is the
customary move to make it official But since jigsaw doesn't have nerves
in his gripper, we approximated that with the z height encoder on
a spring-loaded linear slider and that encoder is so accurate if you slice a human hair
lengthwise into 50 pieces, jigsaw could feel that tiny hair slice
if it was resting on top of the table. So now if jigsaw went to place a piece and he could feel it hadn't
quite snapped into place because it was resting too high, he would employ a wiggle routine
where it would just barely translate the piece into various directions
until he got the feedback from his finger that the piece was fully set down at which point he would give it
the customary final tap like any puzzle solver worth his salt. And so, with Jigsaw's new hardware
and software upgrades he was ready for one more final attempt. And I should mention, by the way,
from a combined hardware software perspective,
this is by far the most challenging build ever on my channel,
and that includes the auto Bullseye dartboard
and the automatic Domino Robot Dominator. So if you really want to appreciate all the juicy details,
including all of the source code, you can find a link to the full
write up in the video description. You boys ready to try this again
officially for the 48th time? Better work this time. What's the confidence levels at? 100%. Have you learned nothing Ian?! Hit the dang button So jigsaw got to work
and after taking pictures of all the pieces, he once again
actually solved the whole puzzle in a mere 55 seconds which meant it was time
to put them all together. And thanks to the wiggle routine you got this jigsaw. The first 20% was knocked out
in less than an hour. Okay, I guess we could sit back down And that meant
while jigsaw was doing all the hard work, I just get to kick back and relax
in a state of Zen gazing at this polar gantry sand garden
that you can build yourself. Because unlike the CrunchLabs build box, which is made for kids, we just launched
HackPack, which is kind of similar It's just more advanced and created
specifically for teenagers and adults. With HackPack, you get a series
of really fun programable robots delivered right to your door,
where we build it together and learn, step by step,
the kinds of engineering skills that go into making the builds
on my channel. They're all super fun and even useful,
and they'll work right out of the box. No programing required, but since my goal is to get you to always
level up your skills with some simple code tweaks
we walk you through, you can level up the functionality
of your robot, or get totally creative with some hacks of your own. So if you want to enhance
or even just take the first step of unlocking the really fun
and rewarding hobby of making stuff, just go to CrunchLabs.com,
or use the link in the video description where we're giving away one free box
as an early subscriber special. and back over with Jigsaw Things were looking very promising
for the world record attempt of an off the shelf puzzle, completely solved
and assembled by a robot with no human intervention. Some pieces would go right in Uno Wow Some required a little bit of wiggle. Oh, actually, yeah. And some required a lot of wiggle But in the end. let’s go Jigsaw C’mon last piece Jigsaw always came through Yes!! Now feeling more confident in Jigsaw’s abilities It was time for him to meet the greatest
jigsaw puzzler the human race had to offer What? Are you done? Who, fun fact also happens to be a national sudoku champion Tammy McLeod I’m the Human Benchmark
according to my child I first wanted her to face off against the best jigsaw puzzler
I personally know my dear friend Kristen Bell aka Anna, or Veronica mars,
or Eleanor, or a bunch more Kristen ever since I told you I was making a puzzle
robot, you've been talking so much trash. A little bit of trash. Yeah. The puzzle robot isn't here today. So instead, I have a friend
I want you to... challenge. This is Tammy. She holds the Guinness World Record for the fastest puzzle Ever solved by a human being. Hi, Kristen. We kick things off
with a simple 30 piece puzzle. And to be honest, Three, two, one. Go! Kristen started off pretty strong. Where's the elephant? Tammy, you don't talk
while you're doing it? I mean, I could trash talk, but- Do it! You're gonna lose Tammy. And then you hit me back. Wow, that was one minute
and five seconds for a 30 piece puzzle. I'm sweating so much and that was a pretty decisive win. But I wanted to see one more face off, only this time with a 500 piece puzzle and a huge advantage in Kristen's favor. I think I'm going to help you on this one. Okay? And with Kristen's renewed confidence,
the clock started and out of the gate Tammy was so confident
in this 2-v-1 matchup. She's just doing flipping? Oh my god, that's crazy organized. She saw no problem in helping our cause. What is that little guy? his tail She’s right SHE’S RIGHT Mark, can I ask you a sincere question? Is this the robot? Admittedly, I did have an
ulterior motive for this matchup and that was espionage. Tammy, I noticed you're using the box. What! Tammy, get real. You have to look at the picture. It’ll just distract me. I look at the pictures. I have good color acuity. I have very bad shape memory. So if you say had to go against an all
white puzzle... That would maybe be my Achilles heel. And this was a very useful
bit of information. But she didn't stop there. Because she also told me the four tips she tells amateur jigsaw puzzlers
to help them solve puzzles faster. First off, dump out all the pieces and turn them over
so you can see them all at the same time. For number two most people start with the edge pieces, and that's not a bad move,
but it's not always the best. For example,
if there's a puzzle that has something like a gold frame around the outside you should set all the edge pieces aside and then leave them till the end,
where there's a lot more information from the interior pieces
to help with the border For tip three look for groupings that catch your eye
and organize them into piles. So for example, you could organize
by colors, textures, or patterns. But either way, you're trying
to reduce the search space into smaller chunks because scanning and pattern matching
are different parts of the brain. And this way you don't have to do them
at the same time, which makes it easier. And finally, if you're down to a bunch of pieces
that all look the same like a blue sky, sort by shape based on
how many ins and outs they have and orient them
all the same way. This will leave you with six total piles and will make it much easier
to find the final matches. And I needed to look no further
than our current match up- Oh no. I look away for one minute. Don't look at Tammy's, Kristen Things have gotten dire. -to understand there must be some validity
to these four steps. I’m not seeing your hands move
or your eyes look at the board I’M PANICKING I'm not trying to- DONE!? 34 minutes and 2 seconds. 500 pieces. Congratulations. Congratulations. Your next puzzle
match will not be against mere humans. And indeed, after setting things up
at CrunchLabs, we were ready to go. Tammy, I'd like to officially introduce
you to jigsaw. This is my upgrade from Kristen And Jigsaw was in no mood for
conversation as he got straight to work. All right, then! As Tammy entered
the matchup of her life with the weight of all humanity
resting on her capable shoulders. What's your strategy here? The only shape I can make use of
is the edge. Okay, how are we doing, jigsaw? not really sure what
I'm supposed to do now... But that’s when I remember I have
a specialized skill set of my own I might need to take measurements
on your neck later. I'm making you something. That's why you don't have any pieces
placed yet on your team. Wow, Tammy you're familiar with the saying
“measure once, cut twice.” No, that's not it. And as if on cue, My teammate, wrapped up his measuring. There's a corner piece! and began cutting. did you have to make him so noisy? Three pieces! Four pieces! Tammy, ignore me... Saying five pieces! It ain’t over till it's over. You're still winning, Tammy.
Don't let it get to your head. And while I was tempted
to count all 1000 Nine pieces! I decided to take up woodworking instead. Are you nervous, Tammy? Even if I lost today,
my family would still love me. Jigsaw. If he lost today,
would you still love him? No My love is very conditional. Having properly reminded jigsaw
what was at stake. Jigsaw has now officially taken the lead. Jigsaw continued to make steady progress
while Tammy... Where’s that last edge? Oh, you need one edge piece faced the limits inherent in biology I really want to
actually help find this for you That way I can say you needed my help. Without the precise search capabilities... I found it, I’m done with the edge! Of a smart phone to laptop nervous system. Click.
I love that sound. Is there any way I could get some water. Jigsaw. Do you need a water break? You notice we are missing
a piece in the middle that's on purpose I was the ideas guy
and jigsaw was the muscle. Jigsaw. That's your next piece
right there buddy. I'm a team player. The only thing is he could be a bit of a control freak So I opted to help Tammy instead. Tammy, do you need this piece?
Do you need this piece? Do you need this piece? Do you need thi– And so while jigsaw continued
to widen the gap I made myself useful by woodworking. Solitairing Finding Waldo. Tammy right here! Nah I’ll look later and just generally reverting
to the role of my youth as the annoying youngest sibling. I love doing puzzles And after all that,
Tammy had made a lot of progress- Tammy, I have good news and bad news. The bad news? There's only eight pieces left -but not enough. The good news is I made you a scarf! In fact, if you assume it
takes one second per side to try all the remaining
combination of pieces she had left working 12 hours a day, skipping weekends. It would take her one
and a half months to finish which made this a good time to throw in the towel. Unless you're about to finish I think I'm good. Oh, this is a moment! 1000 pieces in four hours. Okay, Tammy,
you represented us humans well, But alas, it’s time
we welcome our benevolent robot overlords. It's an honor to lose to him That was beautiful. I gotta say! Good work, jigsaw. At ease So while jigsaw took his victory lap I put the final touches on the trophy I'd carved her. You are the best
the human race has to offer. And you did a great job. Just, you know, not that great if you're a teenager or adult and you've always wanted
to make and build cool stuff but just have it figured out that first step This is it. It's called the CrunchLabs Hack Pack and it's basically a series of really fun programable robots that get
delivered right to your door. Where we build it together and learn, step
by step, the kinds of engineering skills that go into making the builds
on my YouTube channel, and they all work
with no programing required. But since my goal is to take you
from wherever you're at and level you up, you can easily hack
the microcontroller brains of any of these robots in a bunch of ways
to completely level up the functionality. There's also a community where you could share your builds
or post questions, as well as an AI chat bot named Mark Robot
that will check your code for you and help you implement
your most creative ideas. On top of all that, each box has a chance
to contain a platinum diploma. If you're box has it, congratulations
because college is now free for you or a loved one
you want to transfer it to plus you get to come out here and brainstorm one of your own ideas with me
and my team for a day. So if you want to enhance
or even just take the first step of unlocking the really fun
and rewarding hobby of making stuff, just go to CrunchLabs.com or use the link in the video description
where to say thank you We're giving away that free box
as an early subscriber special Thanks for watching.
