Transcriber: Andreas Nawrath
Reviewer: Elisabeth Buffard What's on your mind? AI researchers spend their days playing Sherlock Holmes or maybe Watson, sorting out this great detective puzzle: what's on you mind? What's in there? And to solve this,
we can't build human minds so we build artificial minds. Minds that live in the server racks. Minds made out of code and bits. I'm an AI researcher,
and I spend my time wondering how we reason about what we see. What's in our mind's eye? Today I am going to talk about
seeing and thinking. And I want to start with a puzzle and because it's gonna be
about seeing and thinking, it's gonna be a visual puzzle. And I'd like for you to help me
solve this puzzle, if you please, so get ready,
here we go, let's solve this puzzle together.
Are you ready? All right! Which one of these things is the odd one? I didn't say it was a hard puzzle... (laughter) How many think this is a hard puzzle?
Come on! All right! Pretty easy, right? Think about it. And while you are thinking about it,
let's consider another one. Which one of these does not belong? Now this one is a little harder, at least it is for me! And the conversation that you're having in your head or maybe with your neighbour although I don't see you talking
with your neighbour so much right now, is, you know, maybe, how many
of this thing or that colour are present? To me as an AI researcher, that conversation, that inner dialogue, that's fascinating! It's fascinating! Because from the day we're born we're visual reasoners. And your very first memory, your first memory,
even though you can't recall it were born in the light
of that first image that you saw. And from the day we're born, we take in this world that we are in and we try to interpretate it. The toys we play with, the jumble of blocks
that we see, we manipulate and suddenly, somehow,
we are able to create things that we only have seen in our mind's eye. That early play sets the stage for kindergarten and our entry into simple math and beyond. Although frankly sometimes I think that I encouter things
where I might have missed a lesson or two. The point is, our minds and our eyes work together to make meaning
all the time. We see things we're not aware of,
inside, outside, we take in this wonderful visual world somehow we identify things so you might say that seeing affects how you think. But you know the opposite is true, too. How we are thinking affects
what we are able to see. Now consider faces. Most of you have a face, I think! I see some of you. Faces are so poweful a notion, that we see them even when
what we are looking at is a car's dashboard. Faces pop out. Everywhere I love that. I like that sad face. They pop out! It's almost like you cannot
not see a face! Why? Seeing and thinking interfere and support one another. Think about all of the scenes
that you see. Kitchens, and the sidewalks, crowds, people and all of it. If we're gonna build artifical minds, what can they see? What should they see? And this question keeps us up at night when we work in AI. What should be in the mind? And sure, you see scarves and people, and cats and kitchens
and everything like that. But just because you see them, does that mean that you think about them? And here is sort of the rub. Just because we see shapes, does that mean that we think in shapes? Just because we see geometry, must we think... about geometry? We've only had geometry
in front of our eyes for the last couple of thousand years. And you know, our minds and our eyes were evolving for a lot longer than that. You know, we have walked in our mind through the savannas and the forest
in comparison forever. And our eyes and our minds
have evolved to see scenes like this! And the question when you are building
an artificial mind is "Oh look, if we evolve
to see things like this, is there a power that you possess, is there something that goes on for you that gives you the ability
to see these scenes and think about them? What is that power? Maybe that power underlies
the way you think overall. I happen to think that there is not
ONE power, but maybe all of us have
like 2 super powers. A vision. And I'll describe them. And the first of those super powers is the ability to notice novelty. I bet you are an expert
hunting easter eggs. I bet you are. But have you thought about
how complicated that task is? How did you notice the eggs in the grass? Was it the colour? Was it the shape? What about it? Did you... how should you be
able to find the eggs and be that expert? Noticing novelty requires us
to compare things and the scenes that you see. Take a look at this scene. If you look closely, at this scene, you might notice there are 2 children here. But if you look closer, you recognize, that somebody's forgot his hat. Comparing parts of a scene
that you are seeing is a part of noticing novelty. But, it's not just about the same scene, it's about your expectations, too - right? It's about the things
that you saw in the past the things you thought about in the past. That comes into play with novelty as well. And so, let me say that when our great great great grandmothers... great great great great great great
great grandmothers... looked out on scenes like this, they happened somehow to be able to see the predators lurking
in this visual jumble. I'd like to thank grandma, for not becoming lunch. Novelty. Novelty is this power that let's us do
the mundane things like picking out lunch, or not becoming lunch. That's novelty, that's one of our powers. The other one has to do with our ability to delve into exactly
the right level of detail, no matter what we're seeing. The power of abstraction. Consider this great painting by Van Gogh. Starry night. If you zoom in, the closer you get, the more the sutble colour choices
of Van Gogh pop out at us. And each brush stroke, the vitality comes true. And even when we zoom all the way back, we can still appreciate how this whole painting hangs on those very fine details. But abstraction has another meaning, too. I'm pretty familiar with this scene. And I bet you are, too. The rush hour, check it out. But look closely. The most powerful,
largest image in this scene is the light pole. And you edited it away, automatically and so did I. Why? It doesn't matter to the scene. It doesn't make sense
in the meaning of the scene. Abstraction tells our minds how to
think about the things we are seeing. It gives us the ability
to zoom in or zoom out to exactly the right level of detail, and edit away the non-essential. I think when we are building
these minds of ours, these new minds, we ought to give them
those superpowers, too. But seeing and thinking is not necessarily
into there, either, because, if you think about it, we don't want to give our machines
the ability to remember every blasted thing they have ever seen, I don't think you do it,
I don't think I do it either. But there is something more at play. Perhaps this ability has to do with picking out just the right features
to think about when you're thinking about a scene. But how do you choose the features? Colour, texture, or what, what is it? Our eyes of the mind that
we are going to build, inhabit this messy,
complicated visual world, and they gotta be able to notice
the visual bits in this jumble and pick out the novelty and do so
at exactly the right level of abstraction. How? In order to solve that part of it, I need to talk a little bit now of fractals. Now when I say the word fractals,
you are probably thinking about this. A computer generated image of fractals. You know, the father of fractals,
Mandelbrot, wasn't thinking that way, he was trying to describe the world
that we all live in, the world that is complicated. [unclear] our world. The math behind fractals is very complex, but there really are two observations
that are at the core, two simple observations. And the first is that there are
similar repeated patterns everywhere you look. And I don't mean, you know, similar and repeated patterns
in a wallpaper sort of sense, What I mean instead is, in a very natural sense. The objects that we live with repeat themselves. People, chairs, cars, things. They're similar but they're not the same. Any scene at all, man-made,
computer-generated, natural scenes, even the one you're looking at right now, we can think of as a host of patterns. The second observation is, that the similarity occurs
at different scales. Sunflowers. What is it about this field of sunflowers this image of sunflowers that lets us recognize that
the large sunflowers in the front are the same as the small ones
in the back? We're agents in the world, we walk closer and further away. And perspective gives us similarity
at different scales. But nature does it too. This is a cauliflower. To me this looks alien. But check it out, if you zoom in, on part of this cauliflower, you'll see that each smaller piece
looks like the larger. This image even though it's photographed, looks computer-generated. Even if you look at something
as humble as a leaf, there are patterns to be discovered and similarity at different scales. If there's a fractal formula
for these images, what's the fractal formula
for the real world, the world we live in,
the scenes we see? One fall, a mathematican was walking, and he looked down,
he saw this maple leaf lying there, and as he saw this leaf,
he thought about it, he realized that this maple leaf
seemed to exhibit sort of a secret code. The maple leaf looked like
it was made out of copies of itself. In fact, it looks like it is made of
three copies of the same maple leaf and if you composite them together,
you kinda get to find a maple leaf collage
that looks a lot like a maple leaf. And it was in that one moment,
and in that one leaf that he came up with the answer that this is actually is the fractual formula
for the real world. Copying parts of the world
into smaller pieces. And you can write that code down. And so, when we are building
artificial minds, what we want to do
is we want to build them not from a framework of geometry, but we want to build them in a way
where they can actually handle the world as the world is. Mandelbrot's world. And that is how I began
to build the minds that I make. Not with geometry, but with fractals. Giving that machine and that code the ability to notice novelty. And to zoom in and zoom out. And I began to present to the code
puzzles. And the machine began to solve them. Without the aid of gemoetry. But with fractals. Spoiler alert. Did anybody guess this one? I presented the code 3,000 such problems. And it solved most of them
and it did so in a way, that most of the humans
who took the test didn't. Now, AI researchers have long tested
their programs with human tests of intelligence. Not always with
the best of outcomes, frankly. But as it turns out,
the most widely-administered test of human intelligence is a visual test. It's the Raven's test. The idea is, of a set of choices, pick the one that best fills in the blank. And when I gave this particular
series of tests to the program, it did pretty well. One of those tests used to be used
as an admission criteria for Mensa. And if it still were,
it would just about qualify. Let me bring this back together for us. In no time at all, you and I are gonna live in a world
filled with AI. Filled with it. We're gonna have smart artifacts, robot companions... And if you think you are enchanted
with your iPhone now, just wait. And here is my thinking, that if we give all those artificial minds the ability to have our super powers, the ability to notice novelty, and to zoom in and zoom out
with abstraction, and we train them to see
the world for what it is, this great, grand, beautiful, Mandelbrot fractal world, then our new companions, these new minds, will be able to see the world and think about the world, in the same way you and I do. Together we will share it. Thank you! (applause)
