The following content is
provided under a Creative Commons license. Your support will help
MIT OpenCourseWare continue to offer high-quality
educational resources for free. To make a donation or to
view additional materials from hundreds of MIT courses,
visit MIT OpenCourseWare at ocw.mit.edu. [MUSIC PLAYING] PATRICK H. WINSTON: Well,
what we're going to do today is climb a pretty big
mountain because we're going to go from a
neural net with two parameters to discussing
the kind of neural nets in which people end up dealing
with 60 million parameters. So it's going to be
a pretty big jump. Along the way are
a couple things I wanted to underscore from
our previous discussion. Last time, I tried to
develop some intuition for the kinds of formulas
that you use to actually do the calculations in a
small neural net about how the weights are going to change. And the main thing
I tried to emphasize is that when you have a
neural net like this one, everything is sort of
divided in each column. You can't have the performance
based on this output affect some weight
change back here without going through this
finite number of output variables, the y1s. And by the way, there's no y2
and y4-- there's no y2 and y3. Dealing with this is really
a notational nightmare, and I spent a lot
of time yesterday trying to clean it
up a little bit. But basically, what
I'm trying to say has nothing to do with
the notation I have used but rather with the
fact that there's a limited number of ways in
which that can influence this, even though the number of
paths through this network can be growing exponential. So those equations
underneath are equations that derive
from trying to figure out how the output performance
depends on some of these weights back here. And what I've calculated
is I've calculated the dependence of
the performance on w1 going that way, and
I've also calculated the dependence of performance
on w1 going that way. So that's one of the
equations I've got down there. And another one
deals with w3, and it involves going both
this way and this way. And all I've done in both
cases, in all four cases, is just take the partial
derivative of performance with respect to those weights
and use the chain rule to expand it. And when I do that,
this is the stuff I get. And that's just a whole
bunch of partial derivatives. But if you look at it and let
it sing a little bit to you, what you see is that
there's a lot of redundancy in the computation. So for example, this
guy here, partial of performance
with respect to w1, depends on both
paths, of course. But look at the first elements
here, these guys right here. And look at the first
elements in the expression for calculating the partial
derivative of performance with respect to w3, these guys. They're the same. And not only that, if you
look inside these expressions and look at this
particular piece here, you see that that is
an expression that was needed in order
to calculate one of the downstream weights,
the changes in one of the downstream weights. But it happens to be the same
thing as you see over here. And likewise, this piece is the
same thing you see over here. So each time you move
further and further back from the outputs
toward the inputs, you're reusing a
lot of computation that you've already done. So I'm trying to find a
way to sloganize this, and what I've come up with is
what's done is done and cannot be-- no, no. That's not quite right, is it? It's what's computed is computed
and need not be recomputed. OK? So that's what's going on here. And that's why this is
a calculation that's linear in the depths of the
neural net, not exponential. There's another thing I wanted
to point out in connection with these neural nets. And that has to do
with what happens when we look at a single neuron
and note that what we've got is we've got a bunch of
weights that you multiply times a bunch of inputs like so. And then those are all
summed up in a summing box before they enter some kind
of non-linearity, in our case a sigmoid function. But if I ask you to write down
the expression for the value we've got there, what is it? Well, it's just the sum
of the w's times the x's. What's that? That's the dot product. Remember a few lectures
ago I said that some of us believe that the dot product is
a fundamental calculation that takes place in our heads? So this is why we think so. If neural nets are doing
anything like this, then there's a dot product
between some weights and some input values. Now, it's a funny
kind of dot product because in the models
that we've been using, these input variables are
all or none, or 0 or 1. But that's OK. I have it on good
authority that there are neurons in our head
for which the values that are produced are not
exactly all or none but rather have a kind of
proportionality to them. So you get a real dot product
type of operation out of that. So that's by way of
a couple of asides that I wanted to
underscore before we get into the center
of today's discussion, which will be to talk about
the so-called deep nets. Now, let's see,
what's a deep net do? Well, from last time, you
know that a deep net does that sort of thing, and
it's interesting to look at some of the offerings here. By the way, how good was
this performance in 2012? Well, it turned out
that the fraction of the time that the
system had the right answer in its top five
choices was about 15%. And the fraction of the time
that it got exactly the right answer as its top pick
was about 37%-- error, 15% error if you count it as
an error if it's-- what am I saying? You got it right if you
got it in the top five. An error rate on that
calculation, about 15%. If you say you only get it right
if it was your top choice, then the error rate was about 37%. So pretty good, especially
since some of these things are highly ambiguous even to us. And what kind of
a system did that? Well, it wasn't one
that looked exactly like that, although that
is the essence of it. The system actually
looked like that. There's quite a lot
of stuff in there. And what I'm going to talk about
is not exactly this system, but I'm going to talk about the
stuff of which such systems are made because there's
nothing particularly special about this. It just happens to be
a particular assembly of components that tend to
reappear when anyone does this sort of neural net stuff. So let me explain that this way. First thing I need to talk
about is the concept of-- well, I don't like the term. It's called convolution. I don't like the term because
in the second-best course at the Institute,
Signals and Systems, you learn about impulse
responses and convolution integrals and stuff like that. And this hints at that,
but it's not the same thing because there's no memory
involved in what's going on as these signals are processed. But they call it convolutional
neural nets anyway. So here you are. You got some kind of image. And even with lots of computing
power and GPUs and all that sort of stuff, we're
not talking about images with 4 million pixels. We're talking about images
that might be 256 on a side. As I say, we're not
talking about images that are 1,000 by 1,000 or 4,000
by 4,000 or anything like that. They tend to be
kind of compressed into a 256-by-256 image. And now what we do
is we run over this with a neuron that
is looking only at a 10-by-10 square like so,
and that produces an output. And next, we went
over that again having shifted this neuron
a little bit like so. And then the next thing we do
is we shift it again, so we get that output right there. So each of those deployments
of a neuron produces an output, and that output is associated
with a particular place in the image. This is the process that
is called convolution as a term of art. Now, this guy, or this
convolution operation, results in a bunch
of points over here. And the next thing that
we do with those points is we look in
local neighborhoods and see what the
maximum value is. And then we take
that maximum value and construct yet another
mapping of the image over here using
that maximum value. Then we slide that over like so,
and we produce another value. And then we slide
that over one more time with a different
color, and now we've got yet another value. So this process
is called pooling. And because we're
taking the maximum, this particular kind of
pooling is called max pooling. So now let's see what's next. This is taking a
particular neuron and running it across the image. We call that a kernel, again
sucking some terminology out of Signals and Systems. But now what we're
going to do is we're going to say we could
use a whole bunch of kernels. So the thing that I
produce with one kernel can now be repeated
many times like so. In fact, a typical
number is 100 times. So now what we've got is
we've got a 256-by-256 image. We've gone over it
with a 10-by-10 kernel. We have taken the
maximum values that are in the vicinity
of each other, and then we repeated
that 100 times. So now we can take that, and
we can feed all those results into some kind of neural net. And then we can, through
perhaps a fully-connected job on the final layers of this, and
then in the ultimate output we get some sort of
indication of how likely it is that the thing that's
being seen is, say, a mite. So that's roughly how
these things work. So what have we
talked about so far? We've talked about pooling, and
we've talked about convolution. And now we can talk about
some of the good stuff. But before I get into that,
this is what we can do now, and you can compare this with
what was done in the old days. It was done in the old
days before massive amounts of computing became available
is a kind of neural net activity that's a little easier to see. You might, in the old days,
only have enough computing power to deal with a small
grid of picture elements, or so-called pixels. And then each of these might be
a value that is fed as an input into some kind of neuron. And so you might have a column
of neurons that are looking at these pixels in your image. And then there might be
a small number of columns that follow from that. And finally, something
that says this neuron is looking for things that are
a number 1, that is to say, something that looks like
a number 1 in the image. So this stuff up
here is what you can do when you have a
massive amount of computation relative to the
kind of thing you used to see in the old days. So what's different? Well, what's
different is instead of a few hundred parameters,
we've got a lot more. Instead of 10 digits,
we have 1,000 classes. Instead of a few
hundred samples, we have maybe 1,000
examples of each class. So that makes a million samples. And we got 60 million
parameters to play with. And the surprising thing
is that the net result is we've got a function
approximator that astonishes everybody. And no one quite
knows why it works, except that when you throw an
immense amount of computation into this kind of
arrangement, it's possible to get a performance
that no one expected would be possible. So that's sort of
the bottom line. But now there are a couple of
ideas beyond that that I think are especially interesting,
and I want to talk about those. First idea that's
especially interesting is the idea of
autocoding, and here's how the idea of
autocoding works. I'm going to run
out of board space, so I think I'll
do it right here. You have some input values. They go into a layer of
neurons, the input layer. Then there is a so-called hidden
layer that's much smaller. So maybe in the example,
there will be 10 neurons here and just a couple here. And then these expand to
an output layer like so. Now we can take the output
layer, z1 through zn, and compare it with the
desired values, d1 through dn. You following me so far? Now, the trick is to say, well,
what are the desired values? Let's let the desired
values be the input values. So what we're going
to do is we're going to train this net
up so that the output's the same as the input. What's the good of that? Well, we're going to
force it down through this [? neck-down ?]
piece of network. So if this network
is going to succeed in taking all the possibilities
here and cramming them into this smaller inner layer,
the so-called hidden layer, such that it can reproduce
the input [? at ?] the output, it must be doing some
kind of generalization of the kinds of things
it sees on its input. And that's a very clever idea,
and it's seen in various forms in a large fraction
of the papers that appear on deep neural nets. But now I want to
talk about an example so I can show you
a demonstration. OK? So we don't have GPUs, and
we don't have three days to do this. So I'm going to make up a
very simple example that's reminiscent of what goes
on here but involves hardly any computation. What I'm going to
imagine is we're trying to recognize
animals from how tall they are from the shadows
that they cast. So we're going to recognize
three animals, a cheetah, a zebra, and a giraffe, and
they will each cast a shadow on the blackboard like me. No vampire involved here. And what we're
going to do is we're going to use the shadow as
an input to a neural net. All right? So let's see how
that would work. So there is our network. And if I just clicked into
one of these test samples, that's the height of the shadow
that a cheetah casts on a wall. And there are 10 input
neurons corresponding to each level of the shadow. They're rammed through
three inner layer neurons, and from that it spreads out and
becomes the outer layer values. And we're going to
compare those outer layer values to the desired values,
but the desired values are the same as
the input values. So this column is a
column of input values. On the far right, we have
our column of desired values. And we haven't trained
this neural net yet. All we've got is
random values in there. So if we run the test samples
through, we get that and that. Yeah, cheetahs are short,
zebras are medium height, and giraffes are tall. But our output is just pretty
much 0.5 for all of them, for all of those shadow
heights, all right, [? with ?] no training so far. So let's run this thing. We're just using simple
[? backdrop, ?] just like on our world's simplest neural net. And it's interesting
to see what happens. You see all those
values changing? Now, I need to mention that
when you see a green connection, that means it's a
positive weight, and the density of the green
indicates how positive it is. And the red ones are
negative weights, and the intensity of the
red indicates how red it is. So here you can
see that we still have from our random
inputs a variety of red and green values. We haven't really
done much training, so everything correctly
looks pretty much random. So let's run this thing. And after only 1,000 iterations
going through these examples and trying to make the
output the same as the input, we reached a point where
the error rate has dropped. In fact, it's
dropped so much it's interesting to relook
at the test cases. So here's a test case
where we have a cheetah. And now the output
value is, in fact, very close to the desired value
in all the output neurons. So if we look at
another one, once again, there's a correspondence
in the right two columns. And if we look at the
final one, yeah, there's a correspondence in
the right two columns. Now, you back up from
this and say, well, what's going on here? It turns out that you're
not training this thing to classify animals. You're training it to understand
the nature of the things that it sees in the
environment because all it sees is the height of a shadow. It doesn't know anything
about the classifications you're going to try
to get out of that. All it sees is that there's
a kind of consistency in the kind of data that it
sees on the input values. Right? Now, you might say,
OK, oh, that's cool, because what must
be happening is that that hidden layer,
because everything is forced through that narrow
pipe, must be doing some kind of generalization. So it ought to be the
case that if we click on each of those
neurons, we ought to see it specialize
to a particular height, because that's the sort of stuff
that's presented on the input. Well, let's go see
what, in fact, is the maximum
stimulation to be seen on the neurons in
that hidden layer. So when I click on these
guys, what we're going to see is the input values
that maximally stimulate that neuron. And by the way, I
have no idea how this is going to turn out
because the initialization's all random. Well, that's good. That one looks like
it's generalized the notion of short. Ugh, that doesn't
look like medium. And in fact, the
maximum stimulation doesn't involve any stimulation
from that lower neuron. Here, look at this one. That doesn't look like tall. So we got one that looks
like short and two that just look completely random. So in fact, maybe we
better back off the idea that what's going on
in that hidden layer is generalization
and say that what is going on in there
is maybe the encoding of a generalization. It doesn't look like
an encoding we can see, but there is a generalization
that's-- let me start that over. We don't see the generalization
in the stimulating values. What we have instead
is we have some kind of encoded generalization. And because we got
this stuff encoded, it's what makes these neural
nets so extraordinarily difficult to understand. We don't understand
what they're doing. We don't understand why they
can recognize a cheetah. We don't understand why
it can recognize a school bus in some cases,
but not in others, because we don't
really understand what these neurons
are responding to. Well, that's not quite true. There's been a lot
of work recently on trying to sort that
out, but it's still a lot of mystery in this world. In any event, that's
the autocoding idea. It comes in various guises. Sometimes people talk about
Boltzmann machines and things of that sort. But it's basically all
the same sort of idea. And so what you can
do is layer by layer. Once you've trained
the input layer, then you can use that layer
to train the next layer, and then that can train
the next layer after that. And it's only at the very, very
end that you say to yourself, well, now I've accumulated
a lot of knowledge about the environment and what
can be seen in the environment. Maybe it's time to
get around to using some samples of particular
classes and train on classes. So that's the story
on autocoding. Now, the next thing to talk
about is that final layer. So let's see what the final
layer might look like. Let's see, it might
look like this. There's a [? summer. ?]
There's a minus 1 up here. No. Let's see, there's a
minus 1 up-- [INAUDIBLE]. There's a minus 1 up there. There's a multiplier here. And there's a
threshold value there. Now, likewise, there's some
other input values here. Let me call this one x, and it
gets multiplied by some weight. And then that goes into
the [? summer ?] as well. And that, in turn, goes into
a sigmoid that looks like so. And finally, you get an
output, which we'll z. So it's clear that if you
just write out the value of z as it depends on those inputs
using the formula that we worked with last
time, then what you see is that z is
equal to 1 over 1 plus e to the minus w times
x minus T-- plus T, I guess. Right? So that's a sigmoid
function that depends on the
value of that weight and on the value
of that threshold. So let's look at how those
values might change things. So here we have an
ordinary sigmoid. And what happens if we shift
it with a threshold value? If we change that
threshold value, then it's going
to shift the place where that sigmoid comes down. So a change in T
could cause this thing to shift over that way. And if we change
the value of w, that could change how
steep this guy is. So we might think that the
performance, since it depends on w and T, should be
adjusted in such a way as to make the classification
do the right thing. But what's the right thing? Well, that depends on the
samples that we've seen. Suppose, for example, that
this is our sigmoid function. And we see some examples of a
class, some positive examples of a class, that
have values that lie at that point and
that point and that point. And we have some values that
correspond to situations where the class is not one of the
things that are associated with this neuron. And in that case, what
we see is examples that are over in this vicinity here. So the probability that we
would see this particular guy in this world is associated with
the value on the sigmoid curve. So you could think of
this as the probability of that positive
example, and this is the probability of
that positive example, and this is the probability
of that positive example. What's the probability
of this negative example? Well, it's 1 minus the
value on that curve. And this one's 1 minus
the value on that curve. So we could go through
the calculations. And what we would determine
is that to maximize the probability of seeing this
data, this particular stuff in a set of experiments, to
maximize that probability, we would have to adjust T and
w so as to get this curve doing the optimal thing. And there's nothing
mysterious about it. It's just more
partial derivatives and that sort of thing. But the bottom line is that the
probability of seeing this data is dependent on the
shape of this curve, and the shape of this curve is
dependent on those parameters. And if we wanted to maximize
the probability that we've seen this data, then we have
to adjust those parameters accordingly. Let's have a look
at a demonstration. OK. So there's an ordinary
sigmoid curve. Here are a couple of
positive examples. Here's a negative example. Let's put in some more
positive examples over here. And now let's run the good,
old gradient ascent algorithm on that. And this is what happens. You've seen how the
probability, as we adjust the shape of the curve,
the probability of seeing those examples of
the class goes up, and the probability of seeing
the non-example goes down. So what if we put
some more examples in? If we put a negative
example there, not much is going to happen. What would happen if we put a
positive example right there? Then we're going to start
seeing some dramatic shifts in the shape of the curve. So that's probably
a noise point. But we can put some more
negative examples in there and see how that
adjusts the curve. All right. So that's what we're doing. We're viewing this
output value as something that's related to the
probability of seeing a class. And we're adjusting the
parameters on that output layer so as to maximize the
probability of the sample data that we've got at hand. Right? Now, there's one more thing. Because see what
we've got here is we've got the basic idea
of back propagation, which has layers and layers
of additional-- let me be flattering and call
them ideas layered on top. So here's the next idea
that's layered on top. So we've got an
output value here. And it's a function after
all, and it's got a value. And if we have
1,000 classes, we're going to have 1,000
output neurons, and each is going to be
producing some kind of value. And we can think of that
value as a probability. But I didn't want to
write a probability yet. I just want to say
that what we've got for this output neuron
is a function of class 1. And then there will be
another output neuron, which is a function of class 2. And these values will
be presumably higher-- this will be higher if we are,
in fact, looking at class 1. And this one down here
will be, in fact, higher if we're looking at class m. So what we would like to do
is we'd like to not just pick one of these outputs
and say, well, you've got the highest
value, so you win. What we want to do
instead is we want to associate some
kind of probability with each of the classes. Because, after all,
we want to do things like find the most
probable five. So what we do is
we say, all right, so the actual
probability of class 1 is equal to the output of
that sigmoid function divided by the sum over all functions. So that takes all of
that entire output vector and converts each output
value into a probability. So when we used that
sigmoid function, we did it with the
view toward thinking about that as a probability. And in fact, we assumed
it was a probability when we made this argument. But in the end,
there's an output for each of those classes. And so what we get is, in the
end, not exactly a probability until we divide by a
normalizing factor. So this, by the way, is called--
not on my list of things, but it soon will be. Since we're not talking
about taking the maximum and using that to classify the
picture, what we're going to do is we're going to use
what's called softmax. So we're going to give a
range of classifications, and we're going to associate
a probability with each. And that's what you saw
in all of those samples. You saw, yes, this is
[? containership, ?] but maybe it's also this,
that, or a third, or fourth, and fifth thing. So that is a pretty good
summary of the kinds of things that are involved. But now we've got one more
step, because what we can do now is we can take this output
layer idea, this softmax idea, and we can put them together
with the autocoding idea. So we've trained
just a layer up. And now we're going to detach
it from the output layer but retain those
weights that connect the input to the hidden layer. And when we do that,
what we're going to see is something that
looks like this. And now we've got a
trained first layer but an untrained output layer. We're going to freeze
the input layer and train the output layer
using the sigmoid curve and see what happens
when we do that. Oh, by the way, let's run
our test samples through. You can see it's
not doing anything, and the output is half
for each of the categories even though we've got
a trained middle layer. So we have to train
the outer layer. Let's see how long it takes. Whoa, that was pretty fast. Now there's an extraordinarily
good match between the outputs and the desired outputs. So that's the combination
of the autocoding idea and the softmax idea. [? There's ?] just one more
idea that's worthy of mention, and that's the idea of dropout. The plague of any neural
net is that it gets stuck in some kind of local maximum. So it was discovered
that these things train better if, on every
iteration, you flip a coin for each neuron. And if the coin
ends up tails, you assume it's just died and has
no influence on the output. It's called dropping
out those neurons. And in our next iteration,
you drop out a different set. So what this seems
to do is it seems to prevent this thing from going
into a frozen local maximum state. So that's deep nets. They should be called, by the
way, wide nets because they tend to be enormously
wide but rarely more than 10 columns deep. Now, let's see, where
to go from here? Maybe what we should do is talk
about the awesome curiosity in the current state of the art. And that is that
all of [? this ?] sophistication with output
layers that are probabilities and training using autocoding
or Boltzmann machines, it doesn't seem to help much
relative to plain, old back propagation. So back propagation
with a convolutional net seems to do just about
as good as anything. And while we're on the subject
of an ordinary deep net, I'd like to examine
a situation here where we have a deep net--
well, it's a classroom deep net. And we'll will put
five layers in there, and its job is still
to do the same thing. It's to classify an animal as a
cheetah, a zebra, or a giraffe based on the height of
the shadow it casts. And as before, if it's
green, that means positive. If it's red, that
means negative. And right at the moment,
we have no training. So if we run our
test samples through, the output is always a 1/2
no matter what the animal is. All right? So what we're
going to do is just going to use ordinary back
prop on this, same thing as in that sample that's
underneath the blackboard. Only now we've got a
lot more parameters. We've got five columns,
and each one of them has 9 or 10 neurons in it. So let's let this one run. Now, look at that
stuff on the right. It's all turned red. At first I thought this
was a bug in my program. But that makes absolute sense. If you don't know what the
actual animal is going to be and there are a whole
bunch of possibilities, you better just say
no for everybody. It's like when a biologist
says, we don't know. It's the most probable answer. Well, but eventually, after
about 160,000 iterations, it seems to have got it. Let's run the test
samples through. Now it's doing great. Let's do it again just to
see if this is a fluke. And all red on the right
side, and finally, you start seeing some changes go
in the final layers there. And if you look at the error
rate down at the bottom, you'll see that it kind
of falls off a cliff. So nothing happens
for a real long time, and then it falls off a cliff. Now, what would happen if
this neural net were not quite so wide? Good question. But before we get to that
question, what I'm going to do is I'm going to do a
funny kind of variation on the theme of dropout. What I'm going to
do is I'm going to kill off one
neuron in each column, and then see if I can
retrain the network to do the right thing. So I'm going to reassign
those to some other purpose. So now there's one fewer
neuron in the network. If we rerun that, we see that
it trains itself up very fast. So we seem to be
still close enough to a solution we
can do without one of the neurons in each column. Let's do it again. Now it goes up a little
bit, but it quickly falls down to a solution. Try again. Quickly falls down
to a solution. Oh, my god, how much of
this am I going to do? Each time I knock
something out and retrain, it finds its solution very fast. Whoa, I got it all the way down
to two neurons in each column, and it still has a solution. It's interesting,
don't you think? But let's repeat the
experiment, but this time we're going to do it a
little differently. We're going to take
our five layers, and before we do
any training I'm going to knock out all but
two neurons in each column. Now, I know that with two
neurons in each column, I've got a solution. I just showed it. I just showed one. But let's run it this way. It looks like
increasingly bad news. What's happened is that
this sucker's got itself into a local maximum. So now you can see
why there's been a breakthrough in this
neural net learning stuff. And it's because when
you widen the net, you turn local maxima
into saddle points. So now it's got a way
of crawling its way through this vast
space without getting stuck on a local maximum,
as suggested by this. All right. So those are some, I
think, interesting things to look at by way of
these demonstrations. But now I'd like to go
back to my slide set and show you some
examples that will address the question of whether these
things are seeing like we see. So you can try these
examples online. There are a variety
of websites that allow you to put in your own picture. And there's a cottage industry
of producing papers in journals that fool neural nets. So in this case, a very
small number of pixels have been changed. You don't see the
difference, but it's enough to take this
particular neural net from a high confidence that
it's looking at a school bus to thinking that it's
not a school bus. Those are some things that
it thinks are a school bus. So it appears to be
the case that what is triggering this
school bus result is that it's seeing enough
local evidence that this is not one of the other 999 classes
and enough positive evidence from these local
looks to conclude that it's a school bus. So do you see any
of those things? I don't. And here you can say, OK, well,
look at that baseball one. Yeah, that looks like it's got
a little bit of baseball texture in it. So maybe what it's doing
is looking at texture. These are some examples from
a recent and very famous paper by Google using
essentially the same ideas to put captions on pictures. So this, by the way,
is what has stimulated all this enormous concern
about artificial intelligence. Because a naive viewer looks
at that picture and says, oh, my god, this
thing knows what it's like to play, or be young,
or move, or what a Frisbee is. And of course, it
knows none of that. It just knows how to
label this picture. And to the credit of the
people who wrote this paper, they show examples
that don't do so well. So yeah, it's a cat,
but it's not lying. Oh, it's a little girl, but
she's not blowing bubbles. What about this one? [LAUGHTER] So we've been doing our
own work in my laboratory on some of this. And the way the following set of
pictures was produced was this. You take an image,
and you separate it into a bunch of slices,
each representing a particular frequency band. And then you go into one
of those frequency bands and you knock out a
rectangle from the picture, and then you
reassemble the thing. And if you hadn't
knocked that piece out, when you reassemble it,
it would look exactly like it did when you started. So what we're doing is we
knock out as much as we can and still retain the
neural net's impression that it's the thing that it
started out thinking it was. So what do you think this is? It's identified by a neural
net as a railroad car because this is the image
that it started with. How about this one? That's easy, right? That's a guitar. We weren't able to mutilate that
one very much and still retain the guitar-ness of it. How about this one? AUDIENCE: A lamp? PATRICK H. WINSTON: What's that? AUDIENCE: Lamp. PATRICK H. WINSTON: What? AUDIENCE: Lamp. PATRICK H. WINSTON: A lamp. Any other ideas? AUDIENCE: [INAUDIBLE]. AUDIENCE: [INAUDIBLE]. PATRICK H. WINSTON: Ken,
what do you think it is? AUDIENCE: A toilet. PATRICK H. WINSTON: See, he's
an expert on this subject. [LAUGHTER] It was identified as a barbell. What's that? AUDIENCE: [INAUDIBLE]. PATRICK H. WINSTON: A what? AUDIENCE: Cello. PATRICK H. WINSTON: Cello. You didn't see the little
girl or the instructor. How about this one? AUDIENCE: [INAUDIBLE]. PATRICK H. WINSTON: What? AUDIENCE: [INAUDIBLE]. PATRICK H. WINSTON: No. AUDIENCE: [INAUDIBLE]. PATRICK H. WINSTON:
It's a grasshopper. What's this? AUDIENCE: A wolf. PATRICK H. WINSTON:
Wow, you're good. It's actually not
a two-headed wolf. [LAUGHTER] It's two wolves that
are close together. AUDIENCE: [INAUDIBLE]. PATRICK H. WINSTON:
That's a bird, right? AUDIENCE: [INAUDIBLE]. PATRICK H. WINSTON:
Good for you. It's a rabbit. [LAUGHTER] How about that? [? AUDIENCE: Giraffe. ?] PATRICK H. WINSTON:
Russian wolfhound. AUDIENCE: [INAUDIBLE]. PATRICK H. WINSTON: If
you've been to Venice, you recognize this. AUDIENCE: [INAUDIBLE]. PATRICK H. WINSTON:
So bottom line is that these things
are an engineering marvel and do great things,
but they don't see like we see.
