[Chatbot]
Write a short introduction for Geoffrey Hinton, the subject of this video. Geoffrey Hinton is a University of Toronto professor emeritus who is
known as the "Godfather of AI." He recently left Google so he could
more freely discuss the dangers posed by unchecked AI development. We spoke to him in his London home
about the technology he helped create, its many benefits and why he
suddenly fears humanity is at risk. [Geoffrey Hinton] I got a request from the Wall Street Journal. They want me to correct my obituary. [Christopher Sorensen]
What do you mean? [Hinton]
They want me to correct my obituary! [Sorensen]
They've like pre-written it, right? [Hinton]
They've pre-written my obituary. <laughter> I wonder what Mark Train
would've said about that. [Sorensen]
So. I guess we don't really need an introduction here
so I will just launch right into it. You've recently given a
number of interviews in which you've said
that digital intelligence that is used by chatbots
and other generative AI may be a better kind of intelligence
than the biological intelligence that we have. Can you briefly explain what
made you come to this conclusion? [Hinton]
So, in a digital computer it's designed so
you can tell it exactly what to do and it'll do exactly what you tell it. And even when it's learning stuff, two different digital computers can do
exactly the same thing with the same learned knowledge. And that means that you could
make 10,000 copies of the same knowledge, have them all running
on different computers and whenever one copy
learns something, it can communicate it very
efficiently to all the other copies. So you can have 10,000
digital agents out there, a kind of hive mind, and they can share knowledge extremely
efficiently by just sharing the connection strengths inside the
neural nets. And we can't do that. If you learn something and
you want to tell me about it, you have to use sentences or pictures, and you can only share a very limited
amount of information that way. So it's much, much slower for you to communicate
what you've learned to me than it is for these digital intelligences
to communicate stuff. And that makes them much better. They can learn a whole
lot of stuff between them. [Sorensen]
You've said that digital intelligence is immortal and that biological
intelligence is mortal. What did you mean by this? [Hinton]
So if I learn some connection strengths in a neural net that's being
simulated on digital computers, then if a particular computer dies, those same connection strengths
can be used on another computer. And even if all the
digital computers died, if you'd stored the
connection strength somewhere, you could then just make another digital
computer and run the same weights on that other digital computer. But with us, the knowledge that we learn,
the connection strengths, are specific to our particular brains.
Every brain is a bit different. The neurons in your brain are all a
bit different and you learn so as to make use of all the idiosyncrasies
of your particular brain. And so once you've learned
connection strengths in your brain, if you told me those
connection strengths, they wouldn't do me any good
because my brain's different. So the digital computers are immortal
because you can run that same knowledge on a different piece of hardware. We are mortal because the hardware and
the knowledge are intricately entangled. You can't separate the connection strengths from
the particular brain they're running in. And so if the brain dies,
the knowledge dies. [Chatbot]
Why should we be concerned about digital intelligence taking over
from biological intelligence? [Hinton]
Because I think it's much better at sharing what's learned by a whole
bunch of different digital agents who all share the same weights and
they just share the updates to the weights, and now they can learn 10,000
different things at the same time. But also I think the digital intelligence
probably has a better learning algorithm than the brain's got. All the attempts to find a learning algorithm
in the brain that works as well as the
backpropagation algorithm in these digital intelligences —
so far those attempts have failed. We haven't found anything that scales
up as well to very large systems as the backpropagation algorithm. So I think they've got
two advantages. They've probably got a better learning
algorithm and they can share knowledge much more efficiently than
biological intelligences can. [Sorensen]
At the time when you entered the field
there were two schools of thought on
machine intelligence: mainstream and neural nets. Can you describe the difference
between these two approaches? [Hinton]
I can sort of caricature it. So there's two different models of
what intelligence is all about and one model is that it's all about reasoning
and the way we reason is by using logic. And so that's what's
special about people. And what we should be doing is understanding the kind of logic
that we actually use. And that also went with
the idea that the knowledge you store is
symbolic expressions — so that I can say a sentence to you and you will somehow
store that and then later you'll be able to use it for
inferring other sentences. But what's inside your head is something
a bit like sentences but cleaned up. And there's a completely different model
of intelligence which is that it's all about learning the connection
strengths in a network of brain cells. And what it's good for is things
like perception and motor control, not for reasoning.
That reasoning came much, much later and we are
not very good at it. You don't learn to do it
until you're quite old. And so reasoning is actually a very
bad model of biological intelligence. Biological intelligence is
about things like controlling your body
and seeing things, and that was a totally different paradigm
and had a different idea of what's in inside your head — that it's not stored strings of symbols,
it's just connection strengths. In the symbolic AI view, the crucial question was: What is the form of these
symbolic expressions and how do you do the
reasoning with them? For the neural net view, the central question was
quite different. It was how do you learn these connection
strengths so you can do all these wonderful things? And so learning was
always central to the neural net view. For the symbolic view, they said,
"We'll worry about learning later." "First you have to figure out how the
knowledge is represented" "and how we reason with it." And so these were
totally different views. One took its inspiration from
logic and one from biology. And for a long time the people in the
logic camp thought taking your inspiration from biology was silly. That was a bit strange
since von Neumann and Turing had both thought neural nets were
the way to attack intelligence. But unfortunately they
both died young. [Chatbot]
Can you at a high level describe how a neural network works? [Hinton]
I can try. So let's start off by describing
how it would work for recognizing objects and images. And let's suppose all we wanted to do was say whether or not
there was a bird in the image. And let's suppose a bird's gonna be
roughly in the middle of the image and the main object of attention and you have
to say, "Is this a bird or isn't it?" So you can think of an image. Let's suppose it's a hundred pixels by
a hundred pixels. That's 10,000 pixels. Each pixel is three colors:
RGB — so that's 30,000 numbers. And in computational terms, recognizing a bird in an image
consists of taking 30,000 numbers and outputting one number that
says yes or no, it's a bird. And you could try and write a
standard computer program to do that. And people tried for many, many years, and they could never get it
to work very well. Like for 50 years they were trying to do that. Or you could make a multilayer
neural net. And I'll start off by telling you how
you would wire up a neural net by hand. So what you'd do is you'd
have the pixels and that will be the bottom level. And then you'd have a
layer of feature detectors, and a typical feature detector
might have big positive connection strengths coming from a vertical row
of pixels and big negative connection strengths coming from a
neighbouring vertical row of pixels, and no connection strengths
anywhere else. So if both rows of pixels are bright, it'll get big positive input from here,
but also big negative input from there. So it won't do anything. But if these ones are bright,
giving a big positive input, and these ones are not bright,
so it doesn't get inhibited by these ones, it will get all excited and it will
say, "Hey, I found the thing I like," "which is bright pixels here
and dark pixels here." And that's an edge detector. I just told you how to wire up by hand
using positive and negative weights, something that would detect
a little vertical edge. So now imagine you have a gazillion of
those guys detecting different edges in different locations in the image, in different orientations
and at different scales. That will be your first-layer
feature detectors. Now if I was wiring it by hand my
second-layer of feature detectors, I would maybe have a detector
that takes two edges of joint at a fine angle like this. So it's
looking for this edge and this edge. And if they're both active
at once it would say, "Hey, maybe there's a beak here." It could be all sorts of other things
but it might just be a beak. So you have a
feature that's sort of beak-like. You might also have a feature that detects a whole
bunch of edges in the form of a circle. And so you'd have circle detectors
and potential beak detectors, as well as lots of other
detectors in that layer. But they're detecting slightly
more complicated things. And then in the layer above that you
might have something that detects a potential beak in the right spatial
relationship to a potential circle, a potential eye, so that it
could be the head of a bird. So that would be like your third layer. And maybe if in your third layer you
also got something that detected the foot of a bird and the wing of a bird, then maybe in the next layer you could
have a bird detector - that if several of
those things got active, like OK, here's a head and there's a wing and
there's a foot, it probably is a bird. Ok, so I told you how to wire all those things
up by hand but you'd never be able to do a very good job of it. So, instead of wiring it all up by hand, we
could imagine trying to learn it all. So I've told you the kind
of thing we want to learn, but now I'll tell you how we learn it and the way we learn it
sounds bizarre at first. Instead of wiring in all the connection
strength so you get the detectors you want, you start with random connection
strengths — just random numbers
on all the connections. And so you put in an image of a bird and
you go forward through these layers of feature detectors and it just
behaves completely randomly. And the bird detector at the
output will say, 0.5 it's a bird. It's gonna say one when it's sure it's
a bird and zero and it's sure it's not a bird, so it's going to say about 0.5. And now you can ask
the following question: how can I change all those
connection strengths in the network, so instead of saying 0.5 it's a bird —
let's suppose it is a bird - it says 0.51 it's a bird. So the question you want to ask is: how should I change a particular
connection strength so as to make it more likely that it's a bird? And you can figure that out by
taking the difference between what you got and what you wanted. So you
wanted one and you actually got 0.5. You take that difference and you send that difference
backwards through the network. And then you use some
calculus, which I won't explain. And you are able to compute for every
single connection in the network how much you'd like to make it
bigger or smaller in order to make it more likely to say "bird." Then you adjust all the connection
strengths very slightly in the direction that are making
it more likely to say "bird." Then you show it something
that isn't a bird and now you're gonna adjust
connection strengths so it's less likely
to say that that was a bird. And you just keep going like that
with lots of birds and non-birds, and eventually you'll discover that
it's discovered all these feature detectors - it'll have discovered
beak-like things and eye-like things, and things that detect feet
and wings and all that stuff. And if you train it on
lots of different objects, like a thousand different
categories of object, it'll discover intermediate
feature detectors that are very good for
recognizing all sorts of things. So the magic is that there's this
relatively simple algorithm called backpropagation that takes the error
in the output and sends that error backwards through the network and computes
through all the connections how you should change them to
improve the behavior, and then you change them all a tiny bit and you just keep going
with another example. And surprisingly
that actually works. For many years people thought that would
just get jammed up — it would get stuck somewhere — but no it doesn't,
it actually works very well. [Chatbot]
I'm curious, how do neural
networks handle language? [Hinton]
OK, so now you've got the idea of how we train it to recognize a bird. Imagine now that we take a string of words as the input. The first thing you are going do is
convert a word into an embedding vector — that is, it's a little bunch of numbers
that captures the meaning of the word, or is intended to capture
the meaning of the word. And so your first layer
after the words will be these embedding vectors
for each word. And now we're gonna have lots of layers
of embedding vectors and as we go up through the network, we're gonna make the embedding vectors
for a word get better and better because they're going to take into account
more and more contextual information. So suppose in this sentence — let's
suppose we don't have any capital letters, OK? So suppose in this sentence
you have the word "may." Well, the most probable meaning of "may" is
that it's a modal as in, "He may do that." But obviously there's a completely
different meaning of "may," which is the month. And so initially it doesn't know, just looking at the word "may," it
doesn't know what embedding vector to use. And it'll use a kind of compromise vector, something that's sort of halfway between
the embedding vector that represents the modal "may" and the embedding
vector that represents the month "May." And then at the next layer
it's going refine that vector. It's going to make a
slightly better vector depending on the context that it got,
depending on nearby embedding vectors. So if, for example, nearby there's the
embedding vector for "June," then it'll refine the one for "may" to
be more like a month and less like a modal. But if there's the embedding vector for
"would," it'll make it more like a modal and less like a month. And as you go
through the network, it can refine these embedding vectors
and make them better and better. And the way we are going to train it, is we're going to give it
a string of words as input and we are going to — here
will be one way to do it ... it's not exactly what's done but it's
easy to understand — for the last word, you just put in a kind of neutral word. You say "unknown" and it has a very
vague embedding vector that's kind of the average of all the vectors for all
words. It doesn't know, right? Now, as you go forward
through the network, that last word will be able to
be influenced by previous words, and it starts off very vague, but as you go through these layers
it can get more and more precise. And by the time you get to
the end of the network that embedding vector could look like the
embedding vector for a particular word or for some combination
of words — some average of several words. And you train
the network by saying, "You go through all these layers and
that last word you'd like the embedding vector to look like the embedding vector
for the word that actually was there in the text." And that's how
it predicts the next word. It tries to change this sort of neutral
embedding vector into one that is close to the embedding vector for the
correct word that appeared in the text. And you take the error, the difference between the
embedding vector in the text and the embedding vector produced, and you propagate that backwards through
the network — and it's propagating
backwards through the layers. but it's propagating from this word to
previous words so that they will have the right influence on this word. And that's the backpropagation algorithm
learning to predict the next word. [Sorensen]
So despite some of the theoretical
breakthroughs in this field, these neural networks didn't work very
well for a long time. And why was that? [Hinton]
It was a combination of reasons. So we weren't very good at
initializing them — that, as I said, you put in random weights
and then learn everything. But if you don't carefully decide
what kind of random weights, the thing never gets off the ground. So that was a little technical reason
why they didn't work very well in deep nets with lots of layers
of feature detectors. But the main reason was we didn't have
enough compute power and we didn't have enough data. So people were trying to train these
nets on relatively small training sets without much compute power. And in
that regime other methods work better. Neural nets really come into their own
when you have a lot of data and a lot of compute power, and then you can use a big neural net
and then it works much better than anything else. And we didn't realize that at the time.
So we would occasionally fantasize, "Well, suppose you had a lot more data and
a lot bigger in computer it will work better." But we didn't realize
it will work a whole lot better. And so in the 1990s, it was a relatively dead period for
neural nets because other methods were working better on small problems. And a lot of people in computer
science gave up on neural nets. In psychology they didn't,
because in psychology they wanted something that was
like the brain and neural nets were clearly more like the brain
than symbolic AI. But in computer science,
neural nets sort of came
into disrepute in the 90s. [Sorensen]
So let's fast forward then to another decade, to the 2000s. Was there a moment for you when it
became clear that the approach that you'd been pursuing was the one
that was going to prevail? [Hinton]
OK. In 2006, we figured out how to initialize the
weights much better by doing unsupervised learning and then backpropagation
worked much better. So it was fairly clear then the backpropagation really
was going to work very well. But in 2009, two of my grad students — George
Dahl and Abdelrahman Mohamed — made a much better speech recognizer, actually a slightly
better speech recognizer, but it was slightly better than the
state of the art, using deep neural nets. And then it was fairly clear that
this stuff was going somewhere. And all the big speech groups over the
next few years switched to using neural nets. And then in 2012, that speech stuff
came out in the Android and suddenly the Android caught up with Siri. It was as good at speech as
Siri because it was using neural nets. And in the same year two others
of my graduate students — Ilya Sutskever and Alex Krizhevsky — made a neural net that
was very good at recognizing objects and images and that beat the
state of the art by a lot. And so I think it was this combination
that it was already working for speech recognition and already in production
— the big companies knew that, but the public I don't think were very
well aware of that — but then suddenly it worked much
better for computer vision and that was a turning point. In 2012, when we won the ImageNet
competition by a huge margin, we got almost half the errors of
the other methods and it was a public data set but with a hidden
test set, so you couldn't cheat. [Sorensen]
So let's just focus a bit on 2012 because you said it was a
really pivotal year for this. Can you describe again at
a high level how AlexNet worked? I take it that might have been
named after your graduate student. [Hinton]
That was named after Alex Krizhevsky because he did ... he was a
wizard programmer and he made it work. Ilya helped a lot but it
was mainly Alex's work. So I explained to you when explaining
backprop how you'd have these layers of feature detectors and AlexNet
was basically that kind of a net but with a thousand different
object classes and with about seven layers of feature detectors. And it also used something else
that was developed by Yann LeCun, which is convolutional nets. And I'll try and explain those now
because they were very important. Remember how I said you might make
a detector for a bird's beak by checking two lines ...
by having two lines like that? And if you see those two feature
detectors then you make a beak detector? But that would just be for a specific
location right? In a convolutional net, when you make a feature
detector for one location, you make the same feature detector
for all the locations in the image. So now if it's trained with a
beak here when it's learning and it really says, "I need a
beak detector for that" — so, it learns a feature that detects this
beak — it will automatically make copies for all of the other
locations in the image. So if now the bird occurs
in a different location, it will have the feature
detectors to recognize it. So that idea that you copy the
feature detectors to every location, that's a convolutional net essentially. And that makes the whole thing
generalized much better across position. It can cope now with things changing
position because it's got copies of all these feature detectors in every location. And with convolutional nets
and multiple layers of features, what Alex did was programmed all that
very efficiently on a thing called a graphics processing unit, which was
developed for computer graphics. But it's like a mini super computer, it can do lots and lots of computation
in lots of separate processs all at the same time. And so it gave us about a
factor of 30 compared with a normal computer and a factor of 30 is about sort
of 10 years progressing in computers. So suddenly we could leap 10 years
into the future in terms of compute power. And it was very difficult to program
these GPU boards. Alex managed to
program two of them to collaborate, which was even more difficult. And the last ingredient
was the ImageNet dataset. So someone called Fei-Fei Li and her
collaborators put together a big set of images and then a public competition
where you had about a million images with a thousand different
kinds of objects. So you had about a thousand examples of
each kind of object and you had to learn to recognize those objects. And then the test set
would be different images, which also contain those objects. And so you'd have to generalize
to the different images. And it turned out the best computer
vision technique that had been invented up until then was getting
like 25 per cent errors and Alex got 15 per cent errors. And since then it's gone down
to about 3 per cent errors. It's got much better since then. But it was a huge jump and people in
computer vision were extremely surprised and most of them behaved in a very
admirable way, which is they said, "Hey, we never thought this would work," but hey it works so we're gonna do that
instead of what we were doing." That's what scientists don't usually do. Scientists usually just grow old complaining that
this new stuff is nonsense. [Sorensen]
And how would you describe the pace of innovation that we've
seen in AI since that moment? [Hinton]
It's just got faster and faster. So if you'd asked me in that moment
how long until these neural nets can do machine translation that's
better than the state of the art, I'd have said maybe 10 years. Because machine translation
is the kind of thing that, if you've got a theory that's all
about processing strings of symbols, machine translation is the ideal problem
for you because you have a string of symbols in one language and you have to produce a string
of symbols in another language. And the symbolic people thought, well inside you're just
manipulating strings to do that. The neural net people thought you
have to take this string of symbols, you have to convert it into these big
patterns of neural activity and then you have to convert it back
into symbols at the output. And I was very surprised when it only
took a few years for machine translation to be good and then in another year or
two Google was using it and it greatly improved the quality
of machine translation. Like in languages like Chinese —
this is from memory, but ... — there was a gap between how good the
computer translation was and how good human translation was.
And it just halved that gap overnight. I think it was Chinese that it did that, but in a lot of languages it just
made it a lot better. And since then, it's obviously it's got
considerably better since then. But by 2015 it was already working pretty
well and that really surprised me that it only took three years. [Chatbot]
You say you were surprised at the pace of innovation. What did you think the first time you
used a large language model like ChatGPT? Did we surprise you? [Hinton]
I am just shocked at how ...
how good it is. So it gives very coherent
answers and it can do little bits of reasoning — not very
sophisticated reasoning yet, although it'll get much
better. So, for example, I asked it — this is
GPT-4 now — I asked it a puzzle given to me by a symbolic AI guy
who thought it wouldn't be able to do it. I actually made the puzzle much
harder and it could still do it. And so the puzzle goes like this: the rooms in my house are
either white or blue or yellow. Yellow paint fades to white within a year. In two years time, I would like all the
rooms to be white, what should I do? And a human being will probably say
you should paint the blue rooms white. What GPT-4 said was you should paint the
blue rooms yellow — but that works too because yellow will fade to white. And I don't see how it could do that
without understanding the problem. The idea that it's just sort of
predicting the next word and using statistics — there's a sense in which that's true, but it's not the sense of statistics
that most people understand. It, from the data, it figures out how to extract the
meaning of the sentence and it uses the meaning of the sentence
to predict the next word. It really does understand
and that's quite shocking. [Sorensen]
So have you been surprised by the broader reaction,
the public reaction, to ChatGPT? Well given how well it works, I guess the
public reaction isn't that surprising. But what's interesting is,
most people don't say, "This doesn't understand." They say, "Wow, it understood what I
said and gave me a coherent answer." "What can I use it for?" And I think most people are right about
that and of course it can be used for huge numbers of things. So I know someone who answers letters
of complaint for the health service, and he used to spend 25 minutes composing
a letter that addresses the problem. And so now he just types the problem into GPT-4 and it writes the letter and
then he just looks at the letter and decides if it's OK and sends it out.
And that takes him five minutes now. So he is now five times more efficient and that's gonna
happen all over the place. Like paralegals are
gonna be like that. Programmers are already
getting like that. Programmers can be much more efficient
if they get assistance from things like GPT-4 because it
knows how to program. And you might think it just knows how
to program because he's seen a whole lot of programs. So I have a former graduate student who's
very smart and a very good programmer and he did a little experiment which
is — he's called Radford Neal — he took GPT-4 and he defined a new programming
language with very unusual syntax, and having defined this programming
language just in text to GPT-4, he then gave it a program and said, "What would this do?"
And it answered correctly. So basically it could understand the
definition of a new programming language and figure out what programs in
that language would do. And again, the idea that it's just predicting
the next word doesn't make any sense
in that context. It had to understand
what was going on. [Sorensen]
So what do you see as some of the most promising opportunities for this
type of AI when it comes to benefiting society? [Hinton]
It's hard to pick one because there's so many — Like there'll be a huge increase in productivity for any
job that involves outputting text. There's all sorts of issues about
increasing productivity. In our society, it's not necessarily a good
thing to increase productivity because it might make the
rich richer and the poor poorer. But in a decent society, just increasing productivity
ought to be a good thing. So there'll be things like that. It's wonderful for making predictions. It'll be better at predicting the weather. People don't know by how much yet, but it's already much
better at predicting floods. It can predict earthquakes.
It can design new nanomaterials. So for things like solar panels, you want to be able to design new
nanomaterials, or for superconductivity. I don't know if it's used
for superconductivity yet, but it may well be — you'd
like that at high temperature. It's really good at
designing drugs — that is, finding molecules that'll bind to
some particular other molecule. DeepMind has used it to create AlphaFold. Now that's not a chatbot.
That's just deep learning. But the basic technology
of deep learning has pretty much solved the problem
of how you figure out from the string of bases in a protein,
what shape it will adopt. And if you know what shape it
adopts, you know its function. The chatbots are just gonna
be used everywhere I think. [Sorensen]
And we've also talked a lot about health care. I mean, you
talked about drug discovery, but health care is another field
that could really benefit. [Hinton]
Yes. Both in interpreting medical scans ... Like if you take a CAT scan, there's a lot of information in
the CAT scan that isn't being used and most doctors don't know
what the information is. This will be able to get much more out
of a CAT scan as well as being able to compete with doctors at saying what kind of cancer you have
or how big it's grown. At present for example, when a doctor tells you
the size of a cancer, you'll get a number like it's
three centimetres and a month ago it
was two centimetres. Now that's not a very useful number
if the thing looks like an octopus, right? A neural net will be
able to do much better at understanding the volume of the
cancer and how it's changed. So it's going be tremendous there and
already it's at the level of humans for lots of kinds of scans and it's going
to get better. It's going to be very good
for diagnosing diseases. So at present, there's a large number of people who
die in North America because the doctors misdiagnosed what they had. There's a system that Google's
producing called Med-PaLM 2 which has learned to do diagnoses
and it's already ... I think it's better than
an average doctor now. I'm not quite sure about this because
I'm not at Google anymore and it's very recent, but it's certainly comparable with
doctors and it's going to get better fast. So wouldn't you like to have a
sort of general practitioner family doctor, and you go with some rare
disease and you'd love your family doctor to have already seen hundreds of
cases of that rare disease and Med-PaLM 2 is going to be like that. So it's going to be just, in the end,
much better at diagnosis. [Chatbot]
It sounds like AI will bring many important benefits, but you have expressed concern about
the current pace of innovation. Why? [Hinton]
OK, so for like 50 years I thought that — well for 49 years —
in order to make digital models better, we needed to make them
work more like the brain. So I kept looking at things the brain
does and the digital models don't like rapidly changing connection
strengths in a temporary way, and that can make the
digital models better. And very recently I realized that because
these digital models have this kind of hive mind where when
one agent learns something, all the other agents know it, they might actually already be
better than biological intelligence. And so I kind of completely flipped my
opinion from the idea it's going to be a long time before they can do everything
the brain does — that it's gonna be 30 to 50 years
before they're better than us, which is what I thought for until
very recently. A few months ago, I suddenly realized maybe
they're already better than us. They're just smaller and
when they get bigger then they'll be smarter than us. And that was quite scary. It was a sudden change of opinion
that instead of being 30 to 50 years, it was five years to 20 years, or something like that — and so we needed now to
take really seriously right now what we're going to do about the issue
that these things may become smarter than us. It's a time of huge uncertainty. Nobody really
knows what's going to happen. Maybe things will stall and maybe
they won't become smarter than us, but I don't really believe that. I think
they're going to be smarter than us. But maybe when they
become smarter than us, we'll be able to keep them benevolent
and we will we'll be able to keep them caring much more about people than they
care about themselves, unlike people. But maybe not. And so we need to start thinking very
hard about those issues and I'm not an expert on those issues. I'm just an expert on these
learning algorithms and I suddenly realized these superintelligences
may be here quite soon — and I'm just sounding the alarm so that people
listen to the experts who've been thinking for a long time about how
we might stop them taking control. I want the politicians to listen to
those guys rather than say, "Yeah, yeah, they're sort of sci-fi guys, they're
... it's never gonna happen.". [Sorensen]
Was there like a particular moment when you had this —
you said it was very recent — where you kind of
changed your view on it? I was developing learning algorithms
for biological systems that could run in a biological system, which didn't use backpropagation. And I couldn't make them work as well
as a backpropagation algorithm that we were running in these digital systems. And they would work for small networks. But when I scaled it up, the digital ones always scaled up
much better than the biological ones. And suddenly I thought
it might not be my fault. It might not be that my learning algorithm
was just a bad learning algorithm. It might be that these digital
systems just are better. And that's when I suddenly changed
my mind about how long before we get superintelligence. And then I talked to various
former students of mine, and former colleagues of mine, and some of them encouraged me
to go public with this — not because I had any solutions
that I wanted to recommend. It's not like you can say, "Burn less carbon and
everything will be fine." But because they thought I'm well
known in the field and if I go public by saying superintelligence
might be here quite soon, the politicians might start to believe
that's a possibility and start listening seriously to the researchers who've
been thinking a long time about how we prevent these things from gaining control. [Sorensen]
So from your point of view, what can ... what role can governments
play in helping ensure AI are developed in a responsible way? [Hinton]
So there's all sorts of risks other people have talked about a lot — and that I don't particularly
want to talk about — like they'll take jobs away and increase the gap between
the rich and the poor; they will make it impossible to
know whether the news is fake or real; they will encourage society
to divide into two warring camps that don't listen to each other
and have completely opposing views; they will build battle robots
that are designed to kill people. All of those are well known
risks that I'm not talking about. It's not that I don't
think they're important, I think they're probably
even more urgent, but lots of other people are
talking about those risks. The risk I'm talking about is the risk
that these things will get smarter than us and eventually take over. And for that risk there may be
something governments can do because nobody wants that ... well, if you exclude these super
intelligences. No people want that. And so all the different
governments ought to be able to agree ... they ought to be able to work together
on preventing that because it's in their interests. And that's happened
before, even during the Cold War. The U.S. and Russia could work together
on trying to prevent there being a global nuclear war because
it was so bad for everybody, and for this existential threat, it should be possible for everybody
to work together to limit it if it's possible to prevent it. I don't know whether it's
possible to prevent it, but at least we should be able to get
international collaboration on that particular threat: the existential
threat of AI taking over. One thing I think should be done
is wherever this stuff's being developed, particularly
these big chatbots, governments should encourage the
companies to put a lot of resources, as these things are getting
more and more intelligent, to doing experiments to figure out how
to keep them under control. So they should be sort of looking at how these
things might try and escape and doing empirical work on that, and put a lot of resources into that
because that's the only chance we've got. Before they're ... before
they're super intelligent, we can maybe do experiments and
see what's going to go wrong. And I'm strongly of the belief
you need empirical data on this. You just can't have philosophers
and politicians and legislatures making up rules. You need empirical work looking at these
things and seeing how they go wrong and seeing how you might control them. And that can only be done by
the people developing them. So since you can't stop the development, the best you can do is somehow have
governments put a lot of pressure on these companies to put a lot of
resources into investigating empirically how to
keep them under control when they're not
quite as smart as us. [Sorensen]
And what do you see as the role of these big technology companies where a lot of this development
is happening? Would they do this without that
kind of government regulation? [Hinton]
So a lot of the people in the big companies — all the people I know who are senior in the
big companies are very worried about this and do put work into that. They're very concerned about it, but they have an obligation to
their shareholders, and I think, to make big profits — and making big profits, particularly
in the short term, doesn't align nicely with putting a lot
of effort into making sure it's safe. So you see this in all industries.
In the railway industry in the States, having safety devices that tell you
when a wheel's locked costs money and the big rail companies just
rather have accidents than do that. Google, which is a big company
I know something about, is not quite like that because it
understands that it's going to have a tremendous reputational
loss if bad things happen. And that's why Google didn't release
these chatbots. It kept them private. It didn't want them out there in
the world for people to play with. It wanted to use them to
give you better search results or to complete your Gmail for you, but not to give them
to people to play with. And it could only be responsible like
that until OpenAI and Microsoft put them out there and then Google had to compete. But the big people in the big companies
really care a lot about their reputation and about not having bad effects, but they could maybe be made to care
even more about this safety issue by government doing something to insist
that they put a lot of work into that. And there's other things
that could happen. Like it's very hard within a company
to have people working on long-term existential threats
because they're paid by the company and there's a conflict of interest, which is one of
the reasons I left Google — not because Google did anything wrong because I just don't
want any conflict of interest. One thing the big companies
could certainly do is put more money into funding
foundations that study these things and Google, for example,
put $300 million into a foundation called Anthropic that is studying these things. They could put a lot more money in. [Sorensen]
I'm curious about what advice you would give,
or what guidance you would give, to other researchers in the field — you know, who might be just
entering the field right now and want make sure that they're advancing the field but doing
it in a responsible way? [Hinton]
Well, one piece of advice I'd give is look at how many people are
working on making these things better and how many people are working
on preventing them from getting out of control. And you'll see it's like 99 people are
working on making them better and one person's working on preventing them
getting out of control. So where could you
make the most impact? Probably on working in preventing
them getting out of control. So that's one piece of advice. The other piece of advice is my
general advice for young researchers, which is look for
somewhere where you think everybody's doing it wrong and trust your intuition until you figure
out why your intuition is incorrect. Trust it and work on alternatives ... alternative ways of doing
things when you think everybody else is doing it wrong. And the fact is either you
have good intuitions or you don't. If you've got good intuitions, you should listen to them and follow
your intuition and work on that until you discover why it's wrong. If you've got bad intuitions, it doesn't really matter what you do. So you might as well
follow your intuitions. [Chatbot]
The risks you've described are alarming, but can't you just throw
a switch and shut it down? Aren't humans ultimately still in control? [Hinton]
It's very tempting to think we could just turn it off. Imagine these things are a lot smarter
than us and remember they'll have read everything Machiavelli ever wrote. They'll have read every example in
the literature of human deception. They'll be real experts at doing human
deception because they'll have learned that from us, and they'll
be much better than us. They'll be like you
manipulating a toddler. You know, you say to your toddler, "Do you want peas or cauliflower?" And
your toddler doesn't realize actually he doesn't have to have either. He just thinks which he likes and which he dislikes the most
and says he'll have the other one. So if they can manipulate people, they can manipulate people into
pressing buttons and pulling levers. So we have a nice
example of Donald Trump. Donald Trump can manipulate people
and so he could invade a building in Washington without ever
going there himself. And you didn't have to prevent Donald
Trump from doing anything physical. You'd had to prevent him from talking
to prevent that. And these are chatbots. So the idea that just with talk, they can't do any real damage because
it requires people to do the damage — well, as soon as you
can manipulate people, then you can get
whatever you like done. [Chatbot]
You've spent your career trying to understand how the human brain works and played a critical role in AI development.
What's next for you, Geoffrey Hinton? [Hinton]
OK, so I'm 75 and I've reached the point where I'm not
very good at writing programs anymore because I keep forgetting the
names of the variables I'm using and things like that. And I forget to, I do a copy and paste and forget
to modify the thing I pasted. And so I've slowed down a lot in
programming and it's very irritating. It's extremely irritating not
to be as good as you used to be. And I decided a long time ago
that when I reached that point I would become a philosopher. And so I'm going to
become a philosopher.
