JOHN BRACAGLIA: Hello,
my name is John Bracaglia and I work at Verily, which is
Google's life sciences company. I also lead a group called
The Singularity Network, which is an internal
organization composed of more than 3,000
Googlers, focused on topics about the future
of artificial intelligence, for which we are here today. And it's my pleasure to be here
today with Dr. Max Tegmark. As a brief introduction,
Max Tegmark is a renowned scientific
communicator and cosmologist, and has accepted
donations from Elon Musk to investigate the
existential risk of advanced artificial
intelligence. His research interests include
consciousness, the multiverse, advanced risk from
AI, and formulating an ultimate ensemble
theory of everything. Max was elected fellow of
the American Physics Society in 2012, won "Science"
magazine's Breakthrough of the Year in 2003,
and has written over 200 publications,
nine of which have been cited
more than 500 times. Max Tegmark, everyone. [APPLAUSE] MAX TEGMARK: Thank you so much. It's a really great honor
to be back here at Google and to get to talk in front
of so many old friends, and so much human-level
intelligence and idealism. Does anyone recognize? NASA EMPLOYEE: 20
seconds and counting-- MAX TEGMARK: This
was, of course, the Apollo 11 moon
mission that put Neil Armstrong, Buzz Aldrin,
and Michael Collins on the moon. NASA EMPLOYEE: Ten, nine, MAX TEGMARK: This-- NASA EMPLOYEE: Ignition
sequence starts. MAX TEGMARK: --mission
was not only successful, but I think it was very
inspiring because it showed that when we humans
manage technology wisely, we can do things that our
ancestors could only dream of, right? Now there are some
important lessons I think we can learn from
this, as well, so I want to devote the rest of this talk
to another journey, powered by something much more
powerful than rocket engines, where the passengers are
not just three astronauts, but all of humanity. So let's talk about
our collective journey into the future with AI. My friend Jaan Tallinn
likes to emphasize that just as with
rocketry, it's not enough to just make our
technology powerful. We also have to focus on
figuring out how to control it and on figuring out where
we want to go with it. And that's what we're
going to talk about. I think the opportunities
are just so awesome, if we get this right. During the past
13.8 billion years, our universe has transformed
from dead and boring to complex and interesting,
and it has the opportunity to get dramatically more
interesting in the future if we don't screw up. About 4 billion years ago, life
first appeared here on Earth, but it was pretty dumb
stuff, like bacteria, that couldn't really learn
anything in their lifetime. I call that life 1.0. We are what I call life 2.0
because we can learn things. Which, of course,
in geek speak means we can upload new
software modules. If I want to learn Spanish,
I can study Spanish, and now I have all these new
skills uploaded in my mind. And it's precisely
this ability of humans to design their own software
rather than be stuck with what the software evolution gave
us, which has enabled us to dominate this
Earth, give us what we call cultural evolution. We seem to be gradually
heading towards life 3.0, which is life that can design not
just its software, but also its hardware. Maybe we're at 2.1
right now because we can get cochlear implants
and artificial knees, and a few minor
things like this. But if you were robots that were
able to think as cleverly as right now, of course, there
would be no limits whatsoever to how you could
upgrade yourselves. So let's first talk about
the power of the technology. Obviously, the power of AI has
improved dramatically recently. I'm going to define
intelligence itself, just very broadly,
as the ability to accomplish complex goals. I'm giving such a
broad definition because I want to
be really inclusive and include both all forms
of biological intelligence and all forms of
artificial intelligence. And as you guys here
at Google all know, obviously, a subset of
artificial intelligence is machine-learning, where
systems can improve themselves by using data from
their environment, much like biological
organisms can. And another subset of
that is, of course, deep learning, in which we
use neural-net architectures. And if you look at older
breakthroughs in AI, like when Gary Kasparov got his
posterior kicked by IBM's Deep Blue, the intelligence here
was, of course, mainly just put in by human programmers. And Deep Blue beat Kasparov just
because it could think faster and remember better. Whereas, in contrast,
the recent stuff that you've done here at
Google, like this work by Ilya Sutskever's group, there's
almost no intelligence at all put in by
the humans, right? They just trained
a simple neural-- with a bunch of data. And you put in the numbers that
represent the pixel colors, and it puts out this
caption-- "a group of young people playing
a game of Frisbee," even though the software
was never taught anything about what a Frisbee
is, or what a human is, or what a picture is. And the same stuff, if
you put in other images, it gives other captions, which
are often quite impressive. I find it even more
striking how cool things can be done with video. So this is Google
DeepMind, of course, learning to play Atari games. And for those of you-- those few of you who
haven't seen this before, you need to remember that
this neural network, here, with simple
reinforcement-learning built in, had no idea what a
game was, what a paddle was, what a ball was, or
anything like that. And just by practicing,
gradually it starts to miss the ball less
often and got to a point where it hardly
missed it at all, and plays much better
than I could play this. And the real kicker
is that, of course, the people at
DeepMind, they actually didn't know that there was
this clever trick you can do when you play Breakout, which
is always aim for the corners and try to build a little--
do a little tunnel, there. So once this little
deep-learning software figured that out, it's
just every single time the ball comes back-- look how
just uncannily precise it is. It's just putting it right
back there in the corner and playing. I can only dream
to play this well. Now this is, of course, a
very, very simple environment, that little
two-dimensional game world. But if you're a robot, you can
think of life as a game, just a more complex one. And you could ask
yourself to what extent these sort of techniques
might enable you to learn more interesting things. And so DeepMind more recently,
had three-dimensional robots in a simulated
world, and just asked to see if they could learn
to do things like walk. And this is what happened. [MUSIC PLAYING] This software had never, ever
seen any videos of walking. It knew nothing about
the concept of walking. All the software was doing was
sending random commands to how to bend the different
joints, and it got rewarded every time
this creature managed to move a little bit forward. And it looks a bit funky,
maybe a little awkward, but hey, it actually
learns interesting stuff. So this raises this very
interesting question of how far can AI go? How much of what we humans
can do will machines, ultimately, be able
to do, if we use not just the techniques that we
know of so far, but factor in all sorts of additional
progress that you people in the room and elsewhere
are going to do? I like to think about this
in terms of this landscape. I drew this-- I made this picture inspired
by a paragraph in one of my favorite books by Hans
Moravec from many years ago, where the height,
here, represents how difficult it is for a
computer to do a certain task. And the sea level represents
how good computers are at doing them right now. So what we see here
is that certain tasks, like chess-playing
and arithmetic have, of course, long been
submerged by this slowly rising tide of machine intelligence. And there are some
people who think that there are certain tasks,
like art and book-writing, or whatever, that machines
will never be able to do. And then there are
others who think that the old goal of
AI to really solve intelligence and do
everything that we do will mean that sea levels will
eventually submerge everything. So what's going to happen? There have been a lot of
interesting polls of AI researchers and the
conclusion is very clear-- we don't know. A little bit more specifically,
though, what you find is there are some people in
the techno-skeptic camps who think that AI research
is ultimately doomed. We're never going to get there. Or maybe we're only
going to get there hundreds of years from now. But actually, most
AI researchers think it's going to happen
more in a matter of decades. And some people
think that we don't have to worry so much
about steering this rocket, metaphorically speaking, because
it's not going to happen, that we'll ever get
powerful enough that we have to worry about this-- but that's a minority. And then there are
people who think we don't have to
worry about steering because it's guaranteed that the
outcome is going to be awesome. I call such people
digital Utopians. And I respect that
point of view. And there are also people
who think it's guaranteed that things are going
to suck, so there's no point in worrying
about steering because we're screwed anyway. But most of the
people in surveys tend to land more here, in
the middle, in what I've called the beneficial AI
movement, where you're really motivated, actually,
to ask, what can we do right now to steer
things in a good direction? Because it could be awesome,
or it could be not so great, and it depends on
what we do now. I put this web page
up, AgeofAI.org. We did a survey there for
people from the general public could answer these
same questions. You can go there and do it, too. And I was actually
very interested that the general public response
was almost exactly the same as AI researchers have
done in recent polls. This is from something
I analyzed this weekend with 14,866 respondents. And you see most people
think maybe we're decades away from human-level
AI, maybe it'll be good, maybe there'll be problems. So this is maximally
motivating to think about how we can steer this
technology in a good direction. So let's talk about steering. How can we control-- how can we learn to control AI
to do what we want it to do? NASA EMPLOYEE: Lift-off. MAX TEGMARK: To help with
this, my wife Meia, who's sitting there, and I,
and some other folks, founded The Future
of Life Institute. And you can see we actually
have the word 'steer' up here in our mission statement. Our goal is simply to do
what we can to help make sure that technology is
beneficial for humanity. And I'm quite optimistic that
we can create a really inspiring future with technology,
as long as we win this race
between the growing power of the technology
and the growing wisdom with which we manage it. But I think if we're
going to win this race, we actually have to
shift strategies, because technology is gradually
getting more powerful. And when we invented less
powerful tech, like fire, we very successfully
used the strategy of learning from mistakes. We invented fire-- oopsy-- and then invented the
fire extinguisher. We invented the car-- oopsy-- and then we invented
the seat belt, the airbag, the traffic light, and things
were more or less fine. But when you get beyond
the certain point of the power of the
technology, this idea of learning from
mistakes is just really, really lousy, right? You don't want to make mistakes
if one mistake is unacceptably many. And when we talk about nuclear
weapons, synthetic biology, and certain
superhuman AI, I feel we're at the point
where we really don't want to make mistakes. We want to shift
strategy from being reactive to being
proactive, which is exactly the slogan you said you're
also using for your work here at Google, earlier. I'm optimistic that we can do
this if we really focus on it and work for it. Some people say, nah,
don't talk about this because it's just
Luddite scaremongering when you talk about things
that could go wrong. I don't think it's
Luddite scaremongering. I think it's safety engineering. We started by talking about
the Apollo moon mission. When NASA thought
through, very carefully, everything that could
possibly go wrong when you put three
astronauts on top of this 100-meter tall rocket
full of highly explosive fuel, that wasn't Luddite
scaremongering. What they were doing was
precisely what ultimately led to the success of the mission. And this is what I think we
want to be doing with the AI as well. I think so far, what we've
learned from other technologies here is that we need to
up our game a little bit because we haven't really
absorbed this idea that we have to switch to being proactive. Today is a very special day
in terms of nuclear weapons, because we came pretty
close to September 26 being the 34th anniversary
of World War III. In fact, it might
have ended up this way if this guy, Stanislav Petrov,
hadn't just, on gut instinct, ignored the fact that his early
warning system said that there were five incoming
Minuteman US missiles that should be retaliated against. So how can we do better? How can we win this wisdom race? I'm very, very happy that
the AI community has really started to engage with these
issues a lot in recent years. And thanks to a lot of
people who are in this room here, including Peter Norvig,
and with The Future Life Institute, we organized a couple
of conferences in Puerto Rico. And then earlier this year
in Asilomar, California, where there was really
quite remarkable consensus around a number of very
constructive things we can do to try to
develop this wisdom and steer things in
the right direction. And I want to spend just a
little bit of time hitting some highlights of things here,
from this list of 23 Asilomar Principles, which has now
been signed by over 1,000 AI researchers around the world. First of all, it says
here on item one, that we should define
the goal of AI research not to be just making
undirected intelligence, but to make beneficial
intelligence. So in other words, the
steering of the rocket is part of the design specs. And then there was
also very strong consensus that, hey, if we have
a bunch of unanswered questions that we need to answer, we
shouldn't just say, oh yeah, we should answer them. Well, we should answer them
the way we scientifically know is the best way to
answer hard questions, namely, to research
them, to work on them. And we should fund
this kind of research as just an integral part of
computer science funding, both in companies
and in industry. And I'm actually very,
very proud of Google for being one of the founding
members of the partnership on AI, which aims
very much to support this kind of AI research-- AI safety research. Another principle here that
was very broad agreement was the shared
prosperity principle, that the economic
prosperity created by AI should be shared broadly
to benefit all of humanity. What do I mean by that? Obviously, technology has
kept growing the economic pie. It's been growing our GDP
a lot in recent decades, as you can see if you look
at the top line in this plot, here. But as you're also
generally aware of, this pie hasn't been
divvied up quite equally. And in fact, if you look at the
bottom 90% of income earners, their income has stayed flat,
almost since I was born. Actually, maybe it's my fault.
And the 30% poorest in the US have actually gotten
significantly poorer in real terms, in
recent decades, which has created a great deal of
anger, which has given us the election of Donald Trump. It's given us Brexit. And it's given us a more
polarized society in general. And so there was a very strong
consensus among AI researchers that if we can create so much
more wealth and prosperity, and have machines help produce
all these wonderful goods and services, then
if we can't make sure everybody gets better off
from this, shame on us. Some people say, well,
this is just nonsense because something magical
is going to change in these statistics soon. And the jobs that
get automated away are going to be replaced by
much better, new jobs that don't exist yet. But actually, if you
look at this data, it doesn't support that. We could have made that
same argument 100 years ago, when much more people
worked in farming, that all those
jobs that were lost were going to be replaced by
new jobs that didn't exist yet. And this is what
actually happened. This is-- I made this little
pie chart, here, of all the jobs in the US by size. And you can start going down
the list-- managers, drivers, retail salespersons,
cashiers, et cetera. Only when you get
down to 21st place do you get to a job
category that didn't exist 100 years ago,
namely, software developers. Hi, guys. So clearly what happened is
not that most farmers became software developers. What instead happened was
people who lost, generally, from the Industrial
Revolution and onward, jobs where they were using
their muscles to do work, went into other jobs
where they could use their brains to do work. And these jobs tended
to be better paid, so this was a net win. But they were jobs that
already existed before. Now what's happening
today, which is driving the growth
in income inequality, is similarly that
people are getting switched into other jobs
that had existed before. It's just that this
time, since the jobs that are being automated
away are largely jobs where they'll use
their brains, they often switch to new jobs that
existed before that pay less, rather than pay more. And I think it's a really
interesting challenge for all of us to think about
how we can best make sure that this growing
pie makes everybody better off. Another item here on this
list is principle number 18-- the AI arms race. This was the one
that had the highest agreement of all among
the Asilomar participants. "An arms race in lethal
autonomous weapons should be avoided." Why is that? Well, first of all,
we're not talking about drones, which are
remote-control vehicles where a human is still
deciding who to kill. We're talking here about
systems where the machine itself, using
machine-learning or whatever, decides exactly who
is going to be killed, and then does the killing. And first, whatever
you think about them, the fact is, although
there's been, of course, a huge
amount of investment in civilian uses of
AI recently, it's actually dwarfed by talk
about military spending here, recently. So if you look in the
pie, there's a real risk that the status
quo will just mean that most of the loud sucking
noise trying to recruit AI graduates from MIT and
Stanford and elsewhere, will be to go to military
places rather than to places like Google. And most AI researchers
felt that that would be a great shame. Here's how I think about it. If you look at any
science, you can always use it to develop new
ways of helping people, or new ways of harming people. And biologists fought
really, really hard to make sure that
their science is now known as new ways
of curing people, rather than for
biological weapons. They fought very hard and
they got an international ban on biological weapons passed. Similarly, chemists managed to
get the chemical weapons banned by really speaking
up as a community and persuading politicians
around the world that this was good. And that's why you
associate chemistry now mainly with new materials. And it's very stigmatized
to have bioweapons. So even if some
countries cheat on them, it's so stigmatized
that Assad even gave up his chemical weapons
to not get invaded. And if you want to buy
some chemical weapons to do something silly, you're
going to find it really hard to find anyone who's going to
sell them to you because it's so stigmatized. What there is very widespread
support for in the AI community is exactly the same
thing here, to try to negotiate an
international treaty where the superpowers
get together and say, hey, the main winners of
having an out-of-control arms race and AI weapons is not
going to be the superpowers. It's going to be ISIS and
everybody else who can't afford expensive weapons,
but would love to have little cheap things
that they can use to assassinate anybody with anonymously,
and basically drive the cost of anonymous
assassination down to zero. And this is something that if
you want to get involved in, the United Nations is going
to discuss this in November, actually. And I think the more vocal the
AI community is on this issue, the more likely it is
that that AI rocket here is going to veer in
the same direction as the biology and
chemistry rockets went. Finally, let me say a little
bit about the final Asilomar Principles here. I find it really
remarkable that even though a few years ago, if
you started talking about superintelligence
or existential risks, or whatever, many
people would dismiss you as some sort of
clueless person who didn't know anything about AI. These words are in
here, and yet this is signed by Demis Hassabis,
the CEO of DeepMind. It's signed by Peter Norvig,
who's just sitting over there, by your very own Jeff
Dean, and by, really, a who's-who of AI researchers,
over 1,000 of them. So there's been a much
greater acceptance of the fact that, hey, this is part of this
theory of maybe AI is actually going to succeed
and maybe we need to take these sort of
things into account. Let me just unpack
a little bit what the deal is with all of this. So first of all, why
should we take it seriously at all, this idea of
recursive self-improvement and superintelligence? We saw that a lot
of people expect we can get to human-level
AI in a few decades, but why would that
mean that maybe we can get AI much smarter
than us, not just a little? The basic argument for this
is very eloquently summarized in just this
paragraph by IJ Good, from 1965, a mathematician
who worked with Alan Turing to crack codes
during World War II. I think you've mostly
heard this all before. He basically says that if we
have a computer, a machine, that can do everything
as well as we can, well, one of the things we
can do is design AI systems, so then it can, too. And then you can hire--
instead of hiring 20,000 Google employees to do
work for you, you can get 20 million little
AI things working for you, and they can work much faster. And that the speed
of AI development will no longer be set
by the typical R&D time scale of humans, or years,
but by how fast machines can help you do this, which
could be way, way faster. And if it turns out that we
have a hardware overhang, where we've compensated for
the fact that we really are kind of clueless
about how to do the software of
human-level AI by having massive amounts
of extra hardware, then it might be
that you can get a lot of, through
improvements first, even just by changing the software,
which is something that can be done very, very
quickly, without even having to build new stuff. And then from there
on, things could get-- you might be able to get
machines that are just dramatically smarter than us. We don't know that
this will happen, but basically,
what we see here is that, for linear
researchers viewing this, is at least a possibility
that we should take seriously. Another thing which you see
here is existential risk. So more specifically,
it says here, "risks posed by AI systems,
especially existential risks, must be subject to the
planning and mitigation efforts commensurate with
their expected impact." And existential
risk is a risk which basically can include
humanity just getting wiped out altogether. Why would you possibly
worry about that? There are so many absolutely
ridiculous Hollywood movies with terminator
robots or whatever, that you can't even
watch without cringing. So what are the serious
reasons that people like this sign on to something
that talks about that? Well, the common criticism
that you hear is that, well, machines-- there's no reason to think that
intelligent machines would have human goals if we built them. And after all, why
should they have sort of weird, alpha-male goals
of trying to get power, or even self-preservation? My laptop doesn't protest when
I try to switch it off, right? But there's a very
interesting argument here I just want to share
with you in the form of this silly
little fake computer game I drew for you here. Just imagine that you are this
little blue, friendly robot whose only goal is to save
as many sheep as possible from the big bad wolf. You have not put into
this-- this robot does not have the goal of surviving,
or getting resources, or any stuff like that. Just sheep-saving. It's all about these
cute little sheepies, OK? It's going to-- very
quickly, if it's smart-- figure out that if it walks
into the bomb here and blows up, then it's not going to
save any sheep at all. So a subgoal that it will
derive is-- well, actually, let's not get blown up. It's going to get a
self-preservation instinct. This is a very generic
conclusion if you have a robot, then you program it to
walk to the supermarket and buy you food and
cook you a nice dinner, it's going to, again,
develop the subgoal of self-preservation because
if it gets mugged and murdered on the way back with
your food, it's going to not give you your dinner. So it's going to want to
somehow avoid that, right? Self-preservation
is an emergent goal of almost any goal that
the machine might have, because goals are hard to
accomplish when you're broken. And also, if the robot-- the robot might develop,
find, have an incentive to get a better model of
the world that's in here, and discover that
there is actually a shortcut it can take to get
to where the sheep are faster, then it can save more. So trying to understand more
about how the world works is a natural subgoal
you can get, no matter whatever fundamental goal you
program the machine to have. And then resource
acquisition, too, can emerge, because when
this little robot here discovers that when
it drinks the potion, it can run twice as fast,
then it can save more sheep. So it's going to
want the potion. It'll discover that
when it takes the gun, it can just shoot the wolf and
save all the sheep-- great. So it's going to want
to have resources. As I've summarized
in this pyramid, here, this idea, which
has been very eloquently-- it was mentioned first by
Steve Omohundro, who lives here in the area, and is talked a lot
about in Nick Bostrom's book. The idea is just that
whatever fundamental goal you give a very intelligent machine,
if it's pretty open-ended, it's pretty natural to expect
that it might develop subgoals of not wanting to be switched
off, and try to get resources. And that can be fine. There's not
necessarily a problem, being in the presence of
more intelligent entities. We all did that as kids,
right, with our parents? The reason it was fine
was because their goals were aligned with our goals. So therein lies the rub. We want to make sure that if
we ever give a lot of power to machines of intelligence
comparable or greater to ours, that their goals
are aligned with ours. Otherwise, we can be in trouble. So to summarize, these
are all questions that we need to answer,
technical research questions. How can you make-- how can
you have machines learn, adopt, retain our
goals, for example? And let me just show you
a very short video talking about these issues in
superintelligence and then some. [CLICKS KEYBOARD] And let's see if we have better
luck with video this time. [VIDEO PLAYBACK] - "Will artificial intelligence
ever replace humans?" is a hotly-debated
question these days. Some people claim
computers will eventually gain superintelligence, be
able to outperform humans on any task, and
destroy humanity. Other people say, don't worry. AI will just be another
tool we can use and control, like our current computers. So we've got physicist and
AI researcher Max Tegmark back again to share with
us the collective takeaways from the recent Asilomar
conference on the Future of AI that he helped organize. And he's going to help separate
AI myths from AI facts. - Hello. - First off, Max, machines,
including computers, have long been better
than us at many tasks, like arithmetic, or weaving,
but those are often repetitive and mechanical operations. So why shouldn't I
believe that there are some things that are simply
impossible for machines to do as well as people, say,
making Minute Physics videos, or consoling a friend? - Well, we've traditionally
thought of intelligence as something mysterious
that can only exist in biological
organisms, especially humans. But from the perspective
of modern physical science, intelligence is simply
a particular kind of information processing
and reacting, performed by particular arrangements of
elementary particles moving around. And there's no law
in physics that says it's impossible to do that
kind of information processing better than humans already do. It's not a stretch to say
that earthworms process information better than
rocks and humans better than earthworms. And in many areas, machines
are already better than humans. This suggests that
we've likely only seen the tip of the
intelligence iceberg, and that we're on track to
unlock the full intelligence that's latent in nature and use
it to help humanity flourish, or flounder. - So how do we keep
ourselves on the right side of the flourish-or-flounder
balance? What, if anything,
should we really be concerned about with
superintelligent AI? - Here is what has many top
AI researchers concerned. Not machines or
computers turning evil, but something more subtle-- superintelligence that simply
doesn't share our goals. If a heat-seeking missile
is homing in on you, you probably wouldn't think, no
need to worry, it's not evil. It's just following
its programming. No, what matters to you is what
the heat-seeking missile does and how well it does it,
not what it's feeling, or whether it has
feelings at all. The real worry isn't
malevolence, but competence. Superintelligent AI
is, by definition, very good at
attaining its goals. So the most important
thing for us to do is to ensure that its goals
are aligned with ours. As an analogy, humans are
more intelligent and competent than ants, and if we want to
build a hydroelectric dam where there happens to be
an anthill, there may be no malevolence
involved, but, well, too bad for the ants. Cats and dogs, on
the other hand, have done a great job
of aligning their goals with the goals of humans. I mean, even though
I'm a physicist, I can't help think kittens
are the cutest particle arrangements in our universe. If we build
superintelligence, we'd be better off in the position
of cats and dogs than ants. Or better yet,
we'll figure out how to ensure that AI
adopts our goals, rather than the other way around. - And when exactly
is superintelligence going to arrive? When do we need to
start panicking? - First of all, Henry,
superintelligence doesn't have to be
something negative. In fact, if we get it right,
AI might become the best thing ever to happen to humanity. Everything I love
about civilization is the product of intelligence,
so if AI amplifies our collective
intelligence enough to solve today's and
tomorrow's greatest problems, humanity might flourish
like never before. Second, most AI researchers
think superintelligence is at least decades away. But the research
needed to ensure that it remains
beneficial to humanity rather than harmful
might also take decades, so we need to start right away. For example, we'll
need to figure out how to ensure machines learn the
collective goals of humanity, adopt these goals
for themselves, and retain the goals as
they get ever smarter. And what about when
our goals disagree? Should we vote on what the
machines' goals should be? Should we do whatever
the president wants, whatever the creator of the
superintelligence wants, let the AI decide? In a very real way,
the question of how to live with superintelligence
is a question of what sort of future we want
to create for humanity, which obviously shouldn't just
be left to AI researchers, as caring and socially
skilled as we are. [END PLAYBACK] MAX TEGMARK: So that leads
to the very final point I want to make here today. To win this wisdom race,
creating an awesome future with AI, in addition to doing
these various things I've talked about, we really need to
think about what kind of future we want, what sort of
goal we want to have, where we want to
steer our technology. So just for fun, the survey
I mentioned that we did, we asked people also to say
what they wanted for the future. And I'll just share
with you here. These are from the analysis
I did last weekend. Most people out of
the 14,866 here, say they actually
want AI to go all the way to superintelligence. Although some are
saying no, here. A lot of people want
humans to be in control. Most people actually want
both humans and machines to be in control together. And a small fraction,
[INAUDIBLE],, prefer the machines
to be in control. [LAUGHTER] And then, when asked about
consciousness, a lot of people said, yeah, if
they have machines that are behaving as if they
are as intelligent as humans, they would like
to have them have a subjective experience also,
so the machines can feel good. But some people said,
nah, they prefer having zombie robots that don't
feel conscious, that people don't have to feel guilty about
switching them off or giving them boring things to do. In terms of what a future
civilization should strive for, there was a large majority
who felt we should either try to maximize
positive experiences, or minimize suffering,
or something like that. Then more people who said
let the future civilization pick whatever goals they want,
as long as it's reasonable. Some people said
they didn't even care about it if they thought
the goal that the future wanted was reasonable, even if it was
pointlessly banal, like maybe turning our universe
into paper clips. They were fine with just
delegating it to humans. But most people actually
felt that since we're creating this technology,
we have the right to have some say as to
where things should go. The broadest agreement of
all was on this question that, actually, maybe
we shouldn't just limit the future
of life to forever be stuck on this little planet,
but give it the potential to spread and flourish
throughout the cosmos. And to get people thinking
more about different futures, my wife, Meia,
likes to point out that even though
it's a good idea to visualize positive outcomes
when you plan your own career, and then try to figure
out how to get there, we kind of do the exact
opposite as a society. We just tend to think
about everything that could possibly go wrong
and then we freak out about it. When you watch
Hollywood movies, it's almost always dystopic
depictions of the future, right? So to get away from this
a little bit in my book, the whole chapter 5 is
theories of thought experiments with different future
scenarios, trying to span the whole range of
what people have talked about, and other, so you,
yourselves, can ask what you would actually prefer. And the most striking
thing from the survey was that people
disagree very strongly in what sort of society
they would like. And this is a
fascinating discussion that I would really encourage
you all to join into. I'm just going to end by
saying that I think when we look to the
future, there's really a lot to be excited about. People sometimes ask me, Max,
are you for AI or against AI? And I respond by asking
them, what about fire? Are you for it or against it? And of course, they'll
concede that they're for fire to heat their
homes in the winter and against fire for arson. But it's the same
with all technology, it's always a
double-edged sword. The difference with AI is
just it's much more powerful, so we need to put even more
effort into how we steer it. If you want life to exist for
beyond the next election cycle, and maybe, hopefully, for
billions of years on Earth and maybe beyond,
then just pressing pause on technology forever-- that's actually just
a really sucky idea. Because if we do
that, the question isn't whether humanity
is going to go extinct. The question is just,
what's going to wipe us out? Whether it's going to be the
next massive asteroid strike, like the one that took the dinos
out, or the next super volcano, or another one on a
list of long things that we know are going
to happen to Earth, that technology can create-- sorry, that technology
can prevent, but technology that
we don't have yet. It's going to require further
development of our tech. So I, for one, think that it
would be really foolish if we just run away from technology. I'm much more excited about,
in the Google spirit-- and I love your old
slogan, "Don't Be Evil"-- asking, what can we do
to steer, to develop [? theoretic ?]
technology in a direction so that life can
really flourish? Not just for the
next election cycle, but for a very, very
long time on Earth, and maybe even
throughout our cosmos. Thank you. [APPLAUSE] JOHN BRACAGLIA:
Thanks so much, Max. Now we have time for
questions from the audience. We have a mic over here, which
we can use for questions. And also, I can pass
this one around. And while we're doing that,
I'll pull up the Dory. MAX TEGMARK: Great. And since you mentioned there
were a lot of questions, make sure to keep
the questions brief, and make sure that they
actually are questions. AUDIENCE: AI risk seems to have
become a much more mainstream worry in the last few years. What changed to make that
happen and why didn't we do it earlier? MAX TEGMARK: I agree with you. I'm actually very, very happy
that it's changed in this way, and trying to help
make it change this way was the key reason
we founded the Future of Life Institute and organized
the Puerto Rico conference and the Asilomar
conference, and so on. Because we felt that up
until a few years ago, the debate was kind
of dysfunctional. And what I think has really,
really changed things for the better is that the
AI research community itself has really engaged, joined this
debate and started to own it. I think that's why it's become
more mainstream, and also much more sensible. JOHN BRACAGLIA: [INAUDIBLE] MAX TEGMARK: OK,
so you're the boss. Should we alternate with
online, offline questions? Do you want to
read the questions? JOHN BRACAGLIA: Oh, sure. "What would you most hope
to see a company like Google do to ensure safety as
we transition to a more AI-centric world?" MAX TEGMARK: So
as I said, I think Google already has the soul
to do exactly what's needed. This "Don't Be Evil" slogan
of Larry and Sergey-- I interpret it as though we
shouldn't just build technology because it's cool, but we
should think about its uses. For those of you who
know the old Tom Lehrer song about "Wernher von Braun,"
(SINGING IN A GERMAN ACCENT) Once the rockets go up, who
cares where they come down? That's not my department,
says Wernher von Braun. I view Google's
"Don't Be Evil" slogan as exactly the opposite
of that-- thinking mindfully about how to steer
the technology to be good. And I'm also really
excited again that Google is one of the founding partners
in the Partnership for AI, trying to make sure
that we-- that this happens not just in
what Google does, but throughout the community. And I also think
it's great if Google can pull all of its strings
to persuade politicians all around the world to
seriously fund AI safety research, because the sad
fact is, even though there's a great will for AI researchers
to do this stuff now, there's almost no
funding for it still. What Elon Musk helped us
give out 37 grants for is just a drop in the
bucket of what's needed. And it makes sense that Google
and other private companies want to own the IP on things
that make AI more powerful and build products out of it. But these same
private companies, it's better for them all
if nobody patents the way to make it safe, and keep
others from using it, right? That's something that's great
if it's developed openly by companies who share
it, or in universities, so that everybody can use
the same best practices and raise the quality
of safety everywhere. AUDIENCE: All right,
so Max, I actually talked to you last night
about a lot, like future-- really long, maybe 100 years-ish
what's going to happen. But if you look at AI
nowadays, not a lot of people are focusing on today,
just the imminent risks. So if you think about how
did Trump got elected, and how those things went
wrong in the last few years, you can't really deny
that AI has contributed a lot, especially in
the fake news, that AI's like suggested contents. So is that like focusing on
all energy into the future? So I felt there are
really few people that's looking into today. So do you think
that's a problem, or do you think we need
to do better on that? MAX TEGMARK: Yeah,
I think there's a really great opportunity for
us nerds in the tech community to educate the broader public
and politicians about the need to really engage with this. This is one of the reasons
I wanted to write this book. I think when I watched
the presidential debates for the last election, for
example, completely aside from the issues
they talked about, I thought it was just
absolutely astonishing what they didn't talk about. None of them talked
about AI at all. Hello? They're talking about jobs,
they're not mentioning AI. They're talking about
international security, they're not talking about AI,
like, the biggest technology out there. And I think in addition to
just telling politicians to pay attention, I think it's
incredibly valuable, also, if a bunch of people
from the tech community can actually go into
government positions to add more human-level
intelligence in government, to prevent the world governments
from being asleep at the wheel. AUDIENCE: I mean, actually-- MAX TEGMARK: Maybe
we should just-- JOHN BRACAGLIA: [INAUDIBLE] MAX TEGMARK: We can talk more
afterwards, but give everybody a chance to ask first. AUDIENCE: Hello? Hi. When you introduced
the concept-- when you introduced
the Asilomar Treaty, you mentioned the difference
between undirected intelligence and benevolent intelligence. Don't you think that
if humans succeeded in creating controllable,
benevolent intelligence, that they really have failed
in creating intelligence? Let me rephrase-- MAX TEGMARK: I'm not sure I
fully understood this question. Do you want to just
repeat the punch line? AUDIENCE: I'll rephrase. Do you think that benevolent
intelligence would be the intelligence that we
should strive towards, or should it be general
intelligence that perhaps cannot be controlled? MAX TEGMARK: So that's
a great question. You asked what I think. I am trying to be very
open-minded about what we actually want. And I wrote the book not-- really avoiding saying what
I think the future should be, because I think this is
such an important question, we just need everybody's
wisdom on it. And again, I talk about all
these different scenarios, some of them which correspond to
some of the different options you even listed there. And I'm incredibly
interested to hear what other people
think would actually be good with these things. One thing that Meia and
I found very striking when we discussed this was-- when I was writing the book,
was even though I tried quite hard to emphasize the
upsides of each scenario, there wasn't a single one there
that I didn't have at least some major misgivings about. JOHN BRACAGLIA: "Do you
think deep neural networks would be the way to get
to artificial general intelligence? If not, do you see fundamental
reasons why these do not have the potential for
recursive self-improvement that can speed up the
development of AGI or superintelligence?" MAX TEGMARK: All right,
that's a great question. So I think that although-- let me say two
things about this. First of all, our
brain seems to be, of course, some kind of a
recurrent neural network that's very, very complicated, and it
has human-level intelligence. But I think it
would be a mistake to think that that's
the only route there. I think it'd also be
a mistake to think, assume that that's the
fastest route there. Meia likes to
point out that even though, finally, a few years
ago, there was a beautiful TED Talk demonstrating the
first-ever successful mechanical bird, that came a
hundred years after the Wright brothers built airplanes. And when I flew here yesterday-- you'll be very
surprised to hear this, but I didn't come in
a mechanical bird. It turned out there was a
much easier, simpler way to build flying machines. And I think we're going to
find exactly the same thing with human-level
intelligent machines. The brain is just optimized
for very different things than what your machines
that you build are. The brain is-- Darwinian evolution is
obsessed about only building things that can self-assemble. Who cares if your laptop
can self-assemble? Evolution is obsessed
about creating things that can self-repair. It would be nice if your
laptop could self-repair, but it can't and you're
still using it, so. And also evolution doesn't care
about simplicity for humans to understand how it works,
but you care lot about that. So maybe this is
much more complicated than it needs to be, just
so it can self-assemble, and blah, blah, whatever. My guess is that the first
human-level AI will not be working exactly like the brain. That it will be something
much, much simpler, and maybe we'll use that to create later-- figure out how
human brains work. That said, the deep neural
networks are, of course, inspired by the brain and
are using some efforts-- some very clever
computational techniques that evolution came up with. My guess is that the fastest
route to human-level AI will actually use a combination
of deep neural networks with GOFAI-- various good old-fashioned
AI techniques, more logic-based
things, which have a lot of their own
strength for building, like for building a world
model and things like this. JOHN BRACAGLIA: Live question? MAX TEGMARK: Maybe I should just
add one more thing about this. Also this poses-- the increasing
successes of neural networks also poses a really
interesting challenge. Because when we put AI in charge
of more and more infrastructure in our world, it's
really important that it be reliable and robust. Raise your hand if your computer
has ever crashed on you. That wouldn't have been so
fun if it was the machine that was controlling your
self-driving car, or your local
nuclear power plant, or your nation's nuclear
weapons system, right? And so we need to transform
today's buggy and hackable computers into robust AI systems
that we can really trust. What is trust? Where does trust come from? It comes from understanding
how things work. And neural networks, I think,
are a double-edged sword. They are very powerful, but
we understand them much less than traditional software. So in my group at
MIT, actually, we're working very hard
right now on a project that I call intelligible
intelligence, where-- we're trying to come
up with algorithms where you can transform neural
networks into things which-- where you can really understand
better how they work. I think this is a challenge
that I would encourage you to all think about, too. How can you combine the
power of neural nets with stuff that you
can really understand better, and therefore trust? AUDIENCE: So should
we be afraid that AI will use its superintelligence
to figure it out that its treatment by the
humans is, essentially, slavery with just extra steps? MAX TEGMARK: That's a
wonderful, wonderful question. I haven't talked at all
about consciousness here, but the whole chapter 8
in the book is about that. And a lot of people say things
like, well, machines can never have a subjective
experience and feel anything at all, because
to feel something, you have to be made of cells,
or carbon atoms, or whatever. As a scientist, I really hate
this kind of carbon chauvinism. I'm made of the same kind
of up-quarks, down-quarks, and electrons as all
the computers are. Mine are just arranged in
a slightly different way. And it's obviously something
about the information processing that's all
that matters, right? And moreover, this kind of
self-justifying arguments have been used by people
throughout history to say, oh, it's OK
to torture slaves because they don't have souls. They don't feel anything. Oh, it's OK to torture chickens
today in giant factories because they don't
feel anything. And of course,
we're going to say that about our future
computers, too, because it's convenient for us. But that doesn't mean it's true. And I think it's actually a
really, really interesting question, to first figure
out, what is it exactly that makes an information
processing system have subjective experience? A lot of my colleagues,
whom I really respect, think this is just BS,
this whole question. This is what Daniel
Dennett says-- I looked up in the "MacMillan
Dictionary of Psychology" and it said that consciousness
is that nothing worth reading has ever been written on. But I really disagree with this. And actually, let me
just take one minute and explain why I think this
is actually a scientifically interesting question. So look at this. OK, and ask yourself,
why is it that when I show you 450
nanometer light on the left, and 650 nanometer
light on the right, why do you subjectively
experience it like this, [CLICK] and not like this? Why like this,
and not like this? I put to you that this is
a really fair-game science question that we simply don't
have an answer to right now. There's nothing to do
with wavelengths of light, or neurons, or anything
that explains this, but it's an observational fact. And I would like to understand,
why does it feel like anything, why do we have this experience? You might say,
well, look, we know that there are three
kinds of light sensors in a retina, the cones. And when I-- with a
450 nanometer light to activate one kind, and I
have the longer wavelength to activate the other
kind, and then you can see how they're
connected to various neurons in the back of your brain. But that just
sharpens the question, the mystery of
consciousness, because this proves that it had nothing
to do with light at all, because you can
experience colors even when you're dreaming, when
different neurons in your brain are active, when there is
no light involved, right? So my guess is that
consciousness-- by which I mean subjective
experience-- is simply the way information feels when
it's being processed in certain complex ways. And I think there
are some equations that we will one day
discover that specify what those complex ways are. And once we can
figure that out, it'll both be very useful because
we can put a consciousness detector in the
emergency room, and when an unresponsive patient
comes in, you can figure out if they have locked-in
syndrome, or not. And it will also
enable us to answer this really good question you
asked about whether machines should also be viewed
as moral entities that can have feelings. And above all, and I don't
see Ray Kurzweil here today, but if he can one day upload
himself into a Ray Kurzweil robot and live on for thousands
of years, and he talks like Ray and he looks like Ray
and he acts like Ray, you'll feel that
that's great for Ray. Now he's immortal. But suppose it turns out that
that machine is just a zombie and doesn't feel like
anything to be it, he would be pretty
bummed, wouldn't he? All right? And if in the future, life
spreads throughout our cosmos in some post-biological
form, and we're like, this is so exciting. Our descendants are doing
all these great things and we can die happy. If it turns out that they're all
just a bunch of zombies and all that cool stuff is just
a play for empty benches, wouldn't that suck? JOHN BRACAGLIA: I'll do
another question from the Dory. "What do you think is the most
effective way for individuals to embrace or promote a
security-engineering mentality, i.e., where not even one glitch
is tolerable, when working on AI-related projects?" MAX TEGMARK: Well,
first of all, I think we have a lot
to learn from existing successes in safety engineering,
that's why I started by showing the moon mission. It's not like this is
anything new to engineers. I think it's just that
we're so used to the idea that AI didn't
work, that we didn't need to worry about
the impact of things. And now it is beginning
to have an impact, so we should think it through. And then there are
also a few challenges which are really unique
and specific to AI. Some of the Asilomar
Principles talk about them and this Research Agenda
for AI Safety Research is a really long list of
specifics of safety engineering challenges that we need smart
people like you to work on. And I hope we can support that. AUDIENCE: So also on the
topic of security engineering, a lot of rockets blew up
on the way to the moon. MAX TEGMARK: Yeah. AUDIENCE: And given the
intelligence explosion, it's like we're
only going to have one chance to be able to get
the alignment problem correct. And I think we
couldn't even align on a set of values in this
room, let alone a system that would govern the
world effectively, because there's certainly
some drawbacks of capitalism. So I'm hopeful-- I am
glad that Elon is hedging our bets by making a magic
hat, but it seems like you and your group are focusing
on the alignment problem, and I'm just kind
of just curious what makes you optimistic that we're
going to be able to get it right on the first time? MAX TEGMARK: So first of all,
yeah, a lot of rockets blew up. But you will note that
most of the rockets that blew up, in fact, all the-- that's the rocket that blew
up in the moon mission had no people in them, right? So that was safety engineering. The high-risk stuff, they did
it in a controlled environment where the failure
didn't matter so much. So if you make some
really advanced AI, you want to understand
it really well, maybe don't connect it to the
internet the first time, right? So the downsides are small. There's a lot of things
like this that you can do. And I'm not saying that there's
one thing that we should particularly focus on, either. I think the community
has brainstormed up a really nice, long
list of things, and we should really
try to work on them all, and then we'll figure out some
more challenges along the way. But so the main thing we need to
do is just knowledge that yeah, this is valuable. Let's work on it. Then you asked also
why I'm optimistic. Let me just clarify. There are two kinds of optimism. There's naive optimism,
like my optimism that the sun is going to rise
over Mountain View tomorrow morning, regardless
of what we do. That's not the kind
of optimism I feel about the future of technology. Then there's the
kind of optimism that you're optimistic that
this can go well if we really, really plan and work for it. That's the kind of
optimism I feel here. We have in our hands to
create an awesome future, but so let's roll up
our sleeves and do it. AUDIENCE: Hey, Max. In the paper that you
wrote entitled "Why Does Cheap and Deep
Learning Work So Well?" with Lin, and now Rolnick, as
well, you ask a key question and you draw a
lot of connections between a deep learning
and then core parts of what we know about physics. Low polynomial order,
hierarchical processes, things like that. I'm just curious, what
are the reactions you've received both from
the physics community, and then from the AI community
to that attempt to kind of draw some deep parallels? MAX TEGMARK: Generally
quite positive feedback. And then also people
who have pointed out a lot of additional research
questions related to that, which are really worth doing. And just to bring
everybody up to speed as to what we're talking about,
so we don't devolve into just discussing a nerdy
research paper here, we were very intrigued
by the question of why deep learning works
so well, because if you think about it naively, even if
I just want to classify, take all the Google Images
that have cats and dogs, and I want to write-- and I want to take a
neural network that will take in, say,
1 million pixels and output the probability
that it's a cat, right? If you think about
it just a little bit, you might convince
yourself that it's impossible because how
many such images are there? Even if they're just
black-and-white images, each pixel can be
black or white, there's 2 to the power of 1
million possible images, which is much more images than there
are atoms in our universe. There's only 10
to the 78, right? And for each image, you have
to output a probability. So to specify an arbitrary
function of images, how many parameters
do you need for that? Well, 2 to the 1,000, which you
can't even fit if you store one parameter on each
atom in our cosmos. So how can it work so well? So the basic conclusion
we found there was that, of course, the
class of all functions that you can do well
with a neural network that you can actually run is
its almost infinitesimally tiny fraction of all functions. But then physics tells
us that the fraction of all functions that
we actually care about, because they're
relevant to our world, is also an almost
infinitesimally small fraction. And conveniently,
they're almost the same. I don't think this was luck. I think Darwinian
evolution gave us this particular kind of
neural network-based computer precisely because it's
really well-tuned for tapping into the kind of
computational needs that our cosmos has
dished out to us. And I'll be delighted
to chat more with you later about loose
ends to this, because I think there's a lot
more interesting stuff to be done on that. JOHN BRACAGLIA: Take
a Dory question. "Being humans in the age of AI
seems like an egocentric effort that gives an undeserved
special status to our species. Why should we even
bother to remain humans when we could get to
push our boundaries and see where we get?" MAX TEGMARK: All right. [LAUGHING] "An egocentric effort that gives
us undeserved special status to our species." Well, first of
all, you know, I'm totally fine with
pushing our boundaries and I've been advocating
for doing this. I mean, I find it very
annoying human hubris when we go on a soapbox and
we're like, (DEEPLY) we are the pinnacle of
creation and nothing can ever be smarter
than us, and we try now to build our whole self-worth
on somehow human exceptionalism. I think that's kind of lame. On the other hand-- we should probably make
this the last question. On the other hand,
egocentric efforts-- well, we are the only ones-- it's only us humans who are in
this conversation right now, and somebody needs to have it. So it's really up to us
to talk about it, right? We can't use this kind
of thinking as an excuse to just not talk
about it and just bumble into some completely
uncontrolled future. I think we should take a
firm grip on the rudder and steer in whatever direction
we decide to steer in. So let me thank you again
so much for coming out. It's a wonderful
pleasure to be here. [APPLAUSE] And If you have any more
questions you didn't get in, I'll be here signing books
and I'm happy to chat more. JOHN BRACAGLIA: Thank you all
for coming and thanks, Max, for talking at Google. MAX TEGMARK: And thank
you for having me.
