Translator: Hélène Vernet
Reviewer: Bangyou Xiang The rise of the machines! Who here is scared of killer robots? (Laughter) I am! I used to work in UAVs -
Unmanned Aerial Vehicles - and all I could think seeing
these things is that someday, somebody is going to strap
a machine-gun to these things, and they're going
to hunt me down in swarms. I work in robotics at Brown University
and I'm scared of robots. Actually, I'm kind of terrified, but, can you blame me? Ever since I was a kid,
all I've seen are movies that portrayed the ascendance
of Artificial Intelligence and our inevitable conflict with it - 2001 Space Odyssey,
The Terminator, The Matrix - and the stories they tell are pretty scary: rogue bands of humans running away
from super intelligent machines. That scares me. From the hands,
it seems like it scares you as well. I know it is scary to Elon Musk. But, you know, we have a little bit
of time before the robots rise up. Robots like the PR2
that I have at my initiative, they can't even open the door yet. So in my mind, this discussion
of super intelligent robots is a little bit of a distraction
from something far more insidious that is going on with AI systems
across the country. You see, right now,
there are people - doctors, judges, accountants - who are getting information
from an AI system and treating it as if it is information
from a trusted colleague. It's this trust that bothers me not because of how often
AI gets it wrong. AI researchers pride themselves
in accuracy on results. It's how badly it gets it wrong
when it makes a mistake that has me worried. These systems do not fail gracefully. So, let's take a look
at what this looks like. This is a dog that has been misidentified
as a wolf by an AI algorithm. The researchers wanted to know: why did this particular husky
get misidentified as a wolf? So they rewrote the algorithm
to explain to them the parts of the picture
it was paying attention to when the AI algorithm made its decision. In this picture, what do you
think it paid attention to? What would you pay attention to? Maybe the eyes,
maybe the ears, the snout ... This is what it paid attention to: mostly the snow
and the background of the picture. You see, there was bias in the data set
that was fed to this algorithm. Most of the pictures of wolves were in snow, so the AI algorithm conflated
the presence or absence of snow for the presence or absence of a wolf. The scary thing about this is the researchers had
no idea this was happening until they rewrote
the algorithm to explain itself. And that's the thing with AI algorithms,
deep learning, machine learning. Even the developers who work on this stuff have no idea what it's doing. So, that might be
a great example for a research, but what does this mean in the real world? The Compas Criminal Sentencing
algorithm is used in 13 states to determine criminal recidivism or the risk of committing
a crime again after you're released. ProPublica found
that if you're African-American, Compas was 77% more likely to qualify
you as a potentially violent offender than if you're a Caucasian. This is a real system being used
in the real world by real judges to make decisions about real people's lives. Why would the judges trust it
if it seems to exhibit bias? Well, the reason they use Compas
is because it is a model for efficiency. Compas lets them go
through caseloads much faster in a backlogged criminal justice system. Why would they question
their own software? It's been requisitioned by the State,
approved by their IT Department. Why would they question it? Well, the people sentenced
by Compas have questioned it, and their lawsuits should chill us all. The Wisconsin State Supreme Court ruled that compass did not deny
a defendant due process provided it was used "properly." In the same set of rulings, they ruled that the defendant could not inspect
the source code of Compass. It has to be used properly
but you can't inspect the source code? This is a disturbing set of rulings
when taken together for anyone facing criminal sentencing. You may not care about this because
you're not facing criminal sentencing, but what if I told you
that black box AI algorithms like this are being used to decide whether or not
you can get a loan for your house, whether you get a job interview, whether you get Medicaid, and are even driving cars
and trucks down the highway. Would you want the public
to be able to inspect the algorithm that's trying to make a decisiom
between a shopping cart and a baby carriage
in a self-driving truck, in the same way the dog/wolf
algorithm was trying to decide between a dog or a wolf? Are you potentially a metaphorical dog
who's been misidentified as a wolf by somebody's AI algorithm? Considering the complexity
of people, it's possible. Is there anything
you can do about it now? Probably not, and that's
what we need to focus on. We need to demand
standards of accountability, transparency and recourse in AI systems. ISO, the International Standards
Organization, just formed a committee to make decisions about
what to do for AI standards. They're about five years out
from coming up with a standard. These systems are being used now, not just in loans, but they're being
used in vehicles like I was saying. They're being used in things like
Cooperative Adaptive Cruise Control. It's funny that they call that "cruise control" because the type of controller used
in cruise control, a PID controller, was used for 30 years in chemical plants
before it ever made into a car. The type of controller that's used to drive a self-driving car
and a machine learning, that's only been used
in research since 2007. These are new technologies. We need to demand the standards
and we need to demand regulation so that we don't get snake oil
in the marketplace. And we also have to have
a little bit of skepticism. The experiments in Authority done by Stanley Milgram
after World War II, showed that your average person
would follow an authority figures orders even if it meant harming their fellow citizen. In this experiment, every day Americans would shock an actor past the point of him
complaining about her trouble, past the point of him screaming in pain, past the point of him
going silent in simulated death, all because somebody with no credentials, in a lab coat, was saying some variation of the phrase "The experiment must continue." In AI, we have Milgram's
ultimate authority figure. We have a dispassionate
system that can't reflect, that can't make another decision, that there is no recourse to, that will always say "The system
or "The process must continue." Now, I'm going to tell you a little story. It's about a car trip I took
driving across country. I was coming into Salt Lake City
and it started raining. As I climbed into the mountains,
that rain turned into snow, and pretty soon that snow was whiteout. I couldn't see the taillights
of the car in front of me. I started skidding. I went 360 one way,
I went 360 the other way. I went off the highway. Mud-coated my windows,
I couldn't see a thing. I was terrified some car was going
to come crashing into me. Now, I'm telling you this story
to get you thinking about how something small
and seemingly mundane like a little bit precipitation, can easily grow
into something very dangerous. We are driving in the rain
with AI right now, and that rain will turn to snow, and that snow could become a blizzard. We need to pause, check the conditions, put in place safety standards, and ask ourselves
how far do we want to go, because the economic incentives
for AI and automation to replace human labor will be beyond anything we have seen
since the Industrial Revolution. Human salary demands can't compete with the base cost of electricity. AIs and robots will replace
fry cooks and fast-food joints and radiologists in hospitals. Someday, the AI will diagnose your cancer, and a robot will perform the surgery. Only a healthy skepticism of these systems is going to help keep people in the loop. And I'm confident, if we can
keep people in the loop, if we can build transparent
AI systems like the dog/wolf example where the AI explained
what it was doing to people, and people were able to spot-check it, we can create new jobs
for people partnering with AI. If we work together with AI, we will probably be able to solve
some of our greatest challenges. But to do that, we need
to lead and not follow. We need to choose to be less like robots, and we need to build the robots
to be more like people, because ultimately, the only thing we need to fear
is not killer robots, it's our own intellectual laziness. The only thing we need
to fear is ourselves. Thank you. (Applause)
