Cassie Kozyrkov: The AI safety mindset: 12 rules for a safer AI future

hi I'm delighted to be here and I'm delighted to talk to you about AI safety to give you some thinking tips that will help you think about AI in a way that helps us build a safer more reliable AI future together first tip forget the science fiction science fiction's are actually quite a dangerous distraction it's a lot of the turn they're in the portrayal of AI has to do with personhood autonomous entities instead AI comes from humans like all technology it is built by us and all technology is an echo of the wishes of whoever made it AI is human all the way through humans design it humans build it humans test it humans create the safety nets for it so there's a tiny sprinkling of computer science and mathematical stuff and a lot of the human will around it so what instead should you think about think about technologies that scale really quickly if you scale up a wish without thinking about what you have asked for you can end up in some tricky situations so first nugget don't be distracted by sci-fi instead think of AI as a useful tool and if you're worried about AI being better than human in some way let me remind you that all of our tools are better than human if you think about pen and paper cognitively it's better than you otherwise why would you use pen and paper to take notes it's got better memory than you a calculator multiplies six digit numbers faster than you do and a bucket holds water better than you do all of our tools are better than us at something so what is am I actually better than us at it's about writing software let me put this to you very straightforwardly Computers transform inputs into outputs that's what they're for and with traditional programming the way that you do that transformation so you need some kind of recipe or model to go from the input to the output and the way you make that is by coming up with the instructions by thinking really hard about the problem and handcrafting that solution but every now and then it's really hard for you to come up with what those instructions must be think about taking an image and figuring out what's in it cat or not cat that's your goal what would you have to do with each pixel to turn it into the answer that's kind of hard isn't it so how would you handcraft that recipe it's hard to express what the answer should be so so how do you just come up with it wouldn't you prefer to go it differently instead of using instructions wouldn't you prefer to say here's a bunch of cats here's a bunch of not cats figure it out instead of using instructions you would use examples that is the essence of AI and machine learning so if you want in just one slide what AI is all about this is it you express your wish with examples you say what you want with examples not with instructions and so you would want to use this technology in situations where it's much easier for you to come up with the examples of then the instructions and it also means that you can start to automate tasks where you don't even know how will you do it you can't exactly say what you've done with each pixel to get the result but it's really important for you first to be able to come up with the examples but also for you to be able to say what it means to do your task well so to wake us all up a little bit we're going to have a let's be the project leader kind of game here so what I need from everybody is for you to tell me for each of these photographs in order whether it does or does not have a cat so you're in a shout cat or not cat for each of them I'm gonna say the number you're gonna shout cannot cat so you can think of yourself either as the human project need to hear or as the AI system labeling it as cannot can't so let's go with number one cat exactly - not cat 3 cat 4 cat 5 6 what do you mean maybe cat that wasn't an allowable label big cats not an allowable label so is it cat or is it not cat some of you are saying cat some of you are saying not cat it's got to be one or the other well see here you realize that it depends on what the system is supposed to do for you it depends on the point of the project and for different projects different answers are going to be appropriate so I suppose I'll have to step into those big boots over the leader the human decision maker and I will tell you that the purpose of this project is to serve as a pet recommender and so if it says cat the animal has to be from the cat family and in its standard adult form something that is safe to cuddle let's try number six again now everybody is saying not cat and if you're still saying cat take out more life insurance so you see that the purpose of the system really matters here there is no one objective answer every time this isn't an applet onic forms the objective is always subjective and this means that a system that is built and designed for one purpose may not work for a different purpose similarly datasets collected for one purpose may not work for another purpose so always remember that the objective is subjective it is up to the human leaders to come up with it now next question for you which do you prefer in a worker a reliable worker or an unreliable worker hmm I think you feeling like this is a trick question and it is if you strongly prefer a reliable worker watch out this actually really depends again on the human leader the decision-maker because if you have a brilliant intelligent decision maker of course you want a reliable worker to reliably exactly the instructions that they've been given but what if your decision-maker isn't very good at giving instructions well then you're gonna want an unreliable worker who does everything except what they're told so they don't go and carry out foolish instructions computer systems technologies including AI are the ultimate reliable workers they do exactly what they're told and they scale very quickly whereas by comparison humans are a little less reliable they've got a whole bunch of different motivations and incentives they prefer to sit on the beach or play with their kids or go home early instead of just following instructions and so the quality of the decision-maker really matters when we get to these AI systems they will do exactly what they're told and if they pass testing that means that they are simply delivering exactly what the people who made them asked for so next tip strive to amplify the intelligence of a skilled decision maker rather than amplifying foolishness this means that whoever is going to be building this system needs to have some skills if you want to really see where the essence of AI is danger if you will relative to traditional programming lies let me spell this out for you very directly when you are doing traditional programming you have to come up with the instructions it might take 10,000 lines of code a hundred thousand lines of code maybe and some human being has to worry about every single one of those lines agonize over it whereas when it comes to machine learning and AI there are actually only two lines there there's a whole lot of other programming that the engineer has to do but that's because the technology is are a little bit clunky to work with but there are only actually two real lines there here's what they are this objective that data set good if you give each of these two lines as much thought as you would have given one out of those 10,000 lines you're doing it wrong you have to be much more thoughtful here these two lines Express everything so it really matters what you choose to see here I think of technology is based on data and data at scale like the proliferation of magic lamps and it's nevertheless it's dangerous it's never the genie it is the wisher who is unskilled who doesn't know how to express their wish properly so that the spirit of the wish exactly matches the letter so we need to be really careful and think through what we are saying when we say those two lines and whenever anything goes wrong with AI it comes down to that a poor choice of data set or a poor choice of objective next you've got to remember that mistakes are possible and they will happen so what will you do when they happen I really like this example on Twitter this is a tweet by B Jamie he tweeted that his nest smart front door system has locked him out of the house because it was protecting the residents from Batman here's the thing mistakes will happen humans make mistakes computer systems make mistakes you should never forget that mistakes are possible and you should think like a site reliability engineer person who keeps these systems reliable if you're ever involved with these projects plan for mistakes what will you do when the mistake happens not if it happens and build safety nets so here luckily the engineers had realized that mistakes were possible and so you put a pin code in and get through the front door if they hadn't thought of that poor B Jamie would still be locked out of his house after the mistake happens is not the first time that you should start thinking that mistakes are possible and we should never completely trust any system whether it's based on humans or technology or the combination now AI lets you automate the ineffable and that means that even if you cannot express what those instructions are you can still get an automated that is so powerful that's so incredibly cool but if these instructions are so complicated that you can't even express them then you should expect to read the instructions and from that and know that the system works properly because those instructions are just too long and complicated and quite frankly boring for you to grasp so how long you know that it works the same way as you know with a human student test them carefully you don't know that you're human student does calculus or French properly by opening their brain and seeing how their brain does it I'm glad that we don't do brain surgery on our human students why would we want to do the same thing with our computer systems why not do what is sensible with complex systems like ourselves test them properly now when it comes to testing there's an easy way to mess up here's a little example involving my two cats Huxley and Tesla and some bananas this is your training set now with these systems you need quite a lot more data than just seven but we can use our imaginations imagine that this is what we put in and then we train our system there's some kind of recipe that happens that's supposed to then output the label Huxley Tesla or banana and then we're gonna test it on some examples so here's the examples and tada 100% accuracy right perfect system nope a much better test set would be a test set that the system has never seen before because things with good memories can to memorize their way to a perfect solution that is overfitting in AI is all about that's what we want to avoid and we avoid it same way as we do with human students don't give them examples they've seen before if you give them examples they've seen before you can't trust that they're not memorizing so give them new examples and yeah find out that it's not doing very well so always use pristine data for testing make sure that your system couldn't have cheated and memorize and that's why you've got to get in the habit of splitting your data and leaving some back so that the system can have new things to be tested on and a lot of mistakes that we see out there happen when people don't do this and forget that the point of computers is having really good memory and doing things quickly now even if you were to get a perfect test score here watch out avoid jumping to conclusions let's have a look at these images again what was really precious to me here is the identity of my kitties but what I didn't notice is that in the background of both my training set and my test set whenever I had Tesla there was a radiator and whenever I had Huxley but there wasn't so I might be tempted to conclude that from this great performance that this is an excellent Huxley Tesla detection system meanwhile all it's doing is finding radiators in the background so I shouldn't have that arrogance of thinking that I understand what this all means in the background here's the thing if these sorts of data are the only ones I'm ever going to use and I want this system only to work in my apartment and only on data where Tesla and the radiator always coincide who cares how it does it it's going to do the job properly and if I tested it well on those data it's going to continue to do the job properly but I tell Google engineers they should pretty much tattoo this sentence on themselves if they're going near machine learning the world represented by your training data is the only world you're going to succeed in so you're going to do great in a world where Tesla and the radiator happened together every time that doesn't mean that your system is going to work well anywhere else so it really matters what the nature of your data is and that you have tested it where you need it to work machine learning turns patterns into recipes that's what it's about let's think about machine teaching we're using examples to express ourselves which examples those examples come in data sets and data sets are pretty much textbooks for your system and textbooks have human authors so do data sets data sets are text books text books are data sets both are collected and written according to instructions made by people they are not objective they don't appear independently of humans they're authored by humans and you've heard a lot about algorithmic bias let me remind you very quickly of its definition again algorithmic bias occurs when a computer system reflects the implicit values of the of the humans who created it that will always be around folks because you can't separate the human out of technology and even if we create systems that today are completely palatable to us are we sure that in five hundred years time our descendants won't think that we were a pack of barbarians for having the values that we did so the human part is always going to be in there that's not an excuse for us to be jerks data sets have human authors if the author is a horribly prejudiced individual the student using the textbook will pick up that same prejudice we should worry about that now whose fault would that be if it happened I would put it mostly on the teacher what kind of teacher gives a textbook to a student without opening that textbook first and having a look and making sure that that is what we want our students to learn just because you have a textbook doesn't mean you're gonna have a great result it matters who wrote the textbook and what's in there so have a look and the way you do this in the AI context is you use analytics you open it up and you look this was a gentle example of bias here your author of the textbook me cared more about what happens in my apartment than elsewhere and if I were to move the cats to a different flat and try to use the same classifier it wouldn't work this is a gentle example there are much rougher examples out there as you've heard earlier in the day what can we do about this when the teacher opens the textbook and has a look and make sure that this is what we want the student learning what if the teacher has the same set of prejudices as the author then the teacher will say this is fine if we go and we'll only discover a problem when someone down the line is hurt so what's a way that we can do better what's a way that we can increase our chances of catching bad things more eyes on it first different people with different perspectives thinking about what are these examples that we are using to express ourselves what could go wrong if this is what we let our systems learn we need a diversity of perspectives here in machine learning in AI diversity is a must-have not a nice-to-have that's how you're going to think carefully really honor that question of are these examples the ones that I want my system learning from because again there are only two lines here this objective that data set and if you take those two parts really seriously that is how you are going to build a safe and effective and kind AAA system and so let's strive to amplify the intelligence of our decisions let's think carefully and unmagical e about the examples that we're using we're simply expressing our wishes with examples and it's nice when we have a lot of examples I prefer the term examples to the term data because I feel like we pronounce the data with a capital D too often this is just using examples to express our wishes let's wish responsibly let's take our examples well and where the system scales let's make sure that we get a lot of perspectives on both of those questions to keep ourselves safe so I hope that you will join me in striving for a safer and brighter AI future together thank you [Applause]

Channel: WIRED UK

Views: 8,336

Rating: 4.9069767 out of 5

Keywords: wired video, wired magazine, wired uk, wired, pop culture, science, politics, conde nast, health, technology, new technology, artificial intelligence, machine learning, AI, Barbican Centre, Google

Id: EjBXZrQ7fTs

Length: 21min 30sec (1290 seconds)

Published: Thu Aug 01 2019