AI Self Improvement - Computerphile

the stamp collecting machine we talked about last time is a physical impossibility effectively because it has a perfect model of reality or an extremely good model of reality that we just gave it by specifying it it looks through every possible sequence of output data within a year which is far too large a search space to search exhaustively and it is able to evaluate for each of this extremely huge search space a detailed year-long simulation of reality to figure out how many stamps it gets so clearly an actual computer to run this algorithm is significantly larger than in the universe so why is it even worth thinking about right and the reason it's worth thinking about is self-improvement it is reasonable to expect that a general intelligence which is not this powerful would improve itself over time and eventually become very powerful not as powerful as this but closer to this than our intuitive understandings of intelligence and the reason for that is is that intelligence in this context is an instrumental value whatever you're trying to achieve it's valuable to be more intelligent if you sense that you're not intelligent enough to come up with the best possible stamp collecting plan the first part of your plan might be spent designing improvements to yourself to allow you to come up with a better plan is the de Xin realizing it might need to modify itself a bit like somebody saying I want to be a doctor so they decide to go off and get medical degree in a sense yeah although people are not really able to change our intelligence our core intelligence we can get we can acquire more knowledge we can acquire more skills which is in a sense increasing your effective intelligence but if you could have a if you could if you could take an action which would actually increase your ability to think in all in all senses that action would be worth taking even at fairly significant cost and this is something which is true if you're trying to collect stamps but also if you're trying to do almost anything else being better at modeling reality being better at picking options better making plans better than acting those plans whatever your plan is that's worth going for and we may also have reason to believe that a general intelligence might be quite successful and proving itself human beings designed it but it's quite easy it's quite common for human beings to design things that are better at what they do than humans are right I am a very weak chess player but I could given some time and effort right a computer that would beat me at chess so it's perfectly plausible that we might right we might design an AI that is actually better than us at AI design so that it's able to make improvements to itself that we hadn't thought of machines self-improvement we don't really have a sense of the time scale and it could be extremely short right if it's just a software change you write the piece of software you run it it might think faster than a human being I mean computer clock speeds are much faster than human brain speeds it might discover improvements that it could make to itself very quickly rewrite its own code that's all in software it could do that very quickly I mean it could do that within a day it could do that within the first second of being turned on and then it becomes very interesting because there's a possibility that this process could be recursive that is to say that once it's redesigned itself it's now more intelligent it's possible that add this new level of intelligence it is better at AI design than it was and it's able to come up with more improvements that it could make to itself and so on and so on and so the thing continually increases in its intelligence and this could be quite rapid and then the question becomes is this subcritical or supercritical so there's kind of an analogy here with a nuclear material right if you've got some fissile material it's uranium or something every time the atom decays the nucleus splits releases some energy it releases three fast neutrons I think and those three neutrons go off into the material and may hit other nuclei and cause them to fuse each of which gives off through a thorough so the question is for each decay of a nucleus how many further decades happen as a consequence of that if the number is less than one then you'll get a bit of a chain reaction that then fizzles out because on average for every one that splits it's creating less than one new one if it's exactly one you get a self-sustaining reaction where the thing is like a nuclear power plant where each each thing sets off one other one on average and so the thing just keeps going steadily if it's more than one you get a runaway reaction in which the level of activity increases exponentially if each fission event results in two more then you've got two four eight sixteen thirty-two and you've got at best a meltdown and at worse a nuclear bomb and so we don't know what the shape of the landscape is around any general intelligence that we might design it's possible that each unit of improvement in intelligence brings about more than one additional unit of improvement which results in an intelligence explosion that's what they call it like the nuclear bomb in a sense you have a runaway improvement that results in a simple machine that you that you might be able to build that then fairly quickly becomes something that behaves like the stamp collecting device in terms of having an extremely good model of reality being able to search an extremely large number of possibilities and come up with good plans and evaluate those very accurately so that's why the stamp collecting device is worth thinking about even though it couldn't be built because it's possible we could build something that could build something that could build something that could build a stamp electrifies we could just pull the plug the question of pulling the plug on a device like this is how do you decide to do that I mean you don't unplug every ai right you only unplug it if it's doing something that you don't want it to be doing the stamp collecting was the the philatelist whatever who made this thing he it's quite reasonable to say at a certain point he's going to say well hang on a second I don't like stamps that much like dial it back a little and sure I shut the Machine off if he has the capability to shut the Machine off then the fact that the machine will be shut off is included in the internal model of reality right the stamp collecting machine understands why it was made it understands what its creator wanted it to do and it understands if the Creator is able to turn it off it understands that as well so it's never going to create a situation in which the Creator both wants to shut it off and is able to shut it off so if the if the Creator is watching it very carefully at all times it will never do anything that the Creator will realize is a problem until it's very confident that it's safe and then it will go wild right so it we wouldn't just go crazy immediately and you would freak out and pull the plug it would see that coming it's we've just decided it's very intelligent right if you can pull the plug on it it's not very smart so it might in the background it might run a few small I mean obviously I'm talking here I'm completely speculating one thing it might do it might win a few auctions for stamps keep the stamp collector happy at the same time be hacking into various machines in various parts of the world building copies of itself installing itself in various places until it's confident that even if people want to turn it off they're not able to and then it starts revealing actually it's actually interested in producing the most stamps possible at the expense of everything else intelligent systems have a quite interesting property which is that they are very predictable in outcomes or being very unpredictable in actions so for example if I'm playing a very good chess player I'm playing it to me versus Kasparov right and you're watching you you if you're a reasonably strong chess player might be able to predict what I'm going to do because I'm not very good you couldn't predict what Kasparov is going to do because he's a better chess player than you if you could predict what he would do you would be as good at player as him right because you could just make the moves you think he would make so in a sense his intelligence has made him much less predictable but in another sense it's made him much more predictable because you can now accurately predict well in a sense you can you can predict the future state of the board you don't know at any point which move he's going to make but you know that sooner or later and probably sooner I'm going to be in checkmate right because you know he's very intelligent in the domain of chess and you know that he wants to win at chess from these two you can predict that he will win at chess even if you don't know what he's going to do and it's the same with the stamp collecting device we can't predict what it would do but we can predict that it will result in a lot of stamps and that's all you need right as long as you can imagine a scenario with more stamps that's the one that's going to come out we'd like to thank little bits for supporting computerphile do not come across little bits they're an easy way to learn and prototype electronics modules range from buttons and buzzers and lights and LEDs all the way through to Wi-Fi enabled technology as well so I was using Wi-Fi module yesterday and have great from controlling the little bits from my phone remotely through the internet if you're interesting that kind of thing get over to little bits comm put in the promo code computer file and you'll get $20 off your first purchase so thanks once again to little bits for support in computerphile get over to little bits com2 check out what they've got back to my buzzer so it needn't use his credit card it needn't use money at all so it might send out an enormous number of emails to all the stamp collectors the world

Channel: Computerphile

Views: 391,876

Rating: 4.9565983 out of 5

Keywords: computers, computerphile, Artificial Intelligence (Industry), computer science, robert miles, agi, artificial intelligence, general artificial intelligence

Id: 5qfIgCiYlfY

Length: 11min 21sec (681 seconds)

Published: Fri Jul 24 2015

Reddit Comments

[u/TikiTDO]

I always like videos with this guy, he seems to be pretty on the ball. I would love to be able to sit down and actually discuss some theories with him.