You and AI – the history, capabilities and frontiers of AI

[Music] well hello everybody and welcome to the Royal Society my name's Andy hopper I'm treasurer but perhaps of relevance to this I'm professor of computer technology at Cambridge University as well so this society has been around for a little while and over those centuries 350 something years it has played its part his fellows have played its part in some of the most important discoveries and actually practical use over the years as well and in April 2017 about a year ago we launched a report on machine learning actually it's a series of reports what might be called the digital area on cybersecurity on machine learning actually on teaching computer science that sort of thing however our report on machine learning called for action in a number of areas over the next thirty years we use the phrase careful stewardship in relation to machine learning data and that sort of things to ensure that the benefits of this technology are felt right across society and that we encourage facilitate participate in a public debate on this broad topic and discuss how the benefits are distributed as well as trying to think ahead of some of the perhaps majors and other other things this series of events and lectures which is supported by deep mind we hope will help develop a public conversation about machine learning AI and so on and provide a forum for a debate about the way these technologies may actually will already do affect the lives of everybody on the planet so it's great to see her here at what is our first event and so we have demis hassabis a superstar to give our first lecture in this series and I'm very pleased to say he was an undergraduate in my department so you know boy damn good we like we we like that sort of thing in in my part well everywhere else I guess as well so that's good but then he went on to a PhD in neuroscience at UCL our interesting thing which you'll see actually comes together in his work and then he did a couple of things but co-founded deep mind in 2010 he's very distinguished he's received a fellowship of the Royal Academy of Engineering also the silver medal and is a fellow of the Royal Society of Arts as well in 2014 deepmind was acquired by Google and has grown enormously I know it retains the name deepmind which i think is very interesting positive but also has activities in Edmonton Montreal and an applied team in Mountain View so dennis has done well to the point out for example on the one hand time listed him as one of the 100 most important people he friendship sort people in the world but also he was awarded a Seabee for services to science and technology so welcome Demi's and we look forward to all our minds being improved on your favorite topic thank you very much well thank you Andy for that very kind introduction and thank you all for taking the time to come this evening it's great to see you all here so we're very proud of deepmind to be supporting this very important lecture series here at the Royal Society you know we think that given the potential of AI to both transform and disrupt our lives we think it's incredibly important that there's a public debate in public engagement between the researchers at the sort of forefront of this field and the broader public and we think that's very critical going forwards as more and more this technology comes to affect our everyday lives so what we're hoping here and the idea behind them with this Royal Society lecture series is to sort of open up a forum for to facilitate a kind of open and robust conversation about the potential and the possible pitfalls inherent in the advancing of AI so I look forward to answering all your questions at the end of the talk so today I'm going to to talk about AI but specifically focused around how AI could be applied to scientific discovery itself I thought this particularly appropriate given this is a lecture at the Royal Society but it's also the thing that's I'm most passionate about so this is the reason why I've spent my whole life and my whole career on trying to advance the state of AI is that I believe if we build AI in the right way and deployed in the right way it can actually help advance the state of science itself so I'll come back to that theme throughout this talk so to begin with the kind of way you know there's no exact definition of what AI is but a kind of loose heuristic I think that's worth though that's kind of worth keeping in mind is AI is the sort of science of making machines smart that's what we're trying to do when we embark on this endeavor of building AI and deep mind itself my company we founded it in London in 2010 we can became part of Google in 2014 but we still run independently right here in Kings Cross just up the road and the way to think about deep mind and the vision behind it was to try and bring together some of the world's top researchers in all the different Sublett disciplines that were relevant to AI from neuroscience to machine learning to mathematics and bring them together with some of the world's top engineers and a lot of compute power and to see how far could we push the frontiers of AI and how quickly could we make progress so you can think of it as like an Apollo program effort for AI and nothing until that point until we found a deep mine existed that was really set up to do this in this way another explicit thing behind this the vision that we had for deep mind was to try and come up with a new way to organize science and and what I mean by that this would be a whole lecture in itself is could we fuse together the best from the top academic labs and the blue sky thinking and the rigorous science that goes on in those places with the kind of a lot of the philosophy behind the best startups or best technology businesses in terms of the the amount of energy and focus and pace that they bring to bear are on their missions so would it be possible to kind of fuse together the best from both of these two worlds and that's the way you can think about the culture at deep mine is a kind of hybrid culture that mixes the best from both for those two two fields now what is our mission at deep mind well we articulate it as a kind of two-step process and a slightly tongue-in-cheek but we mean we take it very seriously but so this is how we do articulated step one fundamentally solve intelligence and then we feel if we were to do that then step two would naturally follow use it to solve everything else and so what we mean by solve intelligence is actually to just unpack that slightly is to fundamentally understand this phenomenon of intelligence what it is what kind of process is it and then if we can understand that can we recreate the important aspects of that artificially and make it sort of universally abundant and available so that's what we're what we mean this solve intelligence first part of this mission and I think if we are to do that in a general way both deep mine and and and the research community at large then I think naturally step two will follow in terms of we can bring this technology to bear on all sorts of problems that for the moment seemed quite intractable to us so things like you know perhaps as far afield as climate science all the way to curing diseases like cancer that we don't know how to do yet I think AI you could have a role to play an important role to play as a very powerful tool in all of these different scientific and medical and endeavors so that's the high level mission and that's how a guiding star at deep mind but how do we go plan to go about this more pragmatically so what we talk about is trying to build the world's first general-purpose learning machine and the key words here obviously learning and general and so all the algorithms that we work on at deep mind are learning algorithms and what we mean by that is that they learn how to master certain tasks automatically from raw experience or raw input so they find out for themselves the solution to tasks so they're not pre-programmed with that solution directly by the programmers all the designers instead of that we create a system that can learn and then it experiences things and data and then it figures out for itself how to solve the problem the second word general this is this notion notion of generality so the idea that the same system or same single set of algorithms can operate out of the box across a wide range of tasks potentially tasks that it's never even ever seen before now of course we have an example of a very powerful general-purpose learning algorithm and it's our brains the human mind and and our brains are capable of course of doing both of those things an exquisite example of this being possible and up till now you know our algorithms have not been able to do this so the best computer science has to offer has fallen and it still is way short of what the mind can do now internally at the mind we call this type of AI artificial general intelligence or AGI to English it from the traditional sort of AI that's been you know AI is a field has been going for 6070 years now since the time of Alan Turing and and you know a lot of traditional AI is handcrafted so this is specifically researchers and programmers coming up with solutions to problems and then directly codifying those solutions into in terms of programs and then the program itself the machine just thermally execute the program the solution it doesn't it doesn't adapt it doesn't learn and so the problem with those kinds of systems is they're very inflexible and they're very brittle if something unexpected happens that the programmers didn't cater for beforehand then it doesn't know what to do so it just usually just catastrophic ly fails and this will be obvious to you if you've ever interview know interacted with assistants on your phone often you know they'll be fine if you stick to the kind of script that they already understand but once you start conversing them freely you very quickly realize there's in any real intelligence behind these systems they're just template based question answering systems so by contrast the hallmarks of a GI systems would be that they're flexible and they're adaptive and they're robust and what gives them this kind these kind of properties are these general learning capabilities so in terms of rule-based day oriented or traditional AI that probably the still most famous example of that kind of system was deep blue.the ibm's amazing chess computer that was able to beat the world chess champion at the time Garry Kasparov in the late 90s and these kinds of services are called expert systems and and they're pre-programmed with all sorts of rules and heuristics to allow them to be experts in the particular type of task that they were built for so in this case deep blue was built to play chess now the pawn with these systems is and what you can quickly see is that deep blue was not able to do anything else other than play chess in fact it couldn't even do something simpler like player a strictly simpler game-like noughts and crosses it would have to be reprogrammed again from scratch so I remember I was doing my undergrad in Cambridge actually when this match happened and I remember coming away from this match more impressed with Garry Kasparov's mind than I was with deep blue and that's because of course deep blue is an incredible technical achievement and a big landmark in AI research but Garry was able to more or less compete equally with this brute of a machine but of course Garry Kasparov can do all sorts of other things engage in politics talk three languages and write books all of these other things that deep blue had no idea how to do with this single human mind so Tim to me they felt like there was something critical missing from if this was intelligence or AI something missing from the deep-blue system and I think what was missing was this notion of adaptability and learning so learning to cope with new tasks or new problems and this idea of generality being able to operate across a wide range of very differing tasks so the way we think about AI at the mind is in the framework of what's called reinforcement learning and I'll just quickly should explain to you what reinforcement learning is with the aid of this simple diagram so if you think of the AI system and we call the the AI systems agents at at deep mind internally deep mind here on the left picked up by this this little character and this agent finds itself in some kind of environment and it's trying to achieve a goal in that environment a goal specified by the designers in that environment now if the environment was the real world then the agent you can think of the agent as a robot so a robot situated in a real-world environment alternatively the environment could be a virtual world like a game environment which is what we mostly use at deep mind and then in that case you can think of the agent as like a virtual robot kind of avatar or game character now in either case the agent only interacts with environment in two ways firstly it gets observations about the environment through its perceptual inputs we normally use vision but we are starting to experiment with other modalities like sound and touch but for now almost everything we use is vision input so pixel input in the case of a simulation and the first job where the agent is to build a model of the environment out there statistical model of the environment as accurately as it can based on these noisy incomplete observations it's getting about the world out there so never has full information about how this environment works it can only surmise and approximate it through the observations and the experience that it gets the second job of the agent is once it has that model of the environment and is trying to make plans about how to reach its goal then it has a certain amount of time to pick the action it should take next and from the set of actions available to it at that moment in time and it can do a lot of planning and thinking about if I do a how action a how will the world look how the environment change my direction B how will it change which one of those actions will get me nearer towards its goal and once it's run out of thinking time it has to output the action the best action it's found so far that actually gets executed that may make a change or not make a change to the environment which will then drive a new observation so this whole system continues around in a kind of feedback loop and all the time the agent is trying to pick actions that will get it towards its goal ultimately towards its goal now this is that's reinforcement learning in a nutshell and how it works now we this diagram is pretty simple but there's a lot of very complex technicalities behind trying to solve this reinforcement learning problem in the fullest sense of the word but we know that if we could solve all of these issues technical issues that with this framework is enough to deliver general intelligence and we know that because this is the way biological systems learn including our human minds so in the in the in the primate brain and the human brain it's the dopamine neurons the dopamine system in our brain that implements a form of reinforcement learning so this is one of the methods that humans use to learn the second big key piece of technology that's that's that's created this sort of new renaissance in AI in the last sort of decade is called deep learning and deep learning is to do with high rock your networks so they're kind of you can think of them as loose approximations to the way our neuron knurled our real neural networks our work in our brain and here's an example of a neural network working so imagine that you're trying to train one of these neural networks here on the right this these layers of neurons to distinguish between pictures of cats and dogs so what you would do is you would show these this this AI system many thousands perhaps even millions of different pictures of cats and different pictures of dogs and this is called supervised learning so what you would do is you'd show them a pictures usually you'd show the input layer at the bottom here this picture the raw pixels from this picture of a cat or a dog and then it would the the this neural network would process that picture and then would ultimately output a label either a guess saying I think that's a cat or guess saying I think that's a dog and depending whether it was correct or incorrect you would adjust the neural network adjust the weights between these neurons so that next time you get asked a question about this is the catalog this is a dog you're more likely to output the right answer so and it uses an algorithm called back propagation to do that so it goes back and adjust then your network weights depending whether you got the answer right or wrong so that you're more likely to get the answer right next time and once you do this this this incremental learning process many thousands perhaps even millions of times eventually you get in your network that is really amazing at distinguishing between pictures of cats and dogs if that better than I am so I actually can't tell whether that's a cat or a dog from that particular picture so we our one of our big innovations of deep mind was to pioneer the combination of these two types of algorithms so so we we call this combination while the logically deep reinforcement learning and we use deep learning to process the perceptual inputs to process the observations and make sense of the world out there these these visual inputs that the system is getting and then we use reinforcement learning to make the decisions to pick the right action to get the system towards its goal so we pioneered this sort of field and one of the big things that we we should we demonstrated was we built the world end-to-end learning system and it's called dqn and what we mean by end-to-end is it went all the way from perceptual war perceptual inputs in this case pixels on a screen to making a decision about what action to take so really it was an example of one of these full systems that can go all the way to from processing the vision to making a plan and executing that plan and what we tested it on was atari games that was the first thing we tested on was atari games from the 80s and we tested it on 50 classic games those of you in the audience who are old enough to remember these games which is probably not many of you they were space invaders pac-man these kinds of games that I'm showing here at the bottom and I'm going to show you that the the dqn system how it learnt and how it progressed through its learning in a second in a video and the next slide but just before I show that I just wanted to be clear what you're going to see so the only input that the dqn system gets is the 20,000 pixel values on the screen so so those are that those are the inputs that it gets just these pixel numbers it doesn't know anything about what it's supposed to doing what it's controlling all it knows is these are the pixel values and you've got to maximize the score that was the goal it has to learn everything else from scratch so the architecture we use is is is here on the screen here so this is a neural network you can think of on its side and so on the left-hand side you can see there's the current screen being a B and the pixels on the screen being used as the input then it gets processed through a number of layers and then at the output you've got a whole bunch of actions that can be taken I think it's 19 actions that can be taken the eight joystick movements the eight Joystiq movements with the fire button or doing nothing and so it's got to make a decision about any of those actions to take in the next time step based on the current input screen input so this is how it works on the classic game breakout breakout is one of the the most famous games in Atari games and here in this game you control the the bat and the ball the pink bat at the bottom of the screen and what you're trying to do break through this rainbow color brick wall brick by brick and you're not supposed to let the ball go past your bat otherwise you lose a life so this is I'm going to show you this video now of the agent learning after many hundreds of games of play so this is dqn after 100 games so you can see it's not very good agent yeah it's missing the ball most of the time but it's starting to get the idea that it should move the bat towards the ball now this is after 300 games - 200 more games experience and now it's got pretty good at the game it's about as good as any human can play the game and it pretty much gets the ball back every time even when the ball is going very fast at very a very vertical angle but then we let the the the system carry on playing for another 200 games and then it did this amazing thing which was it figured out that optimal strategy was to dig a tunnel round the left-hand side and then put the ball behind the wall so of course this gets it more reward for less risk right and of course gets rid of the rainbow-colored brick wall more quickly and that was for us really our first big sort of aha moment what watershed moment at deep mind this is now from four or five years ago and we realized we were onto something with these kinds of techniques it was able to discover something new that even the programmers and the brilliant researchers of that system did not know how to do we didn't know what we had you know haven't thought about that solution to the game so then more recently a couple of years ago we started work on what is probably still a most famous program program called alphago and alphago was a system to play the ancient board game chinese board game of go so this is what go looks like for those who don't know and this is what they play in China and Korea in Japan instead of chess and go is actually very simple game there's only two rules basically and I could teach you with it in five minutes but it takes at least a lifetime sometimes some would say many lifetimes to master the game and the aim of the game is the game ends this is a position from the end of the game people so there's two players but black and white and they take turns putting stones on the board and eventually in the game the ball fills up like this you end up counting how many areas of territory did you wall off with your stones and the person that has the side that has walled off the most amount of territory the most amount of squares with their stones wins the game so in this case it's a very close game and white wins by one point now the question is why is this so hard go so hard for computers to play you know I just sold you the beginning of the talk that chess was solved was was was cracked sort of twenty years ago and then since then go has been one of the Holy Grails for AI research and it's much much harder and there's two real reasons two main reasons why go has been much harder than chess one is the huge search space that you need to set the huge number of possibilities in go so there are actually 10 to the power 170 possible positions in go which is way more than there are atoms in the universe there's about 10 to the power 8 II atoms in the observable universe so what that means is if you ran all the world's computers for a million years on calculating all the positions you still wouldn't be haven't have calculated through all the possibilities in go there's just too many to do through brute force and the second and the even harder thing about go is that it was thought to be impossible to explicitly write down by hand and what's called an evaluation function so that's a function that takes a board position and tells the computer which side is winning and by how much and that's a critical part of how the chess programs work that's why deep blue is so powerful a team of chess grandmasters with brilliant programmers at IBM put came together and the program has distilled what was in the minds of the chess grandmasters and try to distill that into an evaluation function that would allow the deep-blue system and his successors to evaluate whether the current position was good or not and then that's what's used to plan out what move you should take and in go this is thought to be impossible because that the game is to e so Taric that sits to almost artistic in a way to be able to evaluate in that sense with hard and fast rules and if you talk to a professional go player they'll tell you the game is a lot more about intuition and feeling than it is about calculation which is a game more like chess which is more about explicit calculate and planning so we made this big breakthrough with alphago and the way we were able to do this is we tackled those two problems this this problem of combinatorial explosion and huge search spaces and this problem of evaluation function with two neural networks so the first neural network we used was called a policy Network and what we did here was we fed in board positions from strong amateur games that we downloaded off the internet and we trained in your network to predict the next move the human player would make so in blue here is the board the current board position with the black and white stones on it and then what the output is another board but here with probabilities that alphago thought for each possible move in the position so the higher the green bar the higher probability would give to a human player playing that move and what this had policy network allowed the system to do is rather than look at all the possible moves in the current position and then all the possible replies to those possible moves you can imagine how quickly that escalates it can instead look at the top three or four most likely and most probable moves rather than the hundreds of possible moves that you could make so that massively reduces down that the breadth of the search tree the second thing we did was we created of what's called of we call a value network and what we did is we took the policy network and we played it against itself millions of times so alphago played against ourselves millions of times and we took get random positions from the middle of those games and we of course know the result of the game which side won and we trained alphago to make a prediction about from the current position about who would end up winning the game and how certain alphago was about its prediction so and eventually once we trained it through millions of positions it was it was able to create a very accurate evaluation function this value network and what this value Network did is took a current board position again in blue here at the bottom of the screen and output a single real number between zero and one and zero men white was going to when 100% chance hun percent confidence in that a one would mean black was going to win hunt and confidence and that and point five would mean the position alphago dries the position to be equal and so here if by combining these two neural networks we solved all of the hard problems inherent in computer go and what you'll notice instead of us building an explicit evaluation function like they do for chess programs you know typing in all these hundreds of different rules so in fact modern chess chess computers have you know the order of about a thousand specific rules about chess and about positions in chess instead of that we didn't have any explicit rules we just let the system learn for itself through experience by playing the game against itself many thousands indeed millions of times so once we had this system we decided to challenge one of the greatest players in the world and incredible south korean grandmaster called Lisa doll and I described in his the Roger Federer of go because that's the equivalent position he occupies you know he's won 18 world titles a bit like Grand Slams and he's considered to be the greatest player of the past decade and we challenged Lisa doll to a match a 1 million dollar challenge match in South Korea in Seoul in two back in 2016 and it was an amazing you know once-in-a-lifetime experience actually in the whole country pretty much came to standstill one thing you got to know about South Korea is they love AI they love technology and they love go so for them this was like the biggest confluence of all their the things they find exciting all together and Lisa Dahl is a is a sort of national hero there he's equipment of like you know David Beckham or something with us so so that's so that that was an incredible experience you know this is this is a picture of the top left of the first press conference you know is literally a huge ballroom full of full of journalists and TV cameras and you know there was over 200 million viewers across Asia for the five-game match which was incredible and alphago we won for won the match and you know it was hugely unexpected even just before the match Lisa Dahl was asked to predict what he thought was going to happen he put it to the niel victory for himself or for one at minimum and in fact it was proclaimed to be a decade before its time both by AI experts including computer go experts and and also go players and the go world and the important thing here was not just the fact that alphago won but actually it was how alphago one that was the critical thing so alphago actually played lots of creative completely new moves and and came up with lots of new ideas they're astounded the go world and in fact are still being studied now you know nearly two years later and i revolutionising the game so it's not a question of alphago just learning about human heuristics and and and motifs and then just regurgitating those motifs reliably sort of regurgitating them it actually created its own unique ideas and here's the most famous example of that i just want to quickly show you this is move 37 in game 2 and in go there is a whole history goes bingo is been around for 3,000 years and and was played professionally for several hundred years in Japan and China and other places and there is this notion go of famous games that are looked all back on and studied for hundreds of years and indeed famous moves in those famous games sort of go down in history and this is considered to be you know gonna follow in that lineage this this move move 37 from game 2 and this is the move here on the right hand side and the alphago here is black and lisa dollars white and when alphago played this move lisa dole sort of literally fell off his chair and the reason is and all the commentators commentating it thought this was a terrible move and and the reason for that is that in the early parts of go in the opening phases of the go game you normally play on the third and fourth lines so go is played on a nineteen by a nineteen board and you normally play on the third and fourth lines and that's the kind of accepted wisdom of how you should play in the opening those are kind of the critical lines but here you'll notice that alphago played this relatively early move move move 37 still very early in the game on the fifth line so one flying further up and this is normally considered to be a huge mistake because you're giving white your opponent huge amount of territory on the side of the board so it's considered to be sort of a very weak move so this sort of thing no professional would ever consider playing and the key thing about what Africa did here is that it played this move and the thing about go is it's can in in Asia it's considered to be kind of like an art form but it's sort of objective art because you know later on any one of us could come up with an original move we could just play around and move and it might be original but the key thing of whether is is did it make a difference and impact the game the result of the game that's what determines whether it's a kind of beautiful and truly creative move and in fact move 37 did exactly that because you'll see the two stones here that I've outlined in the bottom left there surrounded by white stone so they're in big trouble but later on about hundred moves later on the the fighting that was going on in the bottom left-hand corner spilled out into the center of the board ran across all the way across the board and ended up those two stones down the bottom left ended up joining up with that move 37 stone and it was that that that move 37 stone was in exactly the right place to decide that whole battle which ended up winning alphago that game so it was almost as if alphago placed that stone presently a hundred moves earlier to impact this fight elsewhere off the board at exactly the right moment so so this was really you know quite an astounding moment for Guren and computer Co Lisa tholins stuff was incredibly gracious and their absolute genius and what was really amazing was he won a game and it was an incredible game that he won he made an amazing move - and he said after he was very inspired by the match you know I realized it was a really good choice learning to play go this is amazing sort of the reason he played go and it's been an unforgettable experience and he actually went on a three-month unbeaten winning streak in in human championship matches after this match with alphago and he was trying out all sorts of new ideas and techniques and if you're injured in that I'd recommend to you if you want to see the behind the scenes story I recommend you watch this documentary that was done by an independent filmmaker and won all sorts of awards at film festivals that's now available on Netflix which which will really give you sort of behind the scenes look at how alphago was created and what went into it so since then we've continued working on these kinds of systems and and now we've created a new program called alpha zero which advances what we did in alpha going takes it to the next level so what we've done without 4-0 and we just released this just before Christmas or as we generalized alpha go to be able to play not just go now but any two-player game including chess and shogi which is the Japanese version of chess both of which are played professionally around the world and and the second thing we did to generalize it further so it plays more than one game so don't forget this gets at the notion of generality so that was something I I criticized about deep blue deep blue could only that program could only play chess well alpha zero can play any two-player game the second thing is that we we remove this need right now for go if you remember what I said about the policy network is it first trained to mimic human had strong amateur players that we we've shown it from the internet but instead of that where alpha zero does is it starts completely from scratch so it's it can only relies on self learning playing against itself so it starts off when it begins totally randomly so knows nothing about the game or anything about what a good moves or all likely moves it has to learn all of that literally starting from random so it doesn't require any human data to to bootstrap its learning and we tested this program in chess of course there are many already very very strong chess programs way stronger than the human world champion the current top program is called stock fish and it's an open source program and you can you can think of it as the descendant of deep blue twenty years later so it's it's way way stronger now and you can run it on your laptop and it's so strong no human player hat would have a chance of beating it and it's in fact many my chess player friends and I used to chess when I was a lot younger I thought that stockfish could never be beaten like that was that that was the limit at which chess could be played and amazingly alpha0 after just four hours of training so it started off random and then after four hours of this self play self playing and a few million games it was able to beat stockfish twenty-eight nil with 72 draws in a hundred game match so there's really quite astounding results again for the AI world but also for the chess world and we're actually going to publish we've just released preliminary data on this and we're going to publish this on in a big peer-reviewed journal in the next few months and again here just like we'd go where it came up with these new motifs the you know playing on the fifth line in the opening that four thousands over sort of overturned thousands of years of received wisdom human wisdom here in chess even more quickly more amazingly was that it created it seems to invented a new style of playing chess you know and and the summary of that is that it favors mobility and the amount of mobility your pieces have over materiality so in most chess programs you know the way that you write rest codes one of the first rules you you input into a chess program is the value of the pieces you know rook is five points Knight is three points Bishop is three points and so obviously you don't want to swap your rook for a knight because that's minus two points all right so that's one of the very first things were put into the very first chess computers those kinds of rules and what an alpha zero actually is very contextual so in certain positions it will be very happy to sacrifice material to gain mobility so the remaining pieces it has to increase their power on the board and what that means is it can make incredible sacrifices to gain positional advantage really long-term sacrifices and and we released ten sample games from this 200 game match and these are being poured over by chess grandmasters at the moment and there's lots of great YouTube commentaries on this if you're an amateur chess player your interests in chess I recommend you you you you you have a look at a few of these great commentaries or on YouTube that talked about why these games this style this alpha zero star is is so interesting and what's it and the secondly interesting about it is that a lot of these professional chess players commented on how alpha zero seemed to have a much more human style than the top chess programs that have a such much more kind of mechanical style and it's a little bit ugly to the human eye the way that computer chess programs sort of play until now now so these are some of the the breakthroughs that we've had and there are many other breakthroughs and many other domains from from other groups around the world and AI right now is you know become a huge buzz word and with a massive amount of progress has been made in the last five to ten years but I don't wanna give you the impression is that we're anywhere close to yet to solving AI there's actually tons of key challenges that remain in fact it's a very exciting time in some senses I feel like we've all we've done is is dealt with the preliminaries and now we're getting to the heart of what intelligence is and I'll just give you a little taste of some of the things that that I'm personally thinking about and that my team is and each one of these things would be a whole lecture in itself and indeed I think some of the other lecturers in the speakers in this lecture series will probably cover some of these topics so unsupervised learning is a key challenge that is not solved yet so this is what I've been showing you is supervised learning where like the cats and dogs where I tell you that system the answer so that it tries to figure out how to adjust itself so that it's more likely to get the right answer and I've also showed you about reinforcement learning where you get a score or reward so in go you get the machine gets a won for winning and a zero-zero reward for losing right and it wants to get reward but what about the situation where you don't have any rewards and you also don't know the correct answer which in fact is most of the time in fact when we do human learning and babies learn most of time they're not getting any feedback and yet they're still learning things for themselves so how does that happen so that's called unsupervised learning second thing is memory in one-shot learning so what I've shown you is systems that are in the moment so they process currently what's in front of them they make a decision in the moment and then they execute that decision what of course to have true intelligence you need to remember what you've done in the past and how that affects your future plans right and you also need to be able to learn much more efficiently so I've told you that alphago you know alphago needs to play millions of games against itself to learn to get to this level but humans can learn much more quickly right we are able to learn things sometimes in one shot just one example and that's enough and that's something both of those things that are kind of related and actually this is what I studied for as Andy mentioned in for my neuroscience PhD was how the brain does this and it's actually a brain error called the hippocampus which is what I studied for my PhD and is critical to both one-shot learning and episodic memory another thing is imagination based planning so so one thing is to sort of plan by trying out possibilities like in chess or go you know it's quite simple go although guys have got a lots of possibilities the game itself dynamics is very simple you know you the rules are simple you know what will happen if you make a move then how the next state will look of course the real world is much more complicated complicated and that is is not easy to figure out what's going to happen next when you when you make an action so this is where you know imagination comes in this is how we make plans as humans is we imagine viscerally like how we might want to you know job interview to go or a lunch hour or a party or something like that we actually kind of visualize it in our minds and then that allows us to just uh what if I if I said this thing or if I did this thing then how would this other person react and so on and we play these through these scenarios through in our minds before we actually get to the situation and that's extremely efficient way to do planning and it's something that we need in our AI systems learning abstract concepts so what I've shown you here is implicit knowledge so kind of figuring out what this perceptual worlds about but what we really need to learn is about abstractions our high-level concepts and eventually things like language and mathematics which we're nowhere near currently transfer learnings another key thing which is where you take some knowledge we've learned about it from one domain and you apply it to a totally new domain so that might look perceptually completely different but actually underlying structure of that domain is the same as some other domain that you've experienced again our computer systems are not good at doing this kind of learning but humans are exceptionally good at this and then finally of course all the things I've shown you here games Atari games Go games chess games none of them yet involved language which as we all know is key to intelligence so that's a whole field that still needs to be addressed with these kinds of techniques so I just want to talk a little bit now about how this is being already applied even the systems we have today so there's many challenges to come but I think already the systems we have today can be usefully used in science in fact we've seen that by work we've done some work we've done and many other groups are using some of these systems I already talked about deep learning and reinforcement learning in all sorts of very interesting scientific domains so there's being used to discover new exoplanets by analyzing data from telescopes AI systems are being used for controlling plasma in nuclear fusion reactors we've been working on and others on how it can help with quantum chemistry problems and also it's being used a lot in healthcare domains so actually we have a partnership with more fields to help the radiographers quickly triage retinopathy scans or scans of the retina to look for macular degeneration so it was very very Forks we need diverse fields and I could have you know done many slides on different applications that are currently going on with AI and I think this is just the beginning one of things I'm most excited about is applying it to the problem of protein folding so this is the this is the problem of you get an amino acid sequence 2d sequence of the protein structure and you need to figure out the 3d structure the protein will eventually fold into and that's really key to a lot of disease and drug discovery because the 3d structure of the protein governs how it will behave so this is a huge sort of long-standing scientific challenge in biology and we're working quite hard with a project team on on this there with some collaborators from the Crick other scientific applications I see coming up is helping with things like drug design the design of new materials and in biotechnology in areas like genomics and in fact if I was to boil down the kinds of problems the properties of problems that are well suited to the AI we already have today let alone what we're going to create in the future I think it comes down to three pop key properties property one it's got to be a massive combinatorial search base so so that's kind of got to be inherent in the in the problem secondly can you specify a clear objective function or metric to hillclimb against to optimize against it's almost like a score if you like a score in again you have to be able to have some kind of score of how well you're doing towards the final goal and then you either need lots of data to learn from actual real data or an accurate and efficient simulation or simulator so you can generate a lot of data in the way that we do with our game systems so as long as you satisfy those three constraints properties I think we all the a our systems we already have today could potentially be usefully deployed in those areas and I think there's actually a lot of areas in science that already would fit these these these desired properties and then of course there's all sorts of applications to the real world that we're working on in combination with Google including with healthcare we work with the NHS in many projects you're making me assistant on the phone more intelligent and also in areas like education and personalized education and and I think AI is set to revolutionize a lot of these other sectors so just to sort of sum up now you know one of the reasons that I've spent my whole career about on AI is the I've always felt that it's a kind of meta solution to many other problems that that face us today you know if you think about how the world is today one of the big challenges is the amount of information that we're confronted with and that we're producing as a society so and I mean that both in our personal lives in terms of like choosing you know our entertainment to science where there's just so much data now being produced from something like CERN or in genomics you know how do we make sense of it all and indeed the systems that we would like to better and have more control over our incredibly complex systems you know think about climate or the nuclear fusion systems now these incredible lead complex systems that are up in some cases are bordering on chaos systems and so they're very difficult for us to describe with equations and to understand even the top human scientists and experts so you know for a long time big data was the buzzword you know before AI was is now the buzzword AI was the blue you know big big data was the buzzword and I think that actually in a way big data can be seen as the problem you know if you think about it from an industry point of view everybody you know all companies have tons of data now and talk about big data the problem is how do they get the insights out of that data and how do they make use of all of that data so to be useful to their customers and their clients and so on and I think AI is the answer to help find the structure and insights in all of that unstructured data and in fact you can think of I think one way to think about intelligence is as a process an automatic process that converts unstructured information into useful actionable knowledge and you know I think AI could be sort of help us automate that process and for me my personal dream and a lower dream of my team is to make a I assisted science possible or even perhaps create AI scientists that can work in tandem with their human expert counterparts and from a neuroscience point of view one of my dreams as well is to try and better understand our own human minds and I think building AI in this neuroscience inspired way and then and then sort of comparing that that construct that algorithmic construct with the way the human mind works will potentially shed some light I think on some of the unique properties of our own minds things like creativity dreaming perhaps even the big question of consciousness so to sum up then I think you know AI holds enormous promise for the future and I think these are incredibly exciting times to sort of be alive and working in these fields but you know this is where all this sort of potential also comes a lot of responsibility and I just want to mention this and I think some of you will probably have questions about this later you know we believe a deep mind as with all powerful tech oh geez eh I must be build responsibly and safely and used for the benefit of everyone in society and we've been thinking about that from the very beginning of deepmind and this requires lots of things that were actively engaged with right now with the wider community research on the impact of the technology how to control this technology and deploy it and we need a diversity of voices both in the development and the use of the technology and meaningful public engagement which is why we're so happy to be supporting this lecture series and we've just launched our own ethics and society team at deep mind that's involved in working with many outside sort of stakeholders to figure out the best way to go about deploying and using these types of technologies to benefit everyone in society and we've also been involved on industry scale across the whole field in co-founding the partnership on AI which is a cross industry collaboration with problems for profit and nonprofit companies so the biggest companies the world coming together to talk about this and try and agree some best practices and some and and some protocols around how to how to research this technology and how to engage the public with it and all this is happening you know for us right here in the center in the heart of London you know our home we're very proudly British company and you know we work here at Kings Cross with our colleagues at the Crick Institute and that Alan Turing Institute which is based in the British Library all around and Kings Cross is becoming quite a hotbed and UCL of course is round there of a a I research and you know the and dimension of the starts we should leverage in the UK all of our incredible strengths these amazing universities that we have here Oxford Cambridge UCO Imperial and others that have incredible strengths in computer science and we you know I feel very strongly that deep mind needs to play its part in encouraging and supporting this AI ecosystem through sponsorship scholarships and internships and and actually lectures given by deepmind staff and I'm very passionate about establishing the UK as one of the world leaders in AI and I think we have an amazing position and we've we should really be building on our heritage in computing that starts with you know actually Charles Babbage inventing computing really hundred years before we started before before it's time in some senses and then of course that continued with Alan Turing who famously laid down the fund menthols of computing in the 40s and 50s then the World Wide Web you know with with people like Sir Tim berners-lee instrumental in creating the Internet and I feel like you know the next thing in the lineage of those of those types of technologies artificial general intelligence and I think the UK has a huge part to play in then I hope deep mind will will play its part in that too so you know it's great to be opening this this series of lectures you know I think we we need to capitalize on on on what we have here in the UK both we're you know from the ethical side and the technological side and one thing I would say is that it's important for us to be at the forefront of Technology if we want to have a seat at the table when it comes to influencing the debate on the ethical use of this technology and again I would encourage you all to get involved the public you know understand these technologies better and how they're going to affect society and then engage on how you would like to see these technologies deployed for the benefit of everyone you know I think AI can be an incredible benefit to society if we use it responsibly I think it could be one of the most exciting and transformative technologies we'll ever invent thanks for listening so folks we have time for questions and and so on but let me start off with a question just to get things going you've outlined some of the technological possibilities what are they barriers what are the difficulties what stands in the way on the you know deployment the use the science of this some of the things you've talked about yeah I think you know I mentioned in the slide about what the remaining challenges are I think that the you know it's important to remember that there are lots of very difficult things about intelligence we still don't know how to do right so you know I think I outline some of those key areas we're working hard on that and many others are too but we don't know how quickly those solutions will come I think there's going to require some really big breakthroughs are still needed at least as big as the ones we've had and possibly many of those so I think that's to come over the next few years in terms of the barriers to sort of using them you know I think we have to think very carefully about how we you know we want to test these kinds of systems because in a sense these systems are adaptive and they learn so it's a very new type of software in a way right so software generally is you know better better the most handy is you know we we write some software and then you test it and you stress test it and unit testing there's all these ways of testing software and then you know if it's ready to be shipped and deployed and our it goes and you know it's going to behave the way you want now of course one of the advantages of our systems is when you send them when you put them out in the world they'll continue to adapt and learn and and to the new situations they encounter that you may not have thought of but then the question is how do you make sure they still behave the way you want them to and how do you test that kind of system so actually that's a big challenge I can be a little light-hearted in flying I'm told a common phrase from one pilot to another is what's it doing now so a little bit of that sort of lost that sort of stuff so questions let's start off at the front here lady right in the front there's a microphone coming very quickly - yeah sorry Maggie Murphy from The Telegraph hi so I've got a question about switches the implications a I may have an Democracy further down the line so a lot of what you talked about was predicting human behavior I do you think that there's a legitimate concern around predicting human behavior at scale potentially manipulating people serving them political advertising or even with private corporations perhaps gambling or gaming where you have to pay to level up what your thoughts on that yeah I'm not sure I did talk about predicting human behavior but so yes so what we're talking about here is if you're referring to the kind of Facebook stuff I mean we're talking about finding structure in any kind of data so I'm thinking more about scientific data that you've got or you know in the case of our stuff the gaming data so it playing against itself and generating its own data and then finding paths you can think of it's like an intelligent search so you've got this huge combinatorial space and you want to try and find say a new material design or new drug design or indeed a new go position and how do you efficiently search through all of that that amount of data and what you've got to find is structures and patterns that can help you reduce the size of that search really that's what you can think of out of zero and alphago doing and that's where we are at the moment and of course you know eventually these systems could predict all sorts of things potentially but right now I mean the first thing you ought to do is get get the data into some kind of format that you can actually express and the second thing is kind of an objective function some kind of goal that you want this to the system to do so I think you know it's quite different to the kinds of goals that say Facebook has with Widow's systems I have a question from downstairs I'm afraid I don't know who it comes from but here it I'll paraphrase humans are irrational what's the approach approach how do you approach an automatic system which has to deal with some level of irrationality sure well I think in fact that's that's one of the most tricky things about a lot of systems like economics I think is one of the most difficult areas of science because it really is a sort of aggregate of human behavior right and and of course they will tell you better than most scientists about about human irrationality and how that impacts things I mean I think what we've got well potentially we have here is systems that you know can be quite rational and then we've got to think about what aspects of rationality do they need to model if at all to understand the systems at some point probably they're going to need something you know I can imagine they're going to need to understand if they interact with human experts and and and human systems you know a little bit up to empathize about how humans behave and what they can expect from them but I don't think you know I think part of the power of these systems is they could be very rational systems have you an earlier career person and gentleman halfway down with this end up like to say whether you're early yeah kind of and Michael Vick's and startup founder and so there's this discussion between Elon Musk and Mark Zuckerberg about the threat AI poses to humanity and so firstly what your thoughts on that both from a personal perspective it just seems crazy to me because I mean what are we exactly worried about we worried about system we can't turn off are we worried about a Westworld type scenario where there's AI running around us manipulating us it just seems so far off when at the moment we're just talking about games we're not talking about the complexity that you discuss towards the end of your lecture yeah I mean it's a good question I think you know that's why I try to make emphasize we although there's been a some impressive breakthroughs we are still at a nascent stage and we are talking about just you know board games and things I've at the moment but I think my view is a sort of somewhere in between that bait that you're talking about so just for those you don't know II lawns or has sounded the alarm bell a lot about the dangers of AI and it's sort of existential risks of AI and then Mark Zuckerberg replied kind of that there aren't any and we should worry about that and it's all sort of roses and I think actually the real answer comes somewhere between my view is that we need a lot more research has to be done about that what these systems are and what their capabilities could be so you know what type of systems we'd want to design right I think a lot of these things of because we're very early still unknown so you know and and I think a lot of the things people worry about are going to get a lot better so for example the interpretability of these systems this is one thing that's all I get often asked is well you know how does alpha go play go right and we don't know that yet it's a big neural network and it's a little bit like our brains you know we roughly know what it's doing but actually the specifics it's not like a normal program where you can point to it and go like this bit of code is doing this and for safety critical systems perhaps in healthcare and others you know if it was to control a plane you know you'd actually want to know exactly what why the decision was made right and be able to track that back for accountability and make sure there's no bias and for fairness and all these other things and these are this is very active error research and we we have a whole team that researches this and I'm not I'm not you know I think things will get a lot better on that phone in the next five plus years it's because we're at the very beginning of even having these systems working that's why we had don't yet know how to build visualization tools and other things but I think we will do having said that you know that there are you know we got to make sure you're like it's a very very powerful technology and the reason I work on it is because I think it's going to have this amazing transformative effect on the world for the better in things like science and technology no technology and medicine but you know like all powerful technologies it depends I think the technology itself is neutral but it depends on what we as a society decide to use it for you know obviously if we decide to use it it could be used for things like weapons and that will be terrible and we've signed many letters to the effect that there shouldn't be autonomous weapon systems they should always be human decision maker in the and so on but that that's really a political decision and a societal decision which is why it's important we have debates like this because in my view no nobody should be building those kinds of systems but that's that's gonna require you know un agreement and things like this so I actually think those are the things that we should be worrying about near-term and the sorts things I just mentioned to Andi is like if we're gonna have self-driving cars well maybe we should test them before putting them on the road and like be to testing them live you know on the road which is sort of what's happening now is that responsible really and and the question is and then and then them sort of tenor question is how do you ensure mathematically in some sense those systems are safe and they'll only do what you think they're going to do when they're gonna be you know out in the wild and you know they're adaptable learning systems so the I think these are kind of technical questions so I don't think I certainly don't think there's nothing to worry about and I definitely think it's worth worrying about it now even though I think it's many many decades away before you know the sorts of things Elon is worried about will will come to pass if ever and I think we have plenty of time to make sure that doesn't but I think we need to be thinking about it now and not just the researchers but society at large at the back perhaps is the lady whose hand is up there yes sir thank you for career baby I run society inside which is a not-for-profit looking at cross technology learning from biotech GM quantum Tech recently and one of the areas were looking at at the moment is this concept of trusting governance and the lessons of past tech is that technologies are so over excited about themselves that they slightly resist the governance and we see that happening in AI at the moment so I would be really interested to know given the breadth of the places that AI will be applied what your views on on the trustworthiness of governance and the work that you're going to be doing on governance yeah so we engage with government quite regularly are all covered lots of governments actually not just the UK government and I think it's really important in this phase for them to get up to speed with what's actually on technically and the the sorts of questions that they should be thinking about and wrestling with and it's not that I don't think there's anything that you know it needs to be done now in fact I think that will be bad if there was some kind of knee-jerk regulation or something like that because I think even if you were to if I was to waive all you were to ask me like waiver wand what would you regulate we don't know as researchers we don't actually know what the the right protocols are the right safety mechanisms or the right control makers I still an active area of research but that's coming down the line and when that when we do have some kind of agreement around that you know then I could imagine some sort of regulation around that and that's so driving cars though already yes exactly so it's so that so that's so that's with AGI so generally I what I was talking about in terms of specific deployed things I think what we need to do is upgrade our current existing regulations that we have say in transport or in healthcare we already know there's already a lot of regulation around those areas but they need to be upgraded to deal with the new technologies that are coming in and I think that's actually what we should be focusing on now is is is improving those regulations so that they can actually cope with the new world that's coming very fast and I think we have already have committees and organizations that the departments that were well capable of doing that with the right advice from from experts young person right in front here very early career very or a genius yes mid Kimiko thank you for the wonderful presentation I understood everything lovely oh great my question is what's your vision for deep line what do you think it's part will be in the future of AI Yass great question so I hope it will play like a major part in the research so I hope that we will accelerate and progress some of the big breakthroughs that I talked about they're still left to be done like how concepts are dying on memory these things I hope that deep mind will be a big part of discovering that and then the second thing is I'd like us to be a beacon for the ethical use of AI and to make sure that like sort of be a role model if you like for other companies and other organizations as to how they should approach thinking about the ethical questions and the and the philosophical questions behind AI and and how we use it okay we'll go to the lady over here towards the front Amanda Dickens I'm currently a civil servant but this is definitely not a government question and I am very interested that you're using your thinking about how can we use AI to potentially one day explore what consciousness is and I just wanted to try flipping the ethics in AI question and are you thinking about what would be your view on whether an artificial general intelligence of a sufficiently powerful nature we might need to think about at some point does it require rights if it's kind of edging around that border of consciousness yeah so you know a great question again and this is I mean obviously we're straying into a philosophy territory here right which I actually do think AI quickly becomes in some ways when you start thinking about the far future and you know my person I mean you know we don't really know what consciousness is neuroscientists don't really agree and even philosophers don't agree currently right so there's a definition problem although I think interestingly we all feel we have it right so if I was to guess I if I was to guess I would say intelligence and consciousness are what I would call double dissociable so I don't think intelligence requires consciousness and vice versa right I mean I think a lot of animals like if you have pet dog or cat you know I think a lot of us would say they're they're conscious they certainly seem to dream at least my pet dog does and you know and it seems to be you know some aspects of consciousness maybe not as high level as a human but some aspects of that on the other hand you know if you look at something alphago or our Tory programs this there's no question of any kind of consciousness there I feel it's just it feels to me like just an algorithm a sort of machine and I think it's going to be interesting to see but there are lots of debates on that as whether you know it does intelligence require consciousness or vice versa and I think you know if it turns I think the question the answer to that question is going to be interesting either way right if we can create a system I can easily imagine us building a fantastically intelligent system a GI system that doesn't feel conscious in any way like you do to me or I do to you and then the quake then that would be questioning because then you could sort of take apart the AI system and sort of feel like well what's where you know what's what's missing then in that case as compared to the human brain right what is the missing ingredient and isn't that Lee and it would also resolve some philosophical arguments about the nature of intelligence so these are the kinds of things I think as a collateral of what we're doing I think if we think about it in the right way with a like crack with the right collaborations with neuroscientists and psychologists and and and perhaps sociologists it might be an interesting tool to like an experimental world to test the things that the question of consciousness in it you know and things like qualia and sort of related issues which I think will come up against as our systems become more intelligent okay we've got time for a couple more so let me go at the back the gentleman right at the back yes right there and then one more I'll pick at random and thank you very much okay thank you very much again for your informative chair a meta question here which is around paradigms natural consequences you were showing of the deployment of AI through its learning eventually results in the transcending of the paradigm of the system that is operating right alphago sorry what is the paradigm of deepmind in relation to its deployment of AI what system do you use to determine that paradigm and to give an example of of what does that mean practically for example is how do you what system do you use to deploy your resources your attention your energy in a certain aspect of AI for example ie the AI itself the who of the AI as well as for example in relation to the what so what does AI deployed on for example you were talking about some things health etc yeah great so I understand the correct question correctly I think you're talking about the actual organizational process of what we're doing so how do we decide what to research and what to deploy apply it to I think that's what you're asking is that right that's right okay so so in terms of this is actually a great question and relates to the one point I made earlier in the talk about the this new way of organizing science so it's sort of a whole I mean it would be a whole whole lecture in itself that like what do we do differently but you can think about what I've tried to do is bring in some you can think of them as agile software project management methods that I learnt from actually writing computer games and big engineering projects I did in that early in my career and I've tried to translate what does that mean and in Alan an analogous setting in for science can you actually project managed science even right now and also can you assemble you know a large team we have a pretty large team for a research organization round a collective goal and actually build quickly on top of each other's work much more rapidly iterate that than you would get in academia much more like you would get if you're building a product in you know a normal company normal technology company and how can you do all of that without damaging or hindering the bottom-up creativity that comes from you know in the best scientific organizations right the reason science works is big science science with a capital S works because you you have tons of brilliant minds all kind of independently searching you know for the truth right and for their own or having their own ideas and then and then pursuing those ideas so how can you coordinator at in a like fashion without just without with while still encouraging this sort of bottom-up flourishing creativity and I think that's this if there is a secret to why deep mine has been successful is I think we've got that balance sort of just right right so that's the that's that's sort of hopefully answers part one question second question you had was about well how do we decide what to deploy it to yeah well we actually have a huge sort of spreadsheet of factors perhaps we should actually have AI looking at that spreadsheet but we don't do that by humans and and what we do in my apply division which was led by Mustafa Suleiman one of my co-founders he runs our applications group is we have a whole bunch of desired properties for any application we are going to look at right and top of those things are social good fit to the current level of our technology you know how much extra specialization is required so the more specialization is required the less likely we want to do that because ideally we want to use our core technology and then of course there secondarily there are things like commercial opportunity and other things like that but top of that things are are sort of fit with research so that we're not pulling the researchers in directions they wouldn't go in otherwise so it's very important for us we're a research led organization you know research is what we're primarily there for and then secondly is the kind of societal good that we can you know we think we can deliver off the back of using these technologies so that's why healthcare always comes up sort of top of you know on all those categories and there's also the personal motivations of many of my team so many of them are just passionate about of course about solving diseases and helping with medicine and so on but there are other things too like renewable energies so one of the big things we we one of the big achievements we do is we actually use the alphago program a variant of it to to control the cooling systems in the data centers at Google and the Google data sensors a massive right every time you do a school search you're pinging something to one of these data centers and they use vast amounts of power mostly its renewable energy but we would like to reduce that power and what we did is when we we use these processors on on the cooling systems of controlling the fans and the pumps and the water pumps and so on we saved 40% of the energy that those cooling systems used which is huge so 15% overall which is a you know a huge saving in the energy and obviously cost saving as good for environment as well as the cost-saving thank you gentlemen over here and then one quickie and then we're done okay it's just over here on in the front thank you you you're really quite an inspiration to all of us so I studied neuroscience as you did and I have a question about the elements of deep neural networks yeah you know as well as most in the room the neurons are electrically active distributed branch trees synapses are probabilistic yep short-term plasticity which which is universal it's in every neuron in both of our brains do you see any of these elements finding their way into deep Nets yeah when are you on that is it for ten years from you yes now that's a really great question actually so so as you pointed out and everyone should know is when when we call these things in your networks they're incredibly simplified versions of what real our real brains are doing right as you've just suggested so our real neurons are much more complex things and you know they are probabilistic they use they're called spiking your networks that they use they use timing of spike trains for information passing so they have a lot of properties that our simplified point these are called point neurons don't have these are just really simple mathematical objects in and I would say they were inspired by neuroscience rather than actually in any way really mimicking real neuroscience systems but this is an open question for us so I have quite a large neuroscience team a proper neuroscience team at demine and we collaborate with many universities it's around you know 35 40 people so it's one of our biggest teams and we a continual question for us is how neuroscience inspired are we going to be so that so no I said neuroscience inspired not reverse engineering neuroscience so there are other groups around the world there's also there's a big EU billion euro project the Blue Brain Project based in Switzerland that's trying to actually explicitly reverse-engineer the brain so they're trying to call copy called cortical columns we all spiking neurons and all the nuances of those of how real neurons work and the messiness of that now in my view that's that's that's too low level that's an implementation detail because if there's no reason I don't think to assume that an in silico system so something they use a silicon-based system should copy all of the implementation details of a carbon system needs to do like there's all sorts of reasons why our brain has to do things due to our biological constraints now those constraints are different for a computer system so I don't think there's any reason to have to copy all of including all the constraints and the specifics of the biology what I motion in is called systems neuroscience which is what I study which is the algorithmic and computational level so why I'm really interested in is the functions the brain has and mimicking and or at least having making sure I have the same capabilities in my artificial system not the specific implementation details but that line is a moveable line so so it depends on what neuroscientists discover or if they discover there's some real functional difference about using spiking neural networks for example we would then start investigating that in a proper way but and I have you know we have in our neuroscience thing we have experts in all of those different levels of detail and of the brain and they keep us all up to date with the latest literature on that so it's an active ongoing sort of moveable line that we have is - how much detail do we take in from inspiration from the neuroscience one last quickie before I make some concluding remarks gentleman over here hi my name is Samanas and I'm an ml engineer so it's a slightly technical question what are your thoughts on the ability of the deep reinforcement learning framework to deal with unforeseen events because one convenient feature of games is that it's an isolated environment yeah while training the network is seeing the game and only the game there's no interruption there's no break something in the real world like the weather or the price of a stock you know we don't fully understand every single detail that could influence the outcome do you still feel that the framework will be applicable or do we need to evolve something better yeah a fantastic question certainly the systems we have at the moment will not be enough right so that is the next sort of frontier really is how can you deal with unexpected information or probabilistic information or in completely incomplete information so we do have researchers who are looking at things like poker for example that has obviously incomplete information so you're in a situation or Starcraft which is a computer a very rich computer strategy game that doesn't you don't have full information about the board like you do in go but even they're still relatively simple compared to what you get in real-life situations as you pointed out and then the other thing is is the amount of data they use and the amount of experience they need and I mention that with the one-shot learning so that's a very key thing that we want to improve is how can you learn from fewer examples and ideally from just one example so then you could even deal with a Black Swan event potentially right somehow and in fact all of that that list of things that remain challenges almost all of them you can think of as part of the solution to the problem you're talking about so transfer learning for example would help because there you're you're you know you've learned how to deal with some you know some structure in some world that you've got used to and then suddenly this new thing happens to you or new domain and but then you realize that underlying it there's some similar structure you know maybe a hierarchical structure or something even though it seems to look on the surface perceptually totally different and then maybe you can use something from there to improve and speed up your learning in that new domain and you know those are all you know amazingly complex problems that haven't are yet to be solved perhaps your unit your sole summaries so let me just quickly make just a couple of concluding remarks so the Royal Society is a convener and we hope to fuel this debate but please remember the following about the Royal Society we're independent of government we're independent of Industry and we're independent of universities so we can speak truth to power as appropriate in all those dimensions and so you can be sure that we'll make every effort not just to be trusted but by using that independence to be trustworthy in this debate which is going to continue for some while and to damage I can just summarize on behalf of everybody with one word brilliant you

Channel: The Royal Society

Views: 43,292

Rating: 4.783217 out of 5

Keywords: Royal Society, science, scientists, national academy, UK, AI, Artificial intelligence, robots, demis hassabis, machines, machine learning, technology, neuroscience, scientific discovery, scifi

Id: zYfzux7JKHE

Length: 83min 38sec (5018 seconds)

Published: Fri May 11 2018

Reddit Comments

[u/Zaflis]

In short: A recap of what's already been said before, with some new info on top. Also a confirmation on Deepmind working hard on AGI problem.