François Chollet: Keras, Deep Learning, and the Progress of AI | Lex Fridman Podcast #38

the following is a conversation with Francois Shelley he's the creator of Karass which is an open source deep learning library that is designed to enable fast user friendly experimentation with deep neural networks it serves as an interface to several deep learning libraries most popular of which is tensorflow and it was integrated into the tensorflow main codebase a while ago meaning if you want to create train and news Neil networks probably the easiest the most popular option is to use chaos inside tensorflow aside from creating an exceptionally useful and popular library Francois was also world-class AI researcher and software engineer at Google and he's definitely an outspoken if not controversial personality in the AI world especially in the realm of ideas around the future of artificial intelligence this is the artificial intelligence podcast if you enjoy it subscribe on YouTube give us five stars and iTunes supported on patreon or simply connect with me on Twitter at lex Friedman spelled Fri D M a.m. and now here's my conversation with Francois shall I you're known for not sugarcoating your opinions and speaking your mind about ideas and AI especially on Twitter it's one of my favorite Twitter accounts so what's one of the more controversial ideas you've expressed online and gotten some heat for how do you pick yeah no I think if you have if you go through the trouble of maintaining Twitter accounts you might as well speak your mind you know otherwise it's you know what's even the point filling in Twitter accounts they're getting nice Colin just didn't leave it in in the garage yes so what's one thing for which I got out of push back perhaps you know that time I wrote something about the idea of intelligence explosion and I was questioning the ID and the reasoning behind this idea and I guess I was push back on that I guess not a flag for it so yeah so integers explore I'm sure if Mei was the idea but it's the idea that if you were to build general AI problem-solving algorithms well the problem of building such an AI that itself is a problem that could be solved by your eye and maybe it could be so better than that then what humans can do so you're a I could start tweaking its own algorithm good that start being a better version of itself and so on it's ratified in a recursive fashion and so you would end up with an AI with exponentially increasing intelligence all right and I was basically questioning this idea first of all because the notion of intelligence explosion uses an implicit definition of intelligence that doesn't sound quite right to me it considers intelligence as property of a grain that you can consider in isolation like the height of the building for instance right but that's not really what intelligence is intelligence emerges from the interaction between a brain a body like embodied intelligence and an environment and if you're missing one of these pieces then you can actually define interagency so just tweaking a brain to make it smaller and smaller doesn't actually make any sense to me so first of all you're crushing the dreams of many people right so there's a little bit like say Maris I feel a lot of physicists max tegmark people who think you know the universe is an information processing system our brain is kind of an information processing system so what's the theoretical limit like it doesn't make sense that there should be some it seems naive to think that our own brain is somehow the limit of the capabilities and this information is just I'm playing devil's advocate here this information processing system and then if you just scale it if you're able to build something that's on par with the brain you just the process that builds it just continues and it will improve exponentially so that that's the logic that's used actually by almost everybody that is worried about superhuman intelligence yeah so you're you're trying to make so most people who are skeptical that are kind of like this doesn't their thought process this doesn't feel right like that's for me as well so I'm more like it doesn't we the whole thing is shrouded in mystery where you you can't really say anything concrete but you could say this doesn't feel right this doesn't feel like that's how the brain works and you're trying to with your blog post and now making a little more explicit so one idea is that the brain isn't exists alone it exists within the environment so you can't exponentially you have to somehow exponentially improve the environment and the brain together almost yeah in order to create something that's much smarter in some kind of of course we don't have a definition of intelligence that's right that's correct III don't think if you look at very smart people today even humans not even talking about a eyes I don't think their brain and the toughness of their brain is the bottleneck to the actually expressed intelligence to their achievements you cannot just tweak one part of this system back of this brain body environment system and expect capabilities like what emerges out of this system to just you know explode exponentially because anytime you improve one part of the system with many interdependencies like this there's a new bottleneck that arises right and I don't think even today for very smart people their brain is not the bottleneck to the sort of problems they can solve right in fact many various what people to them you know they are not actually solving any big scientific problems in a tense time they like Einstein but you know the the patent clerk days like Iceland became Einstein because this was a meeting of a genius with a big problem at the right time right but maybe this meeting could have noon and never happens and then Iceland there's just been a patent clerk it's and in fact many people today are probably like genius level smart but you wouldn't know because they're not really expressing any of that was brilliant so we can think of the world earth but also the universe is just as the space of problems so all these problems and tasks are roaming it a various difficulty and there's agents creatures like ourselves and animals and so on that are also roaming it and then you get coupled with a problem and then you solve it but without that coupling you can't demonstrate your quote-unquote intelligence exactly intelligence is the meaning of great problem-solving capabilities with a great problem and if you don't have the problem you don't react spreche in intelligence all you're left with is potential intelligence like the performance of your brain are you know haha your IQ is which in itself it's just a number right so you mentioned problem-solving capacity yeah what what do you think of as problem-solving about what can you try to define intelligence like what does it mean to be more or less intelligent is it completely coupled to a particular problem or is there something a little bit more universal yeah I do believe all intelligence is specialized intelligence even human intelligence has some degree of generality well all intelligence systems have some degree of generality they're always specialized in in one category of problems so the human intelligence is specialized in the human experience and that shows at various levels that shows in some prior knowledge that's innate that we have at birth knowledge about things like agents goal-driven behavior visual priors about what makes an object try us about time and so on that shows also in the way we learn for instance is very very fast to pick up language it's very very easy for us to learn certain things because we are basically hard-coded to learn them and we are specialized in solving certain kinds of problem and we are quite useless when it comes to other kinds of problems for instance we we are not really designed to handle very long term problems we have no capability of seeing that the very long term we don't have them how much working memory you know so how do you think about long term using long term planning we're talking about scale of years millennia what do you mean by long term were not very good well human intelligence is specialized in the human experience and humans experience is very short like one lifetime is short even within one lifetime we have a very hard time envisioning you know things on a scale of yells like it's very difficult to project yourself at at the scale of favi at the scale of ten years and so on right we can solve only fairly narrowly scoped problems so when it comes to solving bigger problems larger scale problems we are not actually doing it on an individual level so it's not actually our brain doing it we we have this thing called civilization right which is itself a sort of problem solving system a sort of artificially intelligent system right and it's not running on one brain is ringing on a network of brains in fact it's running on much more than a network of brains it's running on a lot of infrastructure like books and computers and the internet and human institutions and so on and that is capable of handling problems on the on a much greater scale in any individual human if you look at some computer science for instance that's an institution that solves problems and it's it is super human right I took resin on a greater scale it can source cancer much bigger problem than an individual human good and science itself science as a system as an institution is a crime affair artificial intelligence problem solving algorithm that is superhuman yes these computer science is like a theorem prover at a scale of thousands maybe hundreds of thousands of human beings at a scale what do you think is a intelligent agent so there's us humans at the individual level there is millions maybe billions of bacteria on our skin there is that's at the smaller scale you can even go to the particle level as systems that behave you couldn't say intelligently in some ways and then you can look at the earth as a single organism you can look at our galaxy and even the universe is just a little organism do you think how do you think about scale and defining intelligent systems and we're here at Google there is millions of devices doing computation just in a distributed way how do you think what intelligence there's a scale you can always characterize anything as a system I think people who talk about things like intelligence explosion tend to focus on one Asian is basically one brain like one brain considered in isolation like a brain a jaw that's controlling your body in a very like top to bottom can a fashion and that body is person goes into an environment so it's a very hierarchical view you have the brain at the top of the pyramid then you have to bother just plainly receiving orders and then the body is manipulating objects in environment and so on so everything is subordinate to this one thing this epicenter which is the brain but in real life intelligent agents don't really work like this right there is no strong delimitation between the brain and the body stalin's you have to look not just to the brain but at the nervous system but then the nervous system and the body are not free to step and it is so you have to look at an entire animal as one agent but then you start realizing as you observe an animal of any length of time that a lot of the intelligence of an animal is actually externalized that's especially true for humans a lot of our intelligence is externalized when you write down some notes that is externalized intelligence when you write the computer program you are externalizing cognition so it's externalizing books it's generalized in in computers the internet in other humans it's externalizing language and so on so it's there is no like hardly limitation of what makes an intelligent agent it's all about context okay but alphago is better at go than the best humor player you know there's levels of skill here so do you think there is such a ability as such a concept as a intelligence explosion and a specific task and then well yeah do you think it's possible to have a category of tasks on which you do have something like an exponential growth of ability to solve that particular problem I think if you consider specificity corn is probably possible to some extent I also don't think we have to speculate about it's because we have real-world examples of frequency self-improving intelligence systems for instance science problem-solving system and knowledge generation system like a system that experiences the world in some sense and then gradually understands it and can act on it and that system is superhuman and it is clearly recursively self-improving because science feeds into technology technology can be used to build better tools with our computers better instrumentation and so on which in turn I can make sense faster right so science is probably the closest thing we have today to a recursively self-improving super human AI and you can just observe you know it's science its scientific progress to the exploding which you know it's that vision isn't is an interesting question you can use that as a basis to try to understand what we happen with a superhuman AI that as a science track behavior let me linger on it a little bit more what is your intuition why an intelligence explosion is not possible like taking the scientific all the semantic revolutions why can't we slightly accelerate that process so you you can absolutely accelerates any problem solving process so recursively as recursive self-improvement is absolutely a real thing but what happens with recursively seven boring system it's typically not explosion because no system exists in isolation and so tweaking one part of the system means that suddenly another pollow system becomes a bottleneck and if you look at science for instance which is clearly a recursively self-improving clearly a problem-solving system scientific progress is not actually exploding if you look at science what you see is the picture of a system that is consuming an exponentially increasing amount of resources but it's having a linear output in terms of scientific progress and maybe that that will seem like a very strong claim many people are actually saying that you know scientific progress is exponential but when they are claiming this they are actually looking at indicators of resource consumption resource consumption by science the number of papers being published the number of parents being filed and so on which are just just completely credited with how many people are working on science today yeah right so it's actually an indicator of resource consumption but what you should look at is the ad put is progress in terms of the knowledge that sales generates in terms of the scope and significance of the problems that we solve and some people have actually been trying to measure that like Michael Neilson for instance he had a very nice paper I think that was last year about it so his approach to measure a scientific progress was to look at the time line of scientific discoveries over the past you know hundred 150 years and for each measure discovery ask a panel of experts to rate the significance of the discovery and if the output of Sciences institution were exponential you will expect the example density of significance to go up exponentially maybe because there's a faster rate of discoveries maybe because the discoveries are you know increasingly more important and what actually happens if you if you plot this temporal density of significance measured in this way is that you see very much a flat graph you see a flat graph across all disciplines across physics biology medicine and so on and it actually makes a lot of sense if you think about it because thing about the progress of physics a hundred and ten years ago right it was a time of crazy change think about the progress of technology you know 160 years ago when we started it in you know replacing horses with scars when we saw that in electricity and so on it was a time of incredible change and today is also a time a very fast change but it would be an unfair characterization to say that today technology in science are moving way faster than they did 50 years ago 100 years ago and if you do try to regardless plots the temporal density of the significance you have significance idea of seeing a family sorry you do see very flat curves let's fasten and and you can check out the paper that Michael Nielson had about this idea and so the way interpret is as you make progress in an in a given field on any given subtitles it becomes exponentially more difficult to make further progress like the very first person to work on information theory if you enter a new field and still the very early years there's a lot of low-hanging fruits you can take that's right yeah but the next generation of researchers is gonna have to dig much harder actually to make smaller discoveries a probably larger number of small discoveries and to achieve the same amount of impact you're gonna need a much greater headcount and that's exactly the picture you're seeing with science that the number of scientists and engineers is in fact increasing exponentially the amount of computational resources that are available to science is increasing exponentially and so on so the resource consumption of science is exponential but the output in terms of progress in terms of significance is linear and the reason why is because and even though science is recursively self-improving meaning that scientific progress mm-hmm turns into technological progress which in turn helps science if you look at computers for instance our products of science and computers are tremendously useful in spinning up science the internet same thing the engine is a technology that's made possible by various incentive advances and itself because it enables you know scientists to to network to communicate to exchange papers and ideas much faster it is a way to speed eccentric products so even though you're looking at a recursively self-improving system it is consuming Spanish way more resources to produce the same amount of problem-solving so that's the fascinating way to paint and certainly that holds for the deep learning community right if you look at the temporal what did you call it the temporal density of significant ideas if you look at in deep learning I think I'd have to think about that but if you really look at significant ideas in deep learning they might even be decreasing so I I do believe the per per paper significance it's like creasing with signified and the amount of papers is still today exponentially increasing sweating if you look at an aggregate my guess is that you would see a linear progress you're probably aware to some to some the significance of all papers you would see roughly in your profits and in in my opinion it is not coincidence that you're seeing in your progress in science despite exponential resource conception I think the resource consumption is dynamically adjusting itself to maintain linear progress because the we as a community expecting your progress meaning that if we start investing less and sing less progress it means that suddenly there are some low-hanging fruits that become available and someone's going to step in step up and pick them right right so it's very much like a market right for discoveries and ideas but there's another fundamental part which you're highlighting which as a hypothesis as science or like the space of ideas any one path you travel down it gets exponentially more difficult to get a new way to develop new ideas yes and your sense is that fun that's gonna hold across our mysterious universe yes when exponential promise Stringer's exponential friction so that if you tweak one part of a system suddenly some other part becomes a bottleneck for instance let's say let's say develop some device that measures it's an acceleration and then it's it has some engine and it add puts even more acceleration in proportion if it's an acceleration and you drop it somewhere it's not going to reach infinite speed because some it exists in a certain context so the air around its gonna generate friction it's gonna is gonna you know block it at some top speed and even if you were to consider the broader context and lift the bottleneck there like the bottleneck a firm a friction then some other part of the system which starts stepping in and creating exponential friction maybe the speed of light are you know whatever and it's definitely horse true when you look at the problem solving algorithm that is being run by science as an institution science as a system as you make more and more progress this despoiling this recursive self-improvement component you are encountering exponential friction like do more researchers you have working on different ideas the more overhead you have in communication across researchers if you look at you were mentioned in quantum mechanics right well if you wants to start making significant discoveries today significant progress in quantum mechanics there is an amount of knowledge you have to ingest which is huge so there is a very large overhead to even start to contribute there is a large amount of overhead to synchronize across researchers and so on and of course this the significant practical experiments are going to require exponentially expensive equipment because there is your ones I've already been run right so in your senses there is no way escaping there's no way of escaping this kind of friction with artificial intelligence systems yeah no I think science is very good way to model with what we happen with with a superhumans are you serious if improving yeah that's intense I mean that's that's my intuition too it's not it's not like a mathematical proof of anything that's not my points like I'm not I'm not trying to prove anything I'm just trying to make an argument to question the narrative of intelligence explosion which is quite a dominant narrative and you do get a lot of pushback if you go against it because so for many people write AI is not just a subfield of computer science it's more like a belief system I just believe that the world is headed towards an event the singularity past which you know AI will become we go exponential very much and the world will be transformed and humans will become obsolete and if you if you go against this narrative because because it is not really a scientific argument but more for belief system it is part of the identity of many people if you go against this narrative it's like you're attacking the identity of people who believe in it it's almost like saying God doesn't exist at something right so you do get a lot of pushback if you try to question this ideas first of all I believe most people all they might not be as eloquent or explicit as you're being but most people in computer science and most people who actually have built anything that you could call AI quote unquote would agree with you they might not be describing in the same kind of way it's more so the pushback you're getting it's from people who get attached to the narrative from not from a place of science but from a place of imagination yes correct miss correct so why do you think that's so appealing because the usual dreams that people have when you create a super intelligent system past a singularity that would people imagine it somehow always destructive do you have if you were put on your psychology hat what's why is it so appealing to imagine the ways that all of human civilization will be destroyed I think it's a good story you know it's a good story and very interestingly it's mirrors residue stories right reiji's mythology if you look at the mythology of most civilizations it's about the world being headed towards some final event in which the world will be destroyed and some new world order will arise that will be mostly spiritual like the apocalypse followed by products probably yeah it's a very appealing story on a fundamental level and we all need stories we own stories to structure in the way we see the world especially at time scales that are beyond our ability to make predictions right so on a more serious non exponential explosion question do you think there will be a time when we'll create something like human level intelligence or intelligence systems that will make you sit back and be just surprised at damn how smart this thing is that doesn't require exponential growth and an exponential improvement but what what's your sense than a time line and so on that where you'll be really surprised at certain capabilities and we'll talk about limitations and deep learners so when do you think in your lifetime you'll be really damn surprised around 2013-2014 I was many times surprised by the capabilities of deep learning actually that was before we had assess exactly well deepening could do and could not do and it felt like a time of immense potential and then we started you know narrowing it down but I was very surprised so it's a it's it's it's it has already happened was there a moment there must have been a day in there where your surprise was almost bordering on the belief of the narrative that we just discussed what it was there a moment because you've written quite eloquently about the limits of deep learning was there a moment that you thought that maybe deep learning is limitless no I don't think I've ever believed this what was restocking is that it it worked all right they worked at all yes yeah but there's a there's a big jump between being able to do really good computer vision and human level intelligence so I I don't think at any points I wasn't an impression that the results we got in computer vision meant that we were very close to him and even intelligence I don't think we're very close to human ever intelligence I do believe that there's no reason why we want achieve it at some point I also believe that you know it's the problem is with talking about human level intelligence that implicitly you are considering like an axis of intelligence with different levels but that's not really how intelligence works intelligence is very multi-dimensional and so there's the question of capabilities but there's also the question is being human-like and two very different things like you can be potentially very advanced intelligent agents that are not human like at all and you can also build very human-like agents and this out okay two very different things right right let's go from the philosophical to the practical I can give me a history of Karis and all the major deep learning frameworks that you kind of remember in relation to chaos and in general tensorflow Theano the old days you give a brief overview Wikipedia style history and your role in it before return to AGI discussions yeah that's a broad topic so I started working on chaos to the name chaos at the time I actually pick the name like just today I was gonna release it so I started working on it in February 2015 and so at the time there weren't too many people working on deep learning maybe like fewer than 10,000 the software tuning was not really developed so the deepening library was cafe which was mostly C++ why do I say cafe was the main one cafe was vastly more popular than ya know in in late 2014 early 2015 cafe was the one library that everyone was using for computer vision and computer vision was the most popular problem absolutely company like covenants was like the subfield of deplaning it everyone was working on so myself suing in in late 2014 I was actually interested in islands in Rico neural networks which was a very niche topic at the time right III a tree to catherine around 2016 and so I was looking for good tools and I had used torch 7 News Channel you stay on a lot in cable competitions mmm I just cafe and there was no like good solution for Ireland's at the time like there was no reusable open-source implementation of in lsdm for instance so I decided to build my own and that first the pitch for that was it was going to be mostly around lsdm Iconia networks it was going to be in Python an important decision at the time that was Canon are obvious is that the models would be defined yeah a Python code which was kind of like going against the mainstream at the time because cafe Thailand who wants on like all the big libraries were actually going with you approach sharing static configuration files in Yemen to define models so some libraries were using code to define models like torch 7 obviously that was not Python Lezyne was like a piano based very early library that was I think developed I don't remember exactly probably late 2014 Python as well it's Python as well it was it was like on top of Tiano and so I started working on something and in the value proposition at the time was that not only that the what I think was the first reducible open-source implementation FRS diem you could combine Islands and covenants with the same library which is not really possible before like a he was on into incontinence and it was kind of easy to use because so before I was using ten I was actually using psychically on and I loved psychically for its usability so I drew a lot of inspiration from psychic then when I went Cara's it's almost like cycling for neural networks yeah the fit function exactly the fit function like reducing a complex training loop to a single function call right and of course you know some people will say this is hiding a lot of details but that's exactly the point all right the magic is the point all right so it's magical but in a good way it's magical in the sense that it's delightful yeah right yeah I'm actually quite surprised I didn't know that it was born out of desire to implement our hands in lc/ms it was that's fascinating so you were actually one of the first people to really try to attempt to get the major architectures together and it's also interesting you made me realize that that was a design decision at all is defining the model in code just I'm putting myself in your shoes whether the yamo especially if cafe was the most popular it was the most but I might fall if I was I'm if I were yeah I don't it I didn't like the yellow thing but it makes more sense that you will put in a configuration file the definition of a model that's an interesting gutsy move just stick with defining it in code just if you look back other libraries we're doing it as well but it was definitely the more niche option yeah okay Cara's and then girls so I really scare us in March 2015 and it got she's just pretty much from the start so the deep learning community was very small at the time lots of people were starting to be interested in the rest um so it was gonna release it at the right time because it was offering an easy to use it as team implementation exactly at the time where lots of yours started to be intrigued by the capabilities of onin on ins one LP so it it grew from there then I joined Google about six months later and that was actually completely unrelated to took care us actually joined a research team working on image classification mostly like computer vision so I was doing computer vision research at Google initially and immediately when I joined Google I was exposed to the early internal version of tensorflow and the way to appeal to me at the time and that was definitely the way it was at the time is that this was an improved version of Tiano so I immediately knew I had to port cars to this new tensorflow thing and I was actually very busy as as as a noogler as a new Googler so I had not time to work on that but then in November I think twist November 2015 tensorflow got released and it was kind of like my my wake-up call at hey to actually you know go and make it happen so in December I I putted cars to run on two of tensorflow but it was not exactly port it was more like a refactoring where I was abstracting away all the backend functionality into one module then the same codebase could run on top of multiple backends right so on top of things fluor Theano and for the next year yeah no you know stayed as the default option it was you know it was easier to use somewhat let's begin it was much faster especially when he came to Orleans but eventually you know a tensorflow overtook it right and test of all the early tests for similar architectural decisions there's the arrow yeah so what is there was a natural as a natural transition yeah absolutely so what I mean that still carries is the side almost fun project right yeah so it it was not my job assignment it's not I was doing it on the side that so I'm and even though it's great to have you know a lot of uses for a deepening library at the time like throughout 2016 but I wasn't doing it as my main job so things solid changing in I think it's mustard maybe October 2016 so one year later so Rashad who has the lead intensive law basically showed up one day in in our building while I was doing like so I was doing research in things like so I added of computer vision research also collaborations with Christians getting and deep planning for theorem proving it was a really interesting research topic answer Rajat was saying hey we saw chaos we liked it we saw that you had Google why don't you come over for like a quarter and and and work with us I was like yeah that sounds like a great opportunity let's do it and so I started working on integrating the chaos API into tends to flow more tightly so what fold up is a sort of like temporary tents of lonely version of chaos that was in tents for that contrib for a while and finally moved to dance to the core and you know I've never actually gotten back to my old sim doing research well it's it's kind of funny that somebody like you who dreams of or at least sees the power of AI systems the reason and they were improving will talk about has also created a system and makes the the most basic kind of LEGO building that is deep learning super accessible super easy so beautifully so that's the funny irony that you're Billy there's just both you're responsible for both things but so telephoto 2.0 it's kind of there's a sprint I don't know how long I'll take but there's a sprint towards the finish what do you look what are you working on these days whether you're excited about what are you excited about in 2.0 I mean eager execution there's so many things that just make it a lot easier yeah work what are you excited about and what's also really hard what are the problems you have to kind of saw so I've spent the past year and a half working on 1002 and it's been a long journey I'm actually extremely excited about it I think it's a great product it's a delightful product competitive law one we met huge progress so on the carrot side what I'm really excited about is that so you know previously Kara's has been this very easy-to-use high level interface to do deep learning but if you wanted to you know if you wanted a lot of flexibility the chaos framework you know was probably not the optimal way to do things compared to just writing everything from scratch so in some way the framework was getting in the way and in terms of you to you don't have this at all actually you have the usability of the high level interface but you have the flexibility of this lower level interface and you have this spectrum of workflows where you can get more or less usability and flexibility the trade-offs depending on your needs right you can write everything from scratch and you get a lot of help doing so by you know subclassing models and writing some train loops using ego execution it's very flexible is very easy to debug is very powerful but all of these integrates seamlessly with higher level features up to you know the classic workflows which which are very psychically unlike and and you know are ideal for a data scientist machining engineer type of profile so now you can have the same framework offering the same set of api's that enable a spectrum of workflows that are more or less uniform or less high level that are suitable for you know profiles ranging from researchers to data scientists and everything in between yeah so that's super excited I mean it's not just that it's connected to all kinds of tooling you can go on mobile and what that's for light it can go in the cloud or serving and so on and all its connected together now some of the best software written ever is often done by one person sometimes two so with a Google you're now seeing sort of Karass having to be integrated in tensorflow I'm sure it's a ton of engineers working on so and there's I'm sure or a lot of tricky design decisions to be made how does that process usually happen from at least your perspective what are the what are the debates like what a is there a lot of thinking considering different options and so on yes so a lot of the time I spend on Google is actually discussing design discussions right writing design Docs participating in design review meetings and so on this is you know as important as actually writing a cool right well there's a lot of thoughts there's a lot of thought and and a lot of care that is that taken in coming up with these decisions and taking into account all of our users because tensorflow has this extremely diverse user base right it's not it's not like just one user segment where everyone has the same needs we have small-scale production uses large-scale production uses we have startups we have researchers you know it's all over the place and we have to catch up to all of their needs if I just look at the standard the base of C++ or Python there's some heated debate do you have those at Google I mean they're not here in terms emotionally but there's probably multiple ways to do it right so how do you arrive through those design meetings at the best way to do it especially in deep learning where the field is evolving as you're doing it is there some magic to it there's a magic to the process I don't know just magic to the process but there definitely is a process so making design decision is about satisfying a set of constraints but also trying to do so in the simplest way possible because this is what can be maintained is what can be expanding in the future so you don't want to naively satisfy the constraints by just you know for each capability you need available you're gonna come up with one argument new idea and so on you want to design api's and that are modular and hierarchical so that they're there they have an API surface that is as small as possible right and and you want this modular hierarchical architecture to reflect the way that domain experts think about the problem because as the men expect when you're reading about a new media you're reading each toy or some darks pages you already have a way that you're thinking about the problem you already have like certain concepts in mind and and and your thing about how they relate together and when you're reading darks you're trying to build as quickly as possible and mapping between the concepts feature the new API and the concepts in your mind so you are trying to map your mental model as a domain expert to the way things work in the API so you need an API and an underlying implementation that are reflecting the way people think about these things so in minimizing the time it takes them this mapping yes minimizing the time the cognitive load there is in in just industry knowledge about your API an API should not be self referential or RF referring to implementation details it should only be referring to domain-specific concepts that people already never understand brilliant so what's the future of kerosene transfer look like what it stands for 3.0 look like so that's gonna to fall in the future for me to answer especially since I'm now I'm not even the one making these decisions okay but so from my perspective which is you know just one perspective among many different perspectives on the transferor team I'm really excited by developing even higher level api's higher level and Carols I'm really excited by hyper parameter tuning by automated machine learning or two ml I think the future is not just you know defining a model like like us and being Lego blocks and then click fit on it it's more like an automatical model let me just look at your data and optimize the objective view after right so that's that's what what I'm looking - yeah so you put the baby into a room with the problem and come back a few hours later with a fully solved problem exactly it's not like a box of Lego's right it's more like the combination of a kid that's pretty good at Legos blocks of Legos yeah it's just building the thing very nice so that's that's an exciting feature and I think there's a huge amount of applications and revolutions to be had under the constraints of the discussion we previously had but what do you think of the current limits of deep learning if we look specifically at these function approximator x' that tries to generalize from data they have you've talked about local versus extreme generalization you mentioned in your networks don't generalize well humans do so there's this gap so and you've also mentioned that externalization extreme journals asian requires something like reasoning to fill those gaps so how can we start trying to build systems like that all right yes so this is this is by design right deplaning models are like huge parametric models differentiable so continuous that go from an input space to not with space and they're trained with gradient descent so they're trying-- pretty much point by points they are learning a continuous geometric morphing from from an input vector space to not protective space right and because this is done point by point a deep neural network can only make sense of points in experience space that are very close to things that it has already seen in string data at best it can do interpolation across points but that means you know that means in order to train your network you need a dance sampling of the input cross ad with space almost a point-by-point sampling which can be very expensive if you're dealing with complex real-world problems like autonomous driving for instance or our robotics is it's doable if you're looking at the subset of the visual space but even then it's still fairly expensive you seen in millions of examples and it's only going to be able to make sense of things that are very close to waste as seen before and in contrast to that well of course we have human intelligence but even if you're not looking at human intelligence you can look at very simple rules algorithms if you have a symbolic rule it can actually apply to a very very large set of inputs because it is abstract it is not obtained by doing a point by point mapping for instance if you try to learn a sorting algorithm using a deep neural network well you're very much limited to learning point by point what the sorted representation of this specific list is like but instead you could have a very simple sorting algorithm written in a few lines maybe it's just you know two nested loops and it can process any list at all because it is abstract because it is a set of rules so deep learning is really like point by point geometric more things more things train with conditions and meanwhile abstract rules can generalize much better and I think the future is which combine the two so how do we do you think combine the tools how do we combine good point by point functions with programs which is what symbolic AI type systems yeah at which levels the combination happen and you know obviously we're jumping into the realm of where there's no good answers it just kind of ideas and intuitions and so on well if you look at the really successful AI systems today I think they are already hybrid systems that are combining symbolic AI with D planning for instance success robotics systems are already mostly model-based rule-based things like planning algorithms and so on at the same time they're using deep learning as perception modules sometimes they're using deep learning as a way to inject a fuzzy intuition into a rule-based process if you look at a system like an a self-driving car it's not just one big end when your network you know that wouldn't work at all precisely because in order to train that you need a dense sampling of experience space when it comes to driving which is completely unrealistic obviously instead the Salonika is mostly symbolic you know it's software it's programmed by hand it's mostly based on explicit models in this case mostly 3d models of the of the environment around the car but it's interfacing with the real world using deep learning modules right right so the deep learning there serves is the way to convert the raw sensory information to something usable by symbolic systems okay well it's lingering that a little more so dense sampling from input to output you said it's obviously very difficult is it possible in the case of send driving you mean let's say still driving itself driving permit for many people but let's not even talk about self-driving let's talk about steering so staying inside the lane lines following yeah it's definitely a problem cancel reason and two in the planning model but that's like one small subset on a second yeah I don't like you're jumping from the extreme so easily because I disagree with you on that I think well it's it's not obvious to me that you can solve Lane following it's no it's not it's not obvious I think it's doable I think in general you know there is no hard limitations to what you can learn with a DP on network as long as this the search space like is rich enough is flexible enough and as long as you have this dense sampling of the input cross output space the problem is that you know this dense sampling could mean anything from 10,000 examples to like trillions and trillions so that's that's my question so what's your intuition and if you could just give it a chance and think what kind of problems can be solved by getting a huge amounts of data and thereby creating a dense mapping so let's think about natural language dialogue the Turing test do you think the Turing test can be solved with a neural network alone well the deterrent test is all about tricking people into believing that certain to human I don't think that's actually very difficult because it's more about exploiting a human perception and not so much about intelligence there's a big difference between mimicking in Asian behavior an actual intogen behavior so ok let's look at maybe the elect surprised and so on the different formulations of a natural language conversation that are less about mimicking and more about maintaining a fun conversation that lasts for 20 minutes mm-hmm that's a little less about mimicking and that's more about I mean it's still mimicking but it's more about being able to carry forward a conversation with all the tangents that happen in dialogue and so on do you think that problem is learn Irbil with this kind of well the neural network that does the point-to-point mapping so I think it would be very very challenging to do this with deep learning I don't think it's out of the question either I wouldn't read out the space of problems that can be solved or the large neural network what's your sense about the spaces those problems so it useful problems for us in theory it's it's infinite right you can solve any problem in practice while deep learning is great fit for perception problems in general any any problem which is naturally a minimal to explicit handcrafted rules or rules that you can generate device exhaustive search or some program space so perception of intuition as long as you have a sufficient ring there and that's the question I mean perception there's interpretation and understanding of the scene yeah which seems to be outside the reach of current for social systems so do you think larger networks will be able to start to understand the physics and the physics of the scene the three-dimensional structure and relationships divisors in the scene and so on or really that's where symbology has to step in well it's it's always possible to solve these problems with with deplaning is just extremely inefficient a model would be an explicit rule-based abstract model would be a flaw efficient for better and more compressed representation of physics then learning justice mapping between in this situation this thing happens if you change the situation like slightly then this other thing happens and so on do you think is possible to automatically generate the programs that would require that kind of reasoning our dessert have to so the word expert systems fail there's so many facts about the world had to be hand coded and thing is possible to learn those logical statements that are true about the world and their relationships do you think I mean that's kind of what you're improving at a basic level is trying to do right yeah except it's it's much harder to farm any statements about the world compared to family ting mathematical statements statements about the world you know tend to be subjective so can you can you learn rule-based models yes yes differently that's the this is a field of program synthesis however today we just don't really know how to do it so it's it's very much a grad search or research problem and so we are limited to you know the sort of at recession grassroot algorithms that we have today personally I think genetic algorithms are very promising so I was like genetic programming genic priming Zack can you discuss the field of program synthesis like what how many people are working and thinking about it what where we are in the history programs the decision what are your hopes for it well if it we are deep planning this is like the 90s so meaning that already have we already have existing solutions we are starting to have some basic understanding of where this is about but it still I feel that is in its infancy there are very few people I working on it there are very few real-world applications so the one we are world application I'm aware of is a flash fill in Excel it's a way to automatically learn very simple programs to format cells in an excel spreadsheet from from a few examples for instance training a weight from a date things like that oh that's fascinating yeah you know okay that's the disgusting topic I always wonder when I provide a few samples to excel what it's able to figure out like just giving it a few dates mm-hmm what are you able to figure out from the pattern I just gave you it's just a fascinating question and it's fascinating whether that's learn about the patterns and you're saying they're working on that yeah how big is the toolbox currently are we completely in the dark so if you said enjoying the in terms of provinces no I would say so maybe not even even too optimistic because by the nineties you know we already understood that prop we already understood you know the engine of deplaning even though we couldn't release its potential quite today I don't think we've found the engine of problems into this so we're in the winter before backprop yeah anyway yes so I do believe program synthesis in general discrete search over route based models it is going to be a cornerstone of our research in the next century right and that doesn't mean like we're gonna drop deep learning deep learning is immensely useful like being able to learn this is a very flexible adaptable parametric models who's got Henderson let's let's actually mentally use like all it's doing its pattern cognition but being good at pattern recognition given lots of delays is a statistics from me powerful so we are still gonna be working on the planning we are going to be working on programs entities we're going to be combining the two increasingly automated ways mm-hmm so let's talk a little about data you've tweeted about 10,000 deep learning papers have been written about hard coding priors about a specific task in a neural network architecture it works better than a lack of a prior basically summarizing all these efforts they put a name to an architecture but really what they're doing is hard-coding some priors that improved yes yes but we get straight to the point is it's probably true and so you say that you can always buy performance by in quotes performance by either training on more data better data or by injecting tasks information to the architecture that pre-processing however this is an informative about the generalization power the techniques use the fundamental ability to generalize do you think we can go far by coming up with better methods for this kind of cheating for better methods of large-scale annotation of data so building better prize you if she was emitted it's not seeing any more right I'm talking about the cheating but large-scale so basically I'm asking about something that hasn't and from my perspective been researched to too much is exponential improvement in annotation of doing it do you you often think about I mean it's actually been I'm being researched quite a bit you just don't see publications about it's because you know people who publish papers are gonna publish about knowing benchmarks sometimes I enter is a new benchmark people who actually have real-world large-scale dependence they're gonna spend a lot of resources into data annotation and get data annotation pipelines but you don't sink papers that's interesting so do you think there are certainly resources but do you think there's innovation happening oh yeah asked me to clarify a at the point in the twist so machine learning in general is the science of generalization you want to generate knowledge that can be reused across different data sets across different tasks and if instead you are looking at one data set and then you are hard coding knowledge about this task into your architecture this is no more useful than training in network and then saying oh I found these weight values perform well right so that David hah I don't know if you know that David yeah the paper the other day about weight agnostic neural networks this is very interesting paper because it really straights the fact that an architecture even without wickets in architecture is a knowledge about a task it encodes knowledge and when it comes to architectures that are uncraft Admira searchers there in some cases it is very very clear that all they are doing is artificially re-encoding the template that corresponds to the the proper way to solve tasks including given dataset for instance I know if you've looked at a baby data set which is about a natural language question answering it is generated but not by an algorithm so this is question-answer pairs are generated by an algorithm the algorithm is following a certain template turns out if you craft a network that literally encodes this template you can solve this data set with nearly 100% accuracy but that doesn't actually tell you anything about how to solve question answering in general which is the point you know the question is just the linger on it whether it's from the data side from the size of the network I don't know if you've read the blog post by rich Sutton the bitter lesson yeah where he says the biggest lesson that we can read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective so as opposed to figuring out methods that can generalize effectively do you think we can get pretty far by just having something that leverages computation than the improvement of computation yes so I think rich is making very good points which is that a lot of these papers which are actually all about manually hot coding prior knowledge about the task into some system doesn't have to be deep in architecture but into some system right you know is these papers are not actually making any impact instead what's making real long-term impact is very simple very general systems that are agnostic to all these tricks because districts do not generalize and of course the one general and simple thing that you should focus on is that which leverages computation because computation the availability of large-scale computation has been you know increasing exponentially following Moore's law so if your algorithm is all about exploiting this the new algorithm is suddenly exponentially improving right so I think rich is definitely right either you know is right about the past 70 years is regressing the past 70 years I am Not sure that this assessment will still hold true for the next 70 years it's it's might to some extent I suspect it will not because the truth of his assessments is a function of the context right in which in which this research took place and the context is changing like Moore's law might not be applicable anymore for instance in the future and I do believe that you know when you when you we need to pick one aspect of a system when you exploit one aspect of a system some other aspect starts becoming the bottleneck let's say you have unlimited competition well then data is the bottleneck and I think we are already starting to be in a regime where our systems are so large in scale and so data and grain the data today and quality of data and the scale of data is the bottleneck and in this environment the the bitter lesson from rich is it's not going to be true anymore right all right so I think we are gonna move from a focus on a scale of a competition scale to focus on data efficiency their efficiency so that's getting to this the question is symbolic AI but to linger on the deep learning approaches do you have hope for either unsupervised learning or reinforcement learning which are ways of being more data efficient in terms of the amount of data they need that required human annotation so in supervised learning and reinforcement learning of frameworks for learning but they are not like any specific technique so usually when people say reinforcement learning but they really mean is deep printable version which is like one approach research actually very questionable the question I was asking was unsupervised learning with deep neural networks and deep reinforcement learning well he's not really a data efficient because you're still leveraging you know this huge biometric models trying point by point with quite understand it is more efficient in terms of the number of annotations the density of annotations you need so the AG being to to learn the Latin space or on which the data is organized and then map the sparse annotations into it and sure I mean that's that's clearly very good idea it's not real topic I would be working on but it's it's really good idea so we it would get us to solve some problems that it will get us to incremental improvements in in labelled data efficiency do you have concerns about short-term or long-term threats from a a from artificial intelligence yes definitely to some extent and what is the shape of those concerns this is actually something I've briefly written about but the capabilities of deplaning technology can be used in many ways that are concerning from you know massive variance with things like facial recognition in general you know tracking lots of data about everyone and then being able to making sense of this data to do identification to do prediction that's concerning that's something that's being very aggressively preferred by to tell italian states like you know china one thing I am I am very much concerned about is that you know our lives are increasingly online are increasingly digital made of information made of information consumption and information production our digital footprint I would say and if you absorb all of this data and a new are in control of where you consume information you know social networks and so on recommendation engines then you can build a sort of reinforcement loop for human behavior you can observe the state of your minds at time T you can predict how you would react to different pieces of contents how to get you to move your mind you know in a certain direction in the new then you can feed in video as the specific piece of content that would move you in in a specific direction and you can do this at scale you know at scale in terms of doing it continuously in real time you can also do it at scale in terms of skinning these to many many people to entire populations so potentially artificial intelligence even in its current state if you combine it with the internet with the fact that we have all of our lives are moving to digital devices and digital information consumption and creation what you get is the possibility to do to achieve mass manipulation of behavior and mass mass psychological control and this is a very real possibility yeah so you're talking about any kind of recommender system let's look at the YouTube algorithm Facebook anything that recommends content you should watch next yeah and it's fascinating to think that there's some aspects of human behavior that you can you know say a problem of is this person hold the republican beliefs the Democratic beliefs and this is but trivial that's an objective function and you can optimize and you can measure and you can turn everybody into a Republican or everybody absolutely yeah I do believe it's true so the human mind is is very if you look at the human mind as a kind of computer program it is a very large exploit surface right it has many many abilities ways ways ways you can control it for instance when it comes to your political beliefs this is very much tied to your identity so for instance if I'm in control of your news feed on your favorite social media platforms this is actually where you're getting your news from and I can of course I can I can choose to only show you news that will make you see the world in a specific way right but I can also you know create incentives for you to post about some political beliefs and then when I when I get you to express a statement if it's a statement that me as the as a controller I I want you I want to reinforce I can just show it to people who will agree and they will like it and that will reinforce the statement in your mind if this is a statement I want you to believe I want you to abandon I can on the other hand show it to opponents right we will attack you and because they attack you at the very least next time you will think twice about passing it but maybe you will even you know start believing this because you got pushback right so there are many ways in which social media platforms can potentially control your opinions and today the so all of these things are already being controlled by a Uyghur isms algorithms do not have any explicit political goal today while potentially they could like if some totalitarian government takes over you know social media platforms and decides that you know now we are going to use this knowledge for mass surveillance but also for mass opinion comes from and behavior control very bad things could happen but it was really fascinating and and actually quite concerning is that even with that an explicit intent to manipulate you're already saying very dangerous dynamics in terms of has this contact recommendation algorithms behave because right now the the goal the objective function of zalgar isms is to maximize engagement right which seems very innocuous at first right however it is not because content that will maximally engage before you know I get people to react in an emotional way I get people to click on something it is very often content that you know is not healthy due to public discourse for instance fake news are far more likely to get you to click on them than real news simply because they are not constrained to reality so they can be as outrageous as surprising as good stories as you want because the artificial right yeah to me that's an exciting world because so much good can come so there's an opportunity to educate people you can balance people's worldview with other ideas so the there's so many objective functions the space of objective functions that create better civilizations is large arguably infinite but there's also a large space that creates division and and and destruction civil war a lot of bad stuff and the worry is naturally probably that space is bigger first of all and if we don't explicitly think about what kind of effects are going to be observed from different objective functions then we can get into trouble but the question is how do we how do we get into rooms and have discussions so inside Google inside Facebook inside Twitter and think about okay how can we drive up engagement and at the same time create a good society is there is it even possible to have that kind of philosophical discussion I think you can different try so from my perspective I would feel rather uncomfortable with companies that are in control of these newsfeed algorithms with them making explicit decisions to manipulate people's opinions or behaviors even if the intent is good because that's that's a very totalitarian mindset so instead what I would like to see as probably never gonna happen because it's not super realistic but that's actually something I care about I would like all these algorithms to present configuration settings to their users so that their users can actually make the decision about how they want to be impacted by this information recommendation content recommendation algorithms for instance as a as a user of something like YouTube or Twitter maybe I want to maximize learning about a specific topic right so I want the algorithm to feed my curiosity right which is in itself a very interesting problem so instead of maximizing my engagement it will maximize half fast and how much I'm learning and it will also take into account the accuracy hopefully you know if the information I'm learning so yeah the user should be able to determine exactly how these algorithms are affecting their lives I I don't want actually any entity making decisions about in which direction they're gonna try to manipulate me right I want I want technology so aii these algorithms are increasingly going to be our interface to a world that is increasingly made of information right and I want I want everyone to be in control of this interface to interface with the world on their own terms so if someone wants this algorithms to serve you know their own personal growth goals they should be able to configure algorithms in such a way yeah but so I know it's painful to have explicit decisions but there is underlying explicit decisions which is some of the most beautiful fundamental philosophy that that we have before us which is personal growth if I want to watch videos from which I can learn what does that mean so if I have a check box that wants to emphasize learning there's still an algorithm with explicit decisions in it that would promote learning what does that mean for me like for example I've watched a documentary on Flat Earth theory I guess a it was very like that I learned a lot I really glad I watched it was a friend recommended it to me not I don't have such an allergic reaction to crazy people as my fellow colleagues do but it was very well it was very eye-opening and for others it might not be from others they might just get turned off for that same with a Republican Democrat and what it's a non-trivial problem when first of all if it's done well I don't think it's something that wouldn't happen that the youtubes wouldn't be promoting or Twitter wouldn't be it's just a really difficult problem how do we do how to give people control well it's mostly an interface design problem right the way since you want to create technology that's like a mentor or a coach or an assistant so that it's not your boss right you are in control of it you are telling it what to do for you and if you feel like it's manipulating you it's not actually it's not actually doing what you want you should be able to switch to different algorithm you know so that fine-tuned control you kind of learn you're trusting the human collaboration and that's how I see autonomous vehicles too is giving as much information as possible and you learn that dance yourself mmm yeah Adobe I don't use Adobe products like Photoshop yeah they're trying to see if they can inject YouTube into their interface but basically allow you to show you all these videos that cuz everybody's confused about what to do with feature so basically teach people by linking to and that way it's an assistant that shows users videos as a basic element of information yeah okay so what practically should people do to try to to try to fight against abuses of these algorithms or algorithms that manipulate us us it's a very very difficult problem because the star is very little public awareness of these issues there are a few people would think as you know anything wrong with their news feed algorithm even though there is actually something wrong already which is that it's trying to maximize engagement rest of the time which as a very negative side effects right so ideally so the very first thing is to stop trying to purely maximize engagement try to propagate contents based on popularity right instead take into account the goals and the profiles of each user so you will you will be one example is for instance when I look at tactic recommendations on Twitter's like you know they have this a news tab where I will switch recommendations it's always the worst garbage because it's it's content that appeals to them the smallest common denominator to all Twitter users because they are trying to optimize the purely trying to opportunist popularity the purely friendship you know as an engagement but that's not what I want so this should put me in control of some setting so that I define was the objective function and the twitter is going to be following - to show me this cannon so and honestly so this is all about interface design and we are not where it's not realistic to give you this control of a bunch of knobs that define algorithm instead we should purify man charge of defining the objective function like let the user tell us what they want to achieve how they want this algorithm to impact their lives so do you think it is that or do they provide individual article by article reward structure where you give a signal I'm glad I saw this or I'm glad I didn't so like a Spotify type yeah a feedback mechanism it works to some extent I'm kind of skeptical about it because the only way algorithm the algorithm will attempt to relate your choices with the choices of everyone else which might you know if you have an average profile that works fine I'm sure it's pretty far accommodations work fine if you just like mainstream stuff if you don't it can be a it's not optimal election will be in an efficient search for the for the part of the Spotify world that represents you so it's it's a tough problem but do notes that even even a feedback system like what Spotify has does not give me control over what the algorithm is trying to optimize for well public awareness which is what we're doing now it's a good place to start do you have concerns about long term existential threats of artificial intelligence well as I was saying our world is increasingly made of information a Iger ism so increasingly gonna be our interface to this wallet information and somebody will be in control of these algorithms and that puts us in in any kind of a bad situation right it has risks it has risks coming from potentially large companies wanting to optimize their own goals maybe profit maybe something else also from governments I might want to use these algorithms as a means of control organs of populations do you think there's existential threat that could arise from that so kind of existential threats so maybe you're referring to the singularity narrative where robots just take over well I don't not terminator robots and I don't believe it has to be a singularity we're just talking to just like you said the algorithm controlling masses of populations the existential threat being hurt ourselves much like a nuclear war would hurt ourselves mm-hmm that kind of thing I don't think that requires a singularity that requires a loss of control over AI algorithm yes so I do agree to all concerning trends honestly I I wouldn't want to make any any any long-term predictions I don't I don't think today we we really have the capability to see what the dangerous if they are going to be in 50 years in 100 years I do see that we are already faced with concrete and present dangers sir the negative side-effects of content recommendation systems of newsfeed algorithms concerning algorithmic bias as well so we are dedicating more and more decision processes to algorithms some of these algorithms aren't crafted some are Ireland from data but we are we are we are delegating control sometimes it's a good thing sometimes not so much and there is in general very little supervision of this process right so we we're still in this period a very fast change even chaos where society is is restructuring itself turning into an information society I wish itself is turning into an increasingly automated information processing society and well yeah I think the best we can do today is try to to raise awareness around some of these issues and I think we're actually making good progress if you if you look at algorithmic bias for instance three years ago even three years ago very very few people were talking about it and now all the big companies are talking about it there are often not in a very serious way but at least it is part of the public discourse you see people in Congress talking about it so and it all started from raising awareness right so in terms of alignment problem try to teach as we allow algorithms just even recommender systems on Twitter encoding human values and morals decisions to touch on ethics how hard do you think that problem is how do we have lost functions in neural networks that have some component some fuzzy components of human morals well I think this is really all about objective function engineering which it's probably going to be increasingly a topic of concerned if you like for now where we are just using very naive loss functions because the hard part is not actually what you're trying to minimize it's everything else but as the everything else is going to be increasingly automated we're going to be focusing on our human attention on increasingly high level components like what's actually driving the whole learning system like the objective function so the last function engineering is gonna be last function janilla is probably going to be a job title in the future you know and then the tooling you're creating with Kerris essentially takes care of all the details underneath and basically the human expert is needed for exactly that last engineer characters the interface between the data you're collecting and the business goals and your job as an engineer is going to be to express your business goals and your understanding of your business or your product your system as a kind of class function all kind of set of constraints does the possibility of creating an AGI system excite you or scare you or bore you so intelligence can never be be general you know at best it can have some degree of generality like human intelligence it's also always has some specialization in the same way that human intelligence is specialized in a certain category of problems is specialized in the human experience and when people talk about AGI I'm never quite sure if they're talking about very very smart AI so smart that it's Stephens modern humans or they're talking about human-like intelligence because it's our different things let's say presumably I'm impressing you today with my humaneness so imagine that I was in fact a robot so what does that mean I'm impressing you with natural language processing maybe if you weren't able to see me maybe this is a phone call yes Zack okay so companion so that that's very much about building human-like AI and you're asking me you know is this is this an exciting perspective yes I think so yes not so much because of what I artificial human-like intelligence could do but you know from an intellectual perspective I think if you could build truly human right intelligence that means you could actually understand human intelligence which is fascinating right yeah human-like intelligence is gonna require emotions it's gonna require consciousness which is not things that would normally be required by an intelligent system you do get you know we were mentioning your next science as superhuman problem-solving a agent not system it does not have consciousness enough emotions in general so emotions I see consciousness is being understand spectrum as emotions it is a component of the subjective experience that is meant very much to guide behavior generation right hands meant to guide your behavior in zone human intelligence and animal intelligence as evolved for the purpose of behavior generation right including in a social context so that's why we actually need emotions that's why we need cash is an artificial intelligence system developed in different context may well never need them may well may will never become just like science at that point I would argue it's possible to imagine that there's echoes of consciousness in science when viewed as an organism that science is consciousness so I mean how would you go about testing this hypothesis how do you I probed the subjective experience of an abstract system like science well the point of probing any subjective experience is impossible is that I'm not science I'm Lex so I can't probe another entities the another it's no more than when bacteria on my skewer lacks I can ask you questions about your subjective expanse and you can answer me and that's how I know you're conscious yes but that's because you speak the same language you perhaps we have to speak the language of science as I say I don't think consciousness just like emotions of pain and pleasure is not something that inevitably arises from any sort of sufficiently intelligent information processing it is a feature of the mind and if you've not implemented it explicitly is not there so you think it's a fee it's an emergent feature of a particular architecture so do you think it's it's a feature in the Simpsons so again the subjective experience is all about guiding behavior if you if if the problems you're trying to solve don't really involve and bedight agents maybe in a social context generating the view and pursuing goals like this and if you get fans that's not sure what's what's happening even though it is it is a form of artificial AR in artificial intelligence in the sense that it is solving problems this is a community knowledge creating a solutions and so on so if you're not explicitly implementing a subjective experience implementing certain emotions and implementing consciousness it's not going to just spontaneously emerge yeah but so for system like human-like intelligence system that has consciousness yeah do you think he needs to have a body yeah it's definitely I mean doesn't have to be a physical body right and there's not that much difference between a realistic simulation or your world so there has to be something you have to preserve kind of thing yes but human-like intelligence can only arise in the in human right context intelligence in other humans in order for you to demonstrate that you have human-like intelligence essentially yes so what kind of test and demonstration would be sufficient for you to demonstrate human-like intelligence yeah I just started curiosity you you talked about in terms of theorem proving and program synthesis I think you've written about that there's no good benchmarks for this yeah that's one of the problems so let's let's talk programs program synthesis so what do you imagine is the goods I think it's related questions for human-like intelligence therefore program synthesis what's a good benchmark for either both right so I mean you're actually asking asking two questions which is one is about quantifying intelligence and comparing the intelligence of an artificial system to the intelligence of a human and the other is about a degree to which this intelligence is human right is actually two different questions so if you look at you mentioned earlier the Turing test well I actually don't like the Turing test because it's very lazy it's it's all about completely bypassing the problem of defining and measuring intelligence right and instead delegating to a human judge or panel of human judges so it's it's it's at or cop-out right if you want to measure how human-like an agent is I think you have to make it interact with other humans maybe it's it's not necessarily good idea to have these other humans be the judges maybe you should just observe behavior and comparison where the human will actually have done when it comes to measuring how smart our clever an agent is and comparing that today to the degree of human intelligence so we're already talking about two things right the degree I kind of like the magnitude magnitude of an intelligence and its direction right like the norm of the vector right and its direction and the direction is like human likeness and the magnitude the norm is intelligence you could call it intelligence right so the the direction here your sense the the space of directions that are human-like is very narrow yeah so the the way you would measure the magnitude of intelligence in a system in a way that that also enables you to compare it to that of a human well if if you look at different benchmarks for intelligence today they're all too focused on skill at a given task let's scale that playing chess yeah spirit playing goes skillet playing Duda and I I think that's that's not the right way to go about it because you can always be too human it at one specific task the reason why our skill at playing goal or our juggling or anything is impressive is because we are expressing this skill within a certain set of constraints if you remove the constraints the constraints that we have one lifetime that we have this body and so on if you remove the context if you have unlimited string data if you can add access to you know for instance if you look at juggling at if you have no restriction on the hardware then achieving arbitrary levels of skill is not very interesting it and says nothing about the amount of intelligence you've achieved so if you want to measure intelligence you need to rigorously define what intelligence is which in itself units it's a very challenging problem and do you think that's possible if you define integers yes absolutely I mean you can provide many people have provided you know some definition I have my own definition where does your definition begin if it doesn't end well I think intelligence is essentially the efficiency with which you turn experience into generalizable programs so what that means is it's the efficiency with which you turn a sampling of experience base into the ability to process a larger chunk of experience base so measuring skill can be one proxy because many management tasks can be one proxy for measure intelligence but if you want the only measured skill you should control for two things you should control form a mod effects be ins that your system has and the priors that your system has but if you if you control if you look at two agents and you give them the same priors and you give them the same amount of experience there is one of the agents that is going to learn programs representation something the model that will perform well on the larger trunk effects payin space and the other and that is the smaller agent yes so if you you have fixed the experience which generate better programs get better meaning more generalizable that's really interesting and that's a very nice clean definition of oh by the way in this definition it's it is already very obvious that intelligence has to be specialized because you're talking about experience space and you're talking about segments of experience space you're talking about priors and you're talking about experience all of these things define the context in which intelligence emerges and you you can never look at the totality of experience space right so intelligence has to be specialized and but it can be sufficiently large the experience space even though specialized there's a certain point when the experience base is large enough to where it might as well be general it feels general it looks general sure I mean it's it's very less developed for instance many people would say human intelligence is general in fact it is it is quite specialized you know the we can definitely build systems that start from the same innate priors that's what humans have at Birth because we already understand very well what sort of priors we have as humans like many people have worked on this problem most notably as a bethe a spelke from how about I know if you know her his work the rotten and what she calls a core knowledge and it is very much about trying to determine and and and describe what priors we are born with like language skills and so on and all that kind of stuff exactly so we we have some some pretty good understanding of what price we are born with so we could so I've actually been working on a benchmark for the past couple years you know on earth I hope to be able is it at some point is to measure intelligence of systems by culturing for priors culturing for amount of expands and by assuming the same priors as with humans are born with so that you can actually compare this course to human intelligence and you can actually have humans pass the same test in in a way that's fair yeah and so importantly such a benchmark should be such that any amount of practicing does not increase your score so try to picture a game where no matter how much you play this game that does not change your skill at the game can you picture that as a person who deeply appreciates practice I cannot actually so it is not I can a there's actually a very simple trick so in order to come up with a task so the only thing you can measure is skill at the task yes all tasks are gonna involve Pryor's here the trick is to know where they are and and to describe that and then you make sure that this is the same set of priors as what human stuff is so you create a task that assumes this priors that exactly documents is brightest so that's the price I made explicit and I'll no other priors involved and then you generate a certain number of samples in experience base for this task right and this for one task assuming that the task is new for the agent passing it that's one test of this definition of intelligence and between that we set up and now you can scale that to management tasks that all you know each task should be new to the agent bassinet so the switch will be human human interpretive ball and the son of also lets you can actually have a human pass the same test and then you can compare the squash a machine and squash your human which could be a lot as they could even start a task a chemist just as long as you start with the same set of yes so the problem is M missed humans already trained to recognize digits right and but let's say let's say we're considering objects that are not digits some completely arbitrary patterns while humans already come with visual priors about how to process that mm-hmm so in order to to make the game fair you would have to isolate these priors and describe them and then express them as computational rules having worked a lot with vision science people as exceptionally difficult process has been a there's been a lot of good tests and basically reducing all human vision into some good priors and we're still probably far away from that perfectly but as a start for a benchmark that's an exciting possibility yeah so I said with spelke actually lists abjectness as one of the core knowledge buyers abjectness Koha Brickner yeah so we have priors about object nests like about the visual space about time about agents but goal-oriented behavior we have many different priors but what's interesting is that sure we have you know this is pretty diverse and an enriched set of pairs but it was - not that diverse right we are not born into this world wheel with a ton of knowledge about the world with only a small set of cual knowledge he has hardly ever sense of how it feels to us humans that that set is not that large but just even the nature of time that we kind of integrate pretty effectively through all of our perception all of our reasoning maybe how you know do you have a sense of how easy it is to encode those priors maybe it requires building a universe mm-hmm and the human brain in order to encode those priors what do you have a hope that it's can be listed like an accent I don't think so so you have to keep in mind that any knowledge about the world that we are born with is something that has to have been encoded into our DNA by evolution at some point right and Gina is a very very low bandwidth medium like it's extremely long and expensive to include anything into DNA because first of all you need some sort of evolutionary pressure to guide this writing process and then you know the higher level information in trying to write the longer it's gonna take and the thing in the environment that you are trying to encode knowledge but has to be stable over this this duration yes so you can only include into DNA things that constitute an evolutionary advantage so this is actually a very small subset of all possible knowledge about the world you can only encode things that are a stable that are true with a very very long period of time to begin millions of years for instance we might have some visual prior but the shape of snakes right but all what makes a face what's the difference between a face and on face but consider this interesting question do we have any innate sense of the visual difference between a male face and a female face what do you think for human I mean I would have to look back into evolutionary history when the genders emerged but yeah most I mean the faces of humans are quite different to my face of great hips great apes right yeah like you didn't say you couldn't tell the face of human pansy from the face of a male ship and she probably yeah that'll hide us humans of all that so we do have innate knowledge of what makes a face but it's actually impossible for us to have any DNA encoded knowledge of the difference between a female human face and a male human face because the that knowledge that information came up into the world actually very recently if you look at the at the slowness of the process of encoding knowledge into DNA yeah so that's interesting that's a really powerful argument the DNA is a low bandwidth and it takes a long time to encode here that naturally creates a very efficient encoding but hence the yeah one one important consequence of this is that so yes we are born into this world with a bunch of knowledge sometimes I high-level knowledge about the world like the shape the rough shape of the snake of the raft shape of face but importantly because this knowledge takes so long to write almost all of this innate knowledge is shared with our cousins with with great apes right so it is not actually this innate knowledge and that makes us special but to throw it right back at you from the earlier on in our discussion it's that encoding might also include the entirety of the environment of Earth to some extent so it can it can include things that are important to survival and reproduction so the for which there is some evolutionary pressure and things that are a stable constant over very very very long time players and honestly it's not that much information there's also beside the bandwidths constrain and constraints of the writing process there's also a memory constraints like DNA the part of DNA that deals with the human brain is actually very small it's like you know on the order of megabytes right it's not that much high-level knowledge about the world you can encode that's quite brilliant and hopeful for benchmark of that you're referring to of encoding priors actually look forward to i'm skeptical whether you can do in this couple years but hopefully i've been working so honestly it's a very simple benchmark and it's not like a big breakfast or anything it's more like a fun a fun side project right these fun so is imagenet these fun side projects could launch entire groups of efforts towards uh towards creating reasoning systems and so on and i think yeah that's Nicole it's trying to measure a strong generalization to measure the strength of abstraction you know minds right now mind something in a in a fishery contagion and if there's anything through about this science organism is its individual cells love competition so in benchmarks encourage competition so that's uh yeah that's an exciting possibility if you are do you think an AI winter is coming and how do we prevent it not really so an AI winter is something that would occur when there's a big mismatch between how we are selling the capabilities of VI and and the actual capabilities of VI and today's when the planning is creating a lot of value and we keep creating a lot of value in the sense that this is models are applicable to a very wide range of problems that are written today and we are only just getting started with the crimes algorithms to every problem they could be solving so the planning will keep creating a lot of value for the time being what's concerning however is that there's a lot of hype around deplaning anaronnie idea lots of people are over selling the capabilities of these systems not just the capabilities but also over selling them the fact that they might be more or less in a brain like like you've given a kind of a mystical aspect these technologies and also over setting the pace of progress which you know it might look fast in the sense that we have this exponentially increasing number of papers but again that's just a simple consequence of the fact that we have everyone more people coming into the field doesn't mean the progress isn't is actually exponentially fast like let's say you're trying to raise money for your startup or your research lab you might want to tell you know a grandiose stories to investors about how deep learning is just like the brain and hide consume all these incredible problems like self-driving and robotics and so on and maybe you can tell them that the field is progressing so fast and we are gonna have HDI within 15 years or even ten years and oh none of this is true and every time you're like saying these things and an investor or you know a decision maker beliefs them well your this is like the equivalent of taking on credit card debt yeah but for for trust right and maybe this win you know this will this will be what enables you to raise a lot of money but she ultimately you are creating damage to our dimensional fields that's the concern is that that that that's what happens the other day I winters as the the concern is you actually tweet about the skooled autonomous vehicles right there's a almost every single company now have promised that they will have full autonomous vehicles by Twenty twenty one twenty two this is a good example of that the consequences of overhyping the capabilities of AI and the pace of progress because I work especially a lot recently in this area I have a deep concern of what happens when all these companies after I've invested billions have a meeting and say how much do we actually first of all do we have an autonomous vehicles the answer it will definitely be no and second will be wait a minute we've invested one two three for a billion dollars into this and we made no profit and the reaction to that may be going very hard in another directions that might impact either even other industries and that's what we call in the air winter is when there is backlash well no one believes any of these promises anymore because they've turned that be big lies the first time around yeah and this will definitely happen to some extent for autonomous vehicles because the public and decision makers have been convinced that you know around around 2015 they've been convinced by these people who are trying to raise money for a start-up and so on that l5 driving was coming mean maybe 2016 maybe 2017 may 2018 now when 2019 was still waiting for it and so I I don't believe we are going to have a full-on AI winter because we have this technologies that are producing a tremendous amount of free all value right but there is also too much hype so there will be some backlash especially there will be backlash so you know some startups are trying to sell the dream of AGI alright and and the fact that Asia is going to create infinite value like EG is like a free lunch like if you can if you can develop an AI system that passes a certain threshold they of IQ or something then suddenly you have infinite value yes and well there are actually lots of investors buying into this idea and you know they will wait maybe maybe 10 15 years and nothing will happen and and the next time around well maybe maybe there will be an in generation of investors no one will care you know a human memory is very short after all I don't know about you but because I've spoken about AGI sometimes poetically like and I get a lot of emails from people giving me they're usually like a large manifestos of they've they say to me that they have created an AGI system or they know how to do it and there's the long write-up of how to doing it so that was easy man they are there little bit feel like it's generated by an AI system actually but there's usually no guidance recursively sitting here exactly it's you have a transformer generating crank papers about this yeah so what the question is about because you've been such a good you have a good radar for crank papers how do we know they're not onto something how do I so when you start to talk about a GI or anything like the reasoning benchmarks and so on so something that doesn't have a benchmark it's really difficult to know I mean I talked to Jeff Hawkins who's really looking at neuroscience approaches to hug and there's some there's echoes of really interesting ideas in at least just case which is Charlie how do you usually think about this they like preventing yourself from being too narrow-minded and elitist about you know deep learning it has to work on these particular benchmarks otherwise it's trash well you know the thing is intelligence does not exist in the abstract intelligences to be applied so if you don't have a benchmark if you're an improvement and some benchmark maybe it's a new benchmark all right maybe it's not something I've been you again before but you juni is a problem that you're trying so you're not gonna come up with a solution with that a problem so you general intelligence I mean you've clearly highlight a generalization if you want to claim that you have an intelligent system it should come with the benchmark issued yes it should display capabilities of some kind it should it should show that it can create some form of value even if it's a very artificial form of value and that's also the reason why you don't actually need to care about turning which papers actually submit in potential and which do not because if if there is a new technique it's actually creating value you know this is going to be brought to light very quickly because it's actually making a difference so it's the difference between something that's ineffective and something that is actually useful and ultimately usefulness is our guide not just in this field but if you look at science in general maybe there are many many people over the years that have had some really interesting theories of everything but they were just completely useless and you don't actually need to tell the interesting theories from the user series all you need is to see you know is this actually having an effect on something else you know is this actually useful it is this making an impact or not as beautiful put I mean the same applies to quantum mechanics to a string theory to the holographic principle we are doing the planning because it works you know that's like before I started working people you know I considered people working on our neural networks as as cranks very much like you know no one was working anymore and now it's working which is what makes it valuable it's not about being right right it's about being effective and nevertheless the individual entities is a scientific mechanism just like yoshua bengio a young McCune they while being called cranks stuck with it right yeah and so us individual agents even if everyone's laughing at us just stick with it because if you believe you have something you should stick with it and see it's true that's a beautiful inspirational message to end on first of all thank you so much for talking today that was amazing thank you you

Channel: Lex Fridman

Views: 117,842

Rating: 4.9225726 out of 5

Keywords:

Id: Bo8MY4JpiXE

Length: 119min 49sec (7189 seconds)

Published: Sat Sep 14 2019