Stanford Seminar - Artificial Intelligence: Current and Future Paradigms and Implications

I'm gonna be presenting lots of different work from uh certain people and a bunch of it is created by my team on the other side of the bridge I work with about sixty scientists from a bunch of different places in many different countries and so a lot of the work that I'll be sharing that's from vicarious itself is theirs and we're we're lucky to be funded by some of my personal heroes we've raised a little over 120 million dollars from people like Bezos and Zuckerberg and Elon so on and we're actually advised by one of one of Stanford's in faith de leeuw is is one of our advisors so we have a a gender today talked about a couple different things one is about the strengths and limitations of modern AI systems and I'll try and frame it in a way that's acceptable to people who know a lot about it and also understandable to people who know a little bit about it so I'm going to try and walk a very fine line and then I'll run through some of the different economic social and policy implications that I expect this to have in the near term and then this in future and then finally I'd like to give at least my set of predictions from where I sit in the industry about what I expect to be coming on the horizon so I think there's this really interesting relationship or lens we can have on the development of artificial intelligence that kind of maps it to the evolution of animal intelligence so if we go back 600 million years we get to the very first intelligent animals things like jellyfish flatworm sea sponges and then as time passes the animals get more complicated I would argue they don't get any smarter it's just deeper layers of very simple heuristics very narrow kinds of behaviors in the neuroscience world this is called the old brain and it's built on stimulus and response it's build an instinct and then about 100 million years ago or 200 million years later yeah 100 million years ago we got a completely different architecture for intelligence that evolution figured out called the neocortex the new brain it's what gave us primates whales dolphins and you and me and instead of being built on top of stimulus and response and instinct it's built on top of reasoning about causality reasoning about what if and why and mental simulation imagination those are the sort of the hallmarks of mammals versus our older brain ancestors and the argument all makes today is that deep learning and you know largely the current generation of AI systems that we see the proliferation of are in the big yellow box here they're they're largely based on instinctual stimulus response kinds of behaviors and I'll give you some examples from the animal kingdom and also from the computer kingdom so we can compare and contrast so from it's not to say that there's anything wrong with the old brain like old brain animals are really cool they can do all kinds of stuff that can navigate complicated environments that can control their limbs you know they can reproduce the old brain is really really cool but it has some really important limitations to keep in mind when we think about how it maps to AI system so specifically to make an old brain animal you need a couple of hundred million years worth of evolutionary training data and when you finish you're a Volvo process for that animal you're going to get a system that has very low generalization to new environments that has very low ability to learn new skills or adapt and most importantly what will be produced is an animal that gives the illusion of intelligence without actually being smart so like it will give you a perception that the animal is doing a behavior that is what a human might do in the same circumstance without actually having any of the reasoning the back set up but I'll give you some examples of that so in a nutshell when you look at the old brains architecture you have a you know a sound or a video or a picture signal that comes into the system there's a bunch of very narrow heuristics that are stored in the animal's brain and then it and then it takes some action form some response and so here's some sort of fun examples baby geese make this noise let's see if you can hear it and scientists figured out that mother geese will actually love and care for anything that makes that sound so you can take a tape recorder that plays that noise and put it inside of a taxidermy of a wolf and the mother goose will love and care for the wolf as if it's one of its own children and similarly baby ducks exhibit this really cool behavior where they follow their mom and a very orderly fashion and so you know if you what they're actually doing is not flying their mom they're following the first thing they saw when they were born so if the these baby ducks are born they see a dog they will follow the dog everywhere as if it's their mom so it's like a very specific very narrow heuristic the system is using in order to produce intelligent looking behavior and the same thing is true frogs the Frog will actually starve itself to death in front of an iPhone because it's just performing the action if you see this pixel pattern you shoot the tongue and these are the same kinds of behaviors you'll see in old brain AI systems you know deep learning is great it can do lots of really cool looking things I can play tarah games or go games they can even recognize photos and and control robots but it has the same limitations as the old brain does so it takes a lot of training data to make a deep learning system work the resulting model is not going to generalize well to new environments or circumstances and it's not going to understand cause and effect or understand what it's doing or why it's it's going to give you the illusion of intelligence and it even has the same architecture as our old brain friends so I want to give you guys a tutorial about why this is and just to take a quick temperature so who here feels like they could give a concise explanation of how alphago works how many people ok like maybe 10% of you maybe less so ok so I'm gonna give you a tutorial about how at the end of this to be able to tell your friends how alphago works in a way that is I think pretty true it's not exactly how alphago works because that would be much longer talk but it's close enough that it may as well be so let's say we're gonna build an AI system for reading handwritten digits like these step one in building a system like this is to collect thousands or millions of hand labelled human labelled training examples so this is six thousand examples of the number zero this is six thousand examples the number one of the number two and so on you gather 60,000 labeled training examples and you feed them to the system and you win then when you show it a new image like this basically what it's doing is it compares this to every image it's seen and it's training data set so you know flip through all the ones the twos the threes and so on and eventually gets here and it's like wait a minute these are really close matches so you know voila it's a four and if you want to take that system and scale from recognizing digits recognizing photos you again collect a very large corpus of hand label images like for example we've labeled this image corn because somewhere in the image there's corn and then instead of 60,000 training examples you'd show it over a million this is the image net training set courtesy of Stanford and then at the end of that process you could showed a new image and it would say this matches what I know about cars but if you showed a picture like this one it'll say it's a bedroom pillow instead because it doesn't really learn anything about cars it's just doing very dumb pattern matching and similarly you know it may recognize this is a Jaguar but a hand drawing of the exact same image it's gonna have no idea what that is and it will also recognize images like this as Jaguars because it's just doing sort of basic pattern matching and similarly it's important to realize it's not actually learning anything about objects it like doesn't know that there's a such thing as an object it's just matching patterns and it's matching the patterns that sees in photos all mixed together so like if I keep the bus the train and the cat the same in these pictures and I just changed the background a deep learning system will say that it's a nymph in snowplow and a chihuahua so it's it's a very very simple and kind of narrow way of recognizing things but it happens to work if you have a really big data set in a lot of compute and it's susceptible to the kinds of tax maybe some of you seen before where because it's doing a very narrow form of pattern matching you can take an image that it will recognize yes which cues are geometric or are you looking at color maps or that's so that's the funny thing about learning is it's hard to tell you get a big monolithic black box out of it and there's some dark magic you can do to kind of try and figure out how it's coming to the inclusions it's coming to but the the real answer is you really have no idea what the weights inside the network mean and it happens to work some of the time it happens to fail other percent of the time your question multi little neural networks there are certainly other ways of doing pattern recognition and receiving learning Italy decision trees are right and we'll get there is it yellow or it's a green or whatever yeah so that's I think that's getting closer to the second half of the talk about where I think things are going because right now when you when you read about AI you're generally speaking and reading about deep learning and you know every every conference submission deep learning is is basically if that's not on the title somewhere or in the abstract somewhere then you pay a penalty in terms of the likelihood of your paper being accepted so it's also susceptible these attacks where you know you you can take an image that it will recognize as a bus with high confidence you can perturb it very slightly and that's just enough to cause it to miss recognize it as a completely different class and you can do this with with any photo there's lots of different ways of generating these adversarial cases that fool deep learning and do not fool humans just so you know drill home the point that these systems are not doing human-like reasoning about anything they're just doing very narrow sense of pattern recognition and at the same time you can also generate completely nonsense images that it will recognize with very high confidence as all kinds of things actually have a deep learning app on my phone that I can take some pictures of these in your head right now see if you can classify the photos that are in front of you exactly I think that's a that's a side effect of the generator rather than of the technology itself so let's see what this one has to say any guesses so all of them have similar kinds of reactions so you can generate a nonsense patterns that fit into the manifold but the deep learning system has been trained recognized without actually being those objects so this is not this is not a deep learning as bad talk this is just one of the characteristics of deep learning and this is one of them so to make a deep learning system work you need lots of training data and the resulting system is not guaranteed to generalize particularly well to new environments and it doesn't understand what it's doing doesn't have a causal model of its sell for of the world that it can be used to reason and so when you and I watch this is a google deepmind's famous Atari player that's using deep reinforcement learning we watch the system we say ok the Pat it's using the paddle to hit the ball to get the hit the bricks to get points which is not at all what's happening inside the AI system what's actually happening inside the system is they just take three frames or four frames of the game and they paste them together into one image so that they make a whole bunch of copies of this and to give you a sense of the amount of data here this is our 60,000 digits this is the million photos and this is the 50 million Atari frames this is trained on so they paste you know these three frames or four frames together and then they do that 50 million times and then after playing the game at random to start with and playing it more and more you eventually begin to notice oh I went left and I lost points so I'm going to classify that image or that series images as a go right series and images and then vice versa and then over time you're able to to do pattern recognition where when you see a series of frames you classify it as whether you should go left or go right and you can tell that that's what the system is doing because small changes break this algorithm in the same ways that changes break the other you know deep learning systems we looked at so if you want to for example change the rules of the game slightly where the ball bounces higher or the paddles bigger or it's in a different position all of these things you'll need to retrain your deep learning system from scratch with another 50 million frames of the game and you can see it really very clearly so I took that train model and just increase the brightness of the game by 2 percent so increasing the brightness of the game with 2 percent is enough to cause it to do this instead of playing the game and with alphago it's it's almost the same story you take you know an input go board and down here now is our 60,000 digits this is our million photos that's the 50 million Atari frames and this is the billion go boards that's required to Train alpha go to play go and to put that in perspective that's about three thousand years of a human playing go for 24 hours a day 7 days a week without sleep and so it's a it's a lot of training data and then after those billion go boards are played it gets and it tallies up the scores every time it gets a sense of what are the boards that look like losing boards we're the ones look like winning boards and so anytime it's seeing a you know a new go board like this one it can then search over all possible descendant states of the board and pick the ones that are the winningest and voila you have alphago so if you wanted to play a different game like chess instead of go it's another three thousand years of training changing the rules slightly all the things that humans adapt to really well are things that the system is not going to do you can also apply these same kind of architectures to control robots this is Google X's recent achievement called the arm farm they wanted to have a robotic system that could grasp objects and to get these systems to grasp objects that are all roughly the same size it took running it continuous running 14 of these arms continuously for months performing 800,000 practice graphs and collecting 2.8 million images and so after all of that you basically store images like this pair and then you record did I grasp something or not and then you can create a gradient where you're like okay these are pictures where I'm unlikely to grasp these are pictures where I'm more likely to grasp and do the same kind of regression operation that we've done at all the previous examples and note like so the system after all this training it fails twenty percent of time and when it does succeed you can't tell it what to grasp like it's just gonna try and grab something and sometimes it'll grab two things and when it succeeds of grasping something it succeeds in a way I wouldn't characterize as success like it doesn't have a good plan about how to grab this object it happens to work in this particular case but if you change the gripper or even the color of the bin I'm not convinced it would still work especially if you change the color of the bin I think the color of the bin would cause it to have a catastrophic failure so so what can people earning do what is a good at it's it's it's great if you have a lot of data and you have a lot of computers you can make really accurate and narrow predictions and it's basically doing fancy regression and you know if I'm the CEO of Google or Facebook or something that's a really great thing in fact that may be the best thing ever because what I have are a lot of computers and a lot of data and what I want are really narrow predictions about what ad to show someone what person's face might be in a photo and so deep learning is terrific and that's why they invest so much money in it and it has some interesting implications so just this kind of as narrow as it is this kind of pattern matching actually I think is is pretty powerful and transformative so I'm not a policy wonk and not an economist and so I wanted to disclaim all the things I'm about to say as someone who's not that much more informed about this than you are but I'll share with you the stuff I've and this is also you know aggregating from many other sources that I've read about this topic and so for further reading all offer some recommendations but um I think there's three main classes that I I consider any way I want to think about implications of this kind of narrow AI economic social and policy economic implications so my favorite lens on what these kinds of narrow AI systems is is going to do is is as a decrease in the price of prediction because if you view these systems as just lowering the price of prediction then you can use all of the standard economic tools to reason about what kind of effects its gonna have an economy so you can say well the you know decrease in the price of rubber you're gonna see use going up and you're gonna see the value of the complements going up because we're people were using it so more people need the complements and this is there's a book that just came out recently called prediction machines that I would recommend anyone who wants to go deeper into what are the economic implications of you know standard modern AI deep learning so and then when you say use increases like you can use these kinds of systems everywhere to predict whether something is spam or not you know predict what I'm saying when I talk to Siri predict who I'm likely to vote for predict what you might want to show me if you want to change who I'm likely to vote for so as you get down this list it sort of gets darker and a little bit more interesting the other implication is on the value of complements the two complements to systems like this are data and compute and so companies that make computers that are very useful for these kinds of operations like Nvidia are going to see a dramatic increase in the demand for their products companies that have a lot of data that can be utilized in these ways like Facebook or Google are also going to see dramatic increases in the value they're able to capture from that data next certifications around society so it might sound and it sounded to me anyway like a really good thing that we could have systems that would show me more articles that I wanted to see and less articles that I didn't want to see like I don't want to read articles that are boring so show me articles that are interesting and as it turns out this is actually kind of disastrously bad for society because if everyone only sees articles that they like to read and as it turns out people like to read articles that they agree with and that in fact capitalize on their like worst human instincts and prejudices then what we get are a bunch of people who all believe their view of the world is right and all have the most polarized version of their view of the world and so there's this experiment the Wall Street Journal did called red feed blue feed that I would recommend all of you check out and it just takes the most biased articles from both sides of the spectrum and puts them right next to each other on the same topic and you get to see what each side which facts each side chooses to emphasize or invent and how you know when you're in one of these filter bubbles you don't know it no one's telling you that there are other views out there and so a lot of the population of the world who's on these systems just thinks that their perspective on the truth is the truth and so you end up with a society that's much more divisive than it had been when we just had nightly news and we all watch the same four channels or something to singulate on what would be the truth and then the other implications for this is once you have these filter bubbles then you can identify what kind of information would be necessary to cause the bubble to shift one or the other and that's kind of stuff we saw in the election and other countries other entities like what we're seeing with Russia can then choose to influence what's going on in people's minds by making targeted ad buys or targeted information campaigns at these bubbles next certifications I think about what I think about the narrow edits here is about research and policy right now we're being like catastrophic Lee outspent by China on fundamental research on AI robotics other fields and I can tell you what China's strategy is for for AI robotics and being the leader and all of these things and I feel like I can give you a coherent explanation of what the United States the strategy is which is something that makes me uncomfortable I really feel like our government should be thinking very critically about how we should be taking advantage of the new technologies that were born here and helping them to reach the next level more quickly and the other thing is and this goes into the section I'll get into about jobs America's infrastructure is really falling apart right now and has been for a while and to repair it it's not something that narrow AI systems like this can do and so as it turns out you a lot of people with hands and good skills and carpentry and and like stonework and concrete in order to repair these things and we've kind of lost some of our ability to do these things last piece of policy is on immigration so all six of the people you probably recognize almost all six these people so it's it's a SpaceX Google Amazon Yahoo eBay and an apple all of these people were either immigrants themselves or the children of immigrants and so I think one of the things that has made America what it is is our attitude towards welcoming brilliant people from elsewhere and integrating them and by adopting a different stance than that I think that we're killing the Golden Goose yeah since you're going to leave policy before you leave policy yeah your previous point is really saying that we the US needs an industrial policy and it has been the position of the United States government since the end of the Second World War that having a Industrial Policy is something that we'll never do yeah so I'm not advocating for industrial policy specifically and you sound very very informed much more informed than high office so do you think of the NIS as industrial policy or as basic science research because I'm not advocating for more that oh just go back and listen to your words about what you said we need to do yeah funding for fundamental research in AI and robotics previously were a little bit different okay well I retract my previous words and replace them with I want funding and fundamental research in AI robotics we had an interesting wouldn't he pick on the University of Southern California Chancellor of the University of Southern California said the United States government should spend more money to help us compete with China today I yeah he neglected to say that all of the money that is spent and you want to see yeah he's effectively spent educating Chinese nationals to take their AI back to China so it's spending money at USC yeah did not actually a call Perceval it did actually help in with his bowl but just fundraising pretty gutsy right no for whatever but that is yeah so it's a little weird to say we're going to do this to to compete with somebody well you're not gonna give everything you've made to them yeah but I think you're talking about a specific instantiation of how the grants are dispersed and like to whom receive them and I think there can be a little bit more granularity on that to go thing the devil video picture pictures up there the answers changed remarkably oh we have a long history of you know spending a trillion dollars and the result was five signs saying this project funded by yeah exactly and I think this is there's no substitute for like thoughtful allocations of capital and for carefully designed programs and for all the things that are acquired but we can't use the money we don't spend to make advances when we need to go backwards and actually fix the mechanisms by which grants are distributed in like all these things yeah there's lots of bugs in system to fix this topic is about the top of the funnel and and you know the middle on the bottom of the funnel you know there's a hole there talk about that maybe I mean I know point I think is a harder line stance than I think could possibly be justified because even if 80% of the money is lost by like it just sent directly overseas or something and 20% of it fund projects like ARPANET then we end up with something that could be revolutionary so I don't I don't know that you can flush all of it so would you advocate excusing up figuring out which agency to use up in one way or another and juicing up an agency or would you say we need an overall overarching strategy which has a budget I mean I'm going to refer back to my my disclaimer slide and say what I'd advocate for is that someone who's very smart and has spent a lot of time studying these kinds of policies and programs be put in charge of getting us a good one and measuring how successful it is or is not in making adjustments to it so that it is successful I think that would be an interesting stuff I love that disclaimers for things to be good yeah the US would become an agrarian nation yeah I'm not asking for any the money I mean I I think this is this is also the kind of thing where I think there's a false narrative in the press about how it's like US versus China on this were like I think the discoveries that are made on the path towards creating better AI or discoveries that are gonna be broadly beneficial and not selectively beneficial yeah and I don't want to hitch my wagon to anyone who's advocated for you know somebody else wants funding for improved RF technology somebody else got some butter yeah for something else yeah so you have all of these these things that you kick that's where you end up having have an industrial policy job you have to say well what is it that we want to achieve and then you have to back back into it yeah exactly we that would that would be a part of the process that we need to occur yeah haha okay great so and and and and it sounds like lots of people here in this room have strong opinions about this and it would be fun to have a fog conversation after the talk okay so next part of this is kind of talking about where we go from here obviously these deep loading systems are effective at a certain range of things but to get to the next level as you mentioned the being of this talk I don't think we can rely exclusively on big data pattern-matching old brain kinds of systems and I think that we can draw some inspiration for what happened and evolution in order to figure out where exactly we look for the next generation of these systems I'll give some examples from the animal kingdom so whales in captivity are trained to pick up trash in their tanks and trade them with the trainer for fish and one day a seagull died and fell into the tank and instead of getting one fish as reward the animal got to because it was like a big object or something and you imagine a frog would just like eat both rewards and now would be that the whale did not the whale saved the second fish and then used it as bait to catch additional seagulls which had then drowned and trained with a trainer for even more fish create a stockpile of fish at the bottom its tank and then it used those fish to teach the other whales in the tank how to participate in the seagull for fish economy the next favorite example so there's a gorilla in captivity named Coco that this is not Coco but I'm not allowed to use footage that I don't copyright to so this is just a gorilla so Coco was raisin kept in view her favorite thing to do is watch a TV program called Mister Rogers neighborhood which many of you probably saw mr. Rogers found that Coco was a fan of his show so she went to visit him at the zoo and if you recall mr. Rogers show he starts it by like welcoming the children to his house he takes off his shoes and he teaches them the lust of the day the first thing KOCO wanted to do with mr. Rogers when he met him when she met him was to help him take off his shoes because that's how he always started a show last example of these kind of new brain behaviors this is an 18 month old human being put in a new environment it's never been in before looking at objects it's never seen before watching a human it's never met before do an action it's never seen a human do before and it's given no instructions [Laughter] so why is it that humans are able to do and other mammals are able to do these amazing feats of very low training data very high generalization reasoning and the answer is you know we have a radically different circuit in our heads than our our reptile friends or insect friends we still have the old brain it's down there controls our heart rate our breathing our immune system all kinds of important things but then on top of it there's this completely different circuit called the neocortex if you cut it off the brain and you lay it flat it's the size shape and thickness of a dinner napkin and different parts of it correspond different functions like there's a part for hearing a part for seeing part for language part for motor actions part for spatial reasoning but it's all the same roughly the same replicated circuit so if you take a tissue sample from each of those regions and give them to a trained neuroscientist they'll have a hard time telling which is from which region and that's also even true across mammals so if you give a neocortical sample from a whale of rhinoceros a human and a mouse all of those neocortex's have remarkably similar wiring patterns and they can even do these really cool experiments where they they take a ferret and they sever the optic nerve and instead of allowing it to grow into the part of the neocortex it normally processes vision they force it to grow into the part that normally processes is hearing and the ferret can see so it's a it's a replicated circuit that's performing the same kind of mathematical operation and it actually has the inverse characteristics of deep learning so mammalian brains take incredibly little training data in order to learn new concepts new ideas they generalize really well to new environments and circumstances and most importantly they learn this causal model the world that lets us ask what if and why reason backwards from effects their causes and simulate forward in time and that is a really important thing to have if you're in a body and you want to solve problems with your body because we don't get infinite training data to be in this particular room manipulating these particular objects chances are you just get one chance at it and so you need a system that's very very fast at learning to be in new environments and reusing knowledge it already has that's the kindest and so we work at it like curious in contrasts the kind of deep learning architectures where you just have a stimulus a black box and response we work on systems that try and build causal relationships in the world build a causal understanding so we can do forwards and backwards reasoning and I'll give you some examples of that and we focus on these kinds of causal models from a business perspective anyway because there's this really weird paradox that we all live in where the parts that go in robots like motors and plastics and metal and electricity and processors sensors are all really cheap and nobody owns any you know if you go into a factory a hundred years ago and a factory now like basically nothing changes it's like you get color color is what changes and we have people doing jobs that robots have been physically capable of doing you know for over a decade and the reason we have that is because we don't have an intelligence layer that you can use to control the robot so this is a video of basically Rosie the robot from the Jetsons and the trick of this video it's a video from 11 years ago the trick of this video is the robots being controlled by a human so as long as you have a human brain you can do almost anything you want with a robot if I just gave you a normal robot gripper and had you live your everyday life with it I think you could do 90% of the things that you do every day and so functionally we're already living in the Jetsons society we're just missing the advances in AI necessary to make robots useful and ubiquitous and that's what we're building it vicariously that's what a lot of our research leads into it's sort of like until inside robots I'll just get through this part so when we talk about how you know how do we approach the problem of working on AI specifically we're gonna have for embodiment there's a very famous computer science theorem called the no free lunch theorem which basically they prove that every algorithm only gets power by making assumptions so like mp3 was good at compressing songs because it assumes that you're a human and you're listening to music and so it can throw away any data that's too high frequency or too low you can see that's in inaudible for you to hear it can also assume that the left ear on the right ear are hearing roughly the same song so you just store one of the two ears and then you know the diff for the right ear so you make all these assumptions and it gets good at doing what it's designed to do like compressing songs and the strength of deep learning strength of the AI that most people use today is that it doesn't make any assumptions you could use the same architecture to recognize images to play go to play Atari to regulate you know temperature sensor is in a data center you do anything with it and that's both its strength and its weakness and so to get to systems that are more like our brains more like the mammalian brain if you make stronger assumptions and in our view those assumptions come from physics so and I think of them it's sort of like properties of the universe so like spatial regularity is a good one when you look at my face the pixels that make up my face aren't teleporting at random all over your visual field they stay together and when I move my head from side to side my eyes nose and mouth move with my head and when you move your head like this the room transforms itself in the same way every time and so there's all these statistical regularities or the fact that objects exist and objects tend to have attributes that persist over time like this table is still wood and all these things are things we can take advantage of when we look at how do we create models that match the properties of the systems that we're looking to exploit and this is the kind of work that we've been doing it by curious the first thing that we did and this is five six years ago now is we wanted to build a vision system modeled after mammalian neocortex that had human-like ability to generalize to shape recognition and so the way we tested it was the same way humans test each other as can you perceive like a human we showed it captions and the problem with with tech reading text and reading captures occipital efore like an old brain style AI system is that there's like a zillion different ways that even just one letter can vary and so to get your old brainy AI system to recognize all of them you need to show it a gazillion examples whereas when you read all these CAPTCHAs you've probably never seen many of the styles of words that appear on the screen but miraculously you're able to without any training and that's the the kinds of properties that we were looking to capture in our systems yeah yeah it's true I mean so a lot of the earlier work on how do you break CAPTCHAs tried to exploit that kind of idea but what the design of these kinds of systems exploits that breaks that style of system is that notice how in most of the examples the letters are touching each other and so if you have an image like this where most letters are touching each other or this one is even worse and then you're all you have is that input now you have to guess what were the transformations that were applied in terms of rotations scales you know warps noise it's very difficult to figure out where the gradient is to climb into oh it's getting more and more word like oh I'm getting closer and closer to reading a letter here because you don't know where the boundaries the letters are because they're touching each other so you can't pre segment the letters all of the old attacks and captures involved pre segmentation and so in order to be able to solve these in a general way you really need something I mean maybe there's some way to do it that I don't know about that you know a smart person can't figure out but a lot of people have tried because it turns out there's a lot of money be made in scamming banks and like stealing people's stuff and so you kind of need a system like ours so this is something I think it's curious if that you want to give a shot so we this is a you can read the paper online this is something we published a couple of maybe I don't know eight months ago or something that goes into the details of the specifics to the kind of generator models that we focused on about curious and the the neuroscience inspiration for them and how they relate to some the other work I'm not gonna go into too much depth on that today we also tested on reading street signs against the kind of old brain deep learning methods so this is a case where by having stronger bias about what is a mammalian brain doing we're able to build systems that outperform old brain systems and use a lot less training data and for a pictorial representation on the left is like the Google training dataset which is 8 million images on the right is the vicarious training set which if you're having trouble seeing it's because it's in the dot of the eye and then with that lets us do is than we when you're in a new environment and you see a bunch of random objects on a table you can look at them once and then in our system get 50 degrees of 3d rotation invariance and you can put those objects in industrial environments where robots need to manipulate them and the robots can then cope with those environments and changes in lighting and all kinds of things that are difficult for conventional AI systems we can also apply it to control so this is the the the deep mines deep reinforcement learning system called AC 3 or a 3 C actor critic and us a were arson on the left that we have a paper and ICML that you can download and read about how this works exactly it's a little bit too complex to explain and just this talk but we can do things like if you make the image brighter obviously we can keep right on playing you can do things like move the pedal up so this is analogous to a calibration air and a robot as it turns out robots oftentimes will move their arm to an intended end effector state and they'll be off by a little bit and so you need to be able to compensate for for noise and your motors and joints whereas if you're if you're working with a deep learning based system it's going to have a lot of trouble generalizing to these kinds of variations or this one's sort of interesting so this is what happens to deep mind system at the end of the game and because when you go back and think of that that giant array of fifty million training frames was trained on very few of those frames are frames where there's just one brick left most of them are frames where those lots of bricks left and so now it's in it's in a state where it doesn't have a lot of training data anymore and I I'm bet you or you know it's there probably only zero or one games where it happened to have just this brick left and so you see it's missing the ball a whole lot and then when it does hit the ball it doesn't hit it with any intentionality like it has no plan for how does it hit that last brick if I let this video play we have to wait another 15 minutes before clear the board and because it's just responding in a stimulus response way it's not being able to make these long-term plans it's not able to aim to achieve a goal which when you're in a body is a really important feature and when you have a model of causality and you can say okay this is the brick I want to hit you can reason backwards to where your hand needs to be in order to hit the hit the ball in the direction that's gonna enable you to clear the board it also enables you to cope with obstacles like yeah but they didn't have enough training set that covered oh yeah that they could have but I I think the the then the further case comes in in reality you can't synthesize all the end games because there's an infinite number of positions that objects can be on there's an infinite variety of objects that you may encounter or obstacles that may occur during the course of you know you're being in a body and needing to interact with the door is broken or loose or whatever there's a million things you know not for this case but I think the the aim of deep mind is not to play Atari games it's to build systems that start to capture more intelligent behaviors in the way you and I describe intelligence and so this is another one of those examples where how many different configurations of walls can you come up with and eventually we explode the training say because it's it's 50 million frames just to train the no wall version and then you start adding walls every to position you start changing the size and shape of the board the height of the paddle and now we're in a combinatorial explosion yeah and the problem in a constraint if you reduce the degrees of freedom but unfortunately the real world has lots of degrees of freedom an unfortunate thing yes and you know and this isn't like ours where you have a model of causality you can then you know immediately say oh there's a wall there so I just need to aim to the left in order to continue get reward so it's it's it's by watching the game played it watches what happened so it's looking for causal relationships and notices that when the ball or when this entity makes contact with entities that are gray it reverses direction so it's a thing it's we don't give it any knowledge of physics at all it's just learning causality relationships and then from that I learned schemas that it can then use to pick the future of reason backwards right on the slavish pattern-recognition rather than image filtering that says light things are walls for example or things with certain colors or walls and not mapping this to greyscale so that oh so he doesn't know he doesn't know that great things are walls it knows it all noses there's a whole bunch of entities in the world and that's that that's the knowledge it starts with and we can even eyes none of us thought I had but I didn't include in this particular talk where we actually do change the colors randomly that's something we could come to we didn't actually implement that and that's something we could add later and I think humans actually do write what you have it you have a prior that says oh it looks at we called it an affordance right you have an affordance for oh it looks like a button it's probably a button it looks like a wall it's probably well this is another example where you get to that comment Oriole explosion of okay well what if we make the walls move and change their positions velocities and this one really puts pressure on how well can you actually plan because you have to in order to get any points at all you really have to know where everything's going and aim very exposed and give you a visualization of what our systems actually doing so the screen is gonna turn white while the screen is white the game is paused and it's just unrolling all possible futures and so you can see it's it's not the learn dynamics that it's come up with by observing this game and then it picks the where it gets reward which is right down the middle and then it models again so it doesn't move here at all because it is a need to so we can also apply that same codebase to you know solving complicated puzzle games like this where it's it's not about reaction it's about reasoning through a long series long causal chain in order to arrive at a destination where if you make missteps along the way you can get into an unsolvable state does it have every possible does it explore every possible no it's so this is one I can send you the paper if you like that kind of goes into all the details about it so what's it's set up as a probably graphical model and we do belief propagation in it and anyway let's get on to the the Rope so the rhotic stuff this is um so this year we've been putting all of these systems together inside actual robot bodies and having them do the kinds of tasks that children do and adults eventually do to get to this future where robots are increasingly common when we talk about implications of like what happens you know in the next iteration of AI the kinds of stuff that we're building it vicarious and that other people are building elsewhere at other institutions I think that you get a pretty clear picture of where we're going with just one chart this is the index price of labor versus the index price of a robot since 1990 and the robot side is actually this chart was made a little bit prematurely because the robot side has dropped off precipitously in the last two years actually and we've just reached the point where a small robot is less expensive than the annual salary of a Chinese factory worker and so robots are getting very affordable we're in this in the middle of the sort of perfect storm of all these different technologies becoming available like your iPhone X comes with an RGB D camera in it that's a very power efficient and pretty accurate and fast and that's another great tool to have in your button in your toolbox when you're when you're building something that needs to see the world and have depth perception so and this I've already sort of answered this question but if if like if labor is getting incredibly expensive and robots are incredibly affordable why doesn't everyone use them not to pick fun of one of my investors but traditional automation is very rigid like to program a robot to do something today you give it a list of points in 3d space that it's supposed to move it's end-effector through and if you want to tell me how to make a sandwich as a list of 3d coordinates to hold my hands in it's gonna take a month for you to you know convey to me how to do it and then if someone like moves the bread a little bit to the left during the course of me making the sandwich I'm gonna be putting peanut butter on the table so it's a very very brittle way of accomplishing tasks and it especially if you want to do something inside of a car like Elon is doing where you're you're trying to harness wires through the inside of a of a chasis and those wires are bendable and so you need a closed-loop perception perception and visual reasoning system in order to know where to move your hands and I think there's actually this really interesting parallel between robotics right now and computers in 1950 so in 1950 if you wanted a computer to do something for you you basically needed a completely bespoke system that a engineering firm built for you that only did the one thing that you Specht out in the RFP and robots are the exact same situation today if you want a robot to do something you hire a custom engineering firm you read an RFP you get bids and they spend months or a year building a system to do that for you and I think robotics in the future is a lot like computers in the present day where they're just you know cheap ubiquitous general-purpose and we don't think about them anymore in the same way that we don't really think about computers anymore and I think of another what lens I like on this is right now compute has reached the point where it's like electric the price of electricity plus some epsilon and I think in the future as these trends continue we'll get to a world where labor is the price of electricity plus some epsilon which has some implications that you might be wondering about with respect to labor over labor markets if you ask the internet like what's gonna happen in the world where these types of AI systems robot systems exist the internet will tell you that it's the end of the world and that we're all screwed and there aren't any jobs anymore and I actually think there's like a very underplayed counter narrative that I think is worth talking about so you when you're on the internet you may read articles that say stuff like this where you know we're being afflicted with technological unemployment where people can't get jobs anymore anyone guess when it was written wellmaybe undershooting a little bit this is 1930 yeah 1930 yeah so this is not a new problem like the problem of having machines do stuff is not a new problem and I I think one of the things that gives me comfort about the coming job apocalypse is if you look at the last 3,000 years of human history it's it's 3000 years of people building contraptions that take a job that used to take five people a day to do and turn into a job it takes one person a couple of hours to do and that's everything from the printing press to the wheel to the telephone and after 3,000 years of doing that we have more people and we have more jobs and we have a better standard of living than we've ever had and so to me this standard of proof required to believe that this time is different is very very high and I think the best model we have a mapping we have is 1900 Zagreb for revolution where you we went from 40 percent of the u.s. working age population work in a farm to 2 percent and despite that enormous shift in where labour was deployed it wasn't we have 38 percent unemployment it was you know every decade since 1890 basically the labor participation rate has gone up and I think that you know another great example is is you look at ATM machines when ATM machines were introduced people thought it was gonna be the end of bank tellers no Bank was going to play bank tellers anymore and all of those people be out of work and in reality we have more bank tellers now than we've ever had what ATM machines did is they made the unprofitable parts of banking go away taking deposits and giving out withdrawals and then freed up the bank tellers spent all their time selling very profitable things like home loans and credit cards and so on yeah yeah that's true so in some cases technology disruptions do so what if I have the chart here I do okay so this is taxi drivers and ride-sharing so there'd be another example so I agree in some cases there is job destruction and like no one operates a telephone anymore but I would also argue that the result of that no one goes back no one today goes back and says man remember the apocalypse that came when all the telephone operators were unemployed remember how the world fell apart no and says that because the world didn't fall apart like everyone got new jobs and it was fine I there's large communities in New Jersey depressed because of the change it just an empty lived in an area where agriculture has been supported not by growing but by the government pain not to grow and then when government payments of the drawing no see everybody in call centers yeah I grew up in Lancaster Pennsylvania yeah yeah in farm country so III I don't have a myopic view that it's gonna be better for everyone and and the future is full of magic unicorns I think that there is going to be labor displacement but the argument I'm making here is that one that I don't hear from the headlines which is the headlines make the make the cases that we're all screwed and it's going to be like an employment wasteland and I think the what's closer to the truth is that there's gonna be new different kinds of jobs some things will be destructive other things will be creative and the effects aren't always predictable and so this is a classic case in point real like okay the taxi industry is doomed because the ride-sharing and in reality consumption of rides went up massively such that perhaps there is a decline in the taxi business but it's nowhere near the scale that one might have predicted when you look at a world where anyone can someone a ride with their cell phone that isn't connected the taxi services yeah perhaps the one for ancient structures be starting from zero it's indexed right so it's talking about growth and spend so this is what this what this chart is showing is that the spend so what you would expect right if your if the argument is true that people stop taking taxis you'd expect this line the red line to to go to zero right and instead it stays at about a hundred which means that the spend on taxis remains relatively constant in the face of this new product where the spend goes up twelve hundred percent or twelve hundred points on this index scale right you've done a great job opening up a whole series of Pandora's box it is true that for the technological revolutions that have happened yeah up until about ten or fifteen years ago we it it wasn't consistent that new jobs were created in the place the old jobs were eliminated yeah but there was there was a net increase in jobs to to take over for the jobs that were lost as a result of the development of technology sure is and I think you're absolutely right it's unpredictable both ways yeah I think I'm are you against this so the wall of headlines that say it is the apocalypse we are screwed that we can end up losing jobs and it not be the apocalypse but so you can take this out but there's we don't know which way it's gonna go I completely or we definitely don't know ways go and that's exactly the point I'm trying to make so I'm trying to make it so that the point that I hear in the media generally speaking is we know which way it's going to go and it's going to be the apocalypse so that's what I see and if you'll see too if you just Google for like AI and jobs is it's going to the apocalypse so I'm I'm trying to introduce a counter-narrative that says we don't know which way it's gonna go and here's another path down the road that we may end up on that you know is is TBD but here's some examples in the past where it has gone the other way yeah these machines will be used for intelligence augmentation rather than labor replacement right yeah and you know maybe they'll keep us as their pets the economic argument for how it's why it's going gonna play and that's the sort of the you know how the kind of economic prosperity has been allocated over the last forty years you've seen Jeff greater growth in profits and Hana lower growth in payment for labor unions go to me with that says this defeat the revolution in this doesn't necessarily yeah let me just open a whole other ken doors box here so I so I I tend to think of this as a as a Miss misfire of the design of corporate personhood or of a corporate entity so I think right now corporations are legally required to maximize shareholder wealth and I think this is not know in the state of Delaware there at least now so can you tell me where you were did you you can say otherwise and your corporate bylaws and your rules you could make it public yeah and trade your sportingly you are not obligated that's my process are you talking about flexible purpose corporations and B corporations or are you talking about regular C corpse because there is case law on Delaware against Craigslist actually I believe that they were sued for not taking actions to maximize shareholder returns and they lost so the the the headway of are the the segue of this is that sorry in California there's a special thing called a social purpose corporation which yeah I mean I think it's a citation needed thing can you point up a lot those corporations are required to do that it's possible there is such a long yeah don't worry companies but I mean certain companies even you know well-known for-profit companies like Apple say look you know you're just for the show well yeah because apply elsewhere get out of stock you know what we do is we build computers and without companies if you think about what they do there they're found for purpose like FedEx delivers packages yeah bills iPhones like only financial companies are defined are usually found it - like maximize the share so I definitely it a citation on that yeah sorry plenty of case why is there a lawyer in the room has anyone ever said it's one the US that says in general without a special purpose corporation with with provisions for the contrary that by default the corporation has to do share of responsibility to maximize yeah the doesn't have to be in the short term yeah right you plenty you know people used to joke yeah and their shareholders have done very well to to maximize that that value over the long term yeah right it doesn't necessarily mean making short sigh decisions but you're right that at some level that's the purpose of a standard corporation so I have a citation here what is your name the point I was making is that I think that your your count your your point about how corporations generally speaking make decisions that are better for the shareholders and perhaps worse for society I view the purpose of corporations as things that exist for the betterment of society not necessarily for the better minutes of the shareholders which are sub so and like it ends when you have corporations that exist only for the better Michelle shareholders you end up with them making decisions to externalize things like like pollution and streams or hiding the fact that cigarettes cause lung cancer and so these are choices that are I think there there are pressures to do from a shareholder perspective that are counter to what we would think of as being ethical and I think this fits into that circle and this is this is why I said I'm opening other Pandora's box which probably I should have not no not to walk into but yeah there it is degrees of freedom so it doesn't really matter what the laws are we develop a system that it's going to force us in a particular direction it's going to force us in that direction the laws of change and the behavior will change the question is what are the vectors that looking forces to move yeah yeah to the pair's the countervailing vector in Switzerland a year ago they had an election and it was went to a general vote the question was whether every citizen should be guaranteed an income and it lost like 49 to 51 percent was really good at resource allocation mm-hm and up until now human labor has been scarce but if it turns out that there's a lot of excess human labor then one outcome is that everybody gets an income for being formed and that you have to pay yeah all right so I think that's a really interesting possible future and I also I mean the other side of it for me is and this kind of goes into the next slide isn't the last month talking um is I don't in my view I don't think labor is going to be scarce like I think there's so much that humans can do to make our society better than it is right now and we don't live in Jetsons world and we're very far from there because it's so expensive to build things and there's a there's a shortage like in my previous slide seven million unfilled welding positions the united states line this is a random subset of jobs with like there's a lot of unfilled jobs and that's suddenly unfilled welding positions without large-scale infrastructure projects to like fix parts of our nation's infrastructure that are are in decline is everything bullying new stuff and so i think there is a lot of opportunities for labor to be done and there are areas where it's gonna be really hard to have a robot it is general-purpose enough to be able to do the motor actions required to navigate construction sites and perform those tasks so i think there are lots of opportunities for labor so yes you may be right but the other side would argue that if i can 3d print it what is it man yeah so new technologies may emerge and all kinds interesting stuff can happen so i I guess the and this comes back to so what what does this mean what opportunities might this create what risks are there we cover some of the risk in my view there's so many different enabling technologies that come together in a world where not not only is prediction cheap in the big data internet since the word but labor gets cheaper because the hardware in the software and the sensor isn't the chips necessary to make labor cheap get more abundant and so I think that that's the the final takeaway I would say is like what is where's a where is there a rising tide in this particular area I would say there izing tide is around the different enabling technologies that come together to make autonomy more affordable more Universal and and to think critically about if you're looking at this from a business or a cylon engineering perspective where we're using labor now where would we use labor if it were dramatically less expensive and how would that change the way this aspect of business respects society tends to function so that's all the thoughts I for you today happy to take questions thanks for attention yeah thank you for talkin the answers a question with your basic premise of reptilian brain or brain brain the fact is mammalian brain a toddler for example to be able to open a door the toddler has had to have been exposed to many many millions of images it's trend that's common in old mammals and learning is essential absolutely well this is absolutely not true with insects for example it washed out of the cocoon the minute gets out of the cocoon is able to hunt for prey late sag you know their sting the prey without no learning whatsoever yeah so you see some of that too like horses when they're born can immediately walk which is something that takes humans years to to be able to develop or maybe it depends on how the child comes along but maybe six months so yeah there's definitely a mix of learned behaviors in training data is really important part of it it from the calculations I've done anyway if you make some reasonable assumptions about you know the the kid is colorblind and happens to only have monocular vision and it's got some deficiencies so the visions a little blurry so if you make some assumptions like that and calculate the amount of training data that a three-year-old gets during that period it's in the order of terabytes or maybe tens of terabytes so it's not an unfathomably large amount of training data for today's computers to consume is very difficult to make these things because there's a process of biological process called myelination the pestis the nervous system of a usual human child does not work right until the nerves are coated with fat it's called myelination it doesn't work so it doesn't matter how many times you've been closed to whatever picture or whatever if that part of your nervous system is not functioning at be able to capture that information that doesn't matter yeah and until that happens and when that happens typically blue with human children it doesn't take very many repetitions and maybe none at all so it's very complicated to try to do it that way yeah I guess the calculations from for my perspectives even assuming that we we've had myelination happen at birth even assuming that it's only on the order of terabytes or tens of terabytes a day which is not a ton of up to 10 or markedly smarter so I should have actually included a section on birds this is sometimes people bring this up sometimes they don't but birds have a very very cortex they basically have an analogous cortex like structure in their brains that does the same functions as our cortex and it's a very similar structure so I would say that that the computations that the birds are doing the computers we're doing are if they're not the same computation they're very very close the the cephalopod case I have no idea how an oculus works octopuses i think are fascinating i wish i know until recently they were really listened to be incapable of smart and social behavior nothing a lot has to do with our inability to observe and have noticed intelligent behavior in the lower animals and so i would be careful to draw conclusions on the you know role of the cortex into making a machine able to so problem generalize and because fish for example you know my experience has shown you know amazing Ashland actually we don't know we don't know very much and it might be just that we just don't know that interesting we haven't observed intentions because we're not able to actually perhaps we don't even know what to look for yeah detective I would love to so I would love if you could maybe give me some some more concrete after the talk give me some more concrete examples of this stuff because I it's it's I think it's really incredibly fascinating and it's something I want to know more about because I think there's some lessons we could draw from there to the examples that I have in my talk and there's some there lots of others we can come up with but it's basically trying to show that intelligent looking behaviors are actually simple heuristics behind the scenes and I agree it's incredibly difficult unless you do the kinds of experiments that are the ones that I talked about where you like have the iPhone in front of the Frog or you have the the the the the tape recorder inside of the wolf taxidermy it's hard to reveal is the animal smart or is it just following a simple pattern and it's a really interesting area of research and one that I wish I knew I wish I had even more examples from so let's talk about that after this yeah I mean we this I would say that's a whole separate thing the behaviors that I think are most interesting for this era are this coming era of AI like a are getting in sort of the era after this one the era I'm talking about is the ones we're like the robots can like make a sandwich in a kitchen you know it's an arbitrary kitchen and so they like know you know how to take the forks out and like move the peanut butter around and stuff and that's those are tasks where you don't need a justification you don't need conceptual knowledge and the kinds of explanations people give that are sort of ego defending and like it's you know there's a whole layer of nuance there that is even far beyond the kinds of stuff that I'm talking about but I mean hopefully someday the machines will give us long-winded explanations of why they did something I'm still hung up on a slide about five slides back showing the normalized expenditure on vaccines versus rideshare services can you back up yeah well let's see if we can figure it out together the thing that I'm hopeful that it says and maybe doesn't say this but the thing I'm hopeful it says is that after the introduction of uber spend on taxis wasn't significantly affected the fact that that's normalized to 7 2015 means nothing what if you had normalized that with 100 as mid-2016 or as January 2017 you would have had along zero that this started rising I mean if I normalize it here you know yeah I wish they were both in dollars and that's something I have to do but I guess that the question I'm asking is does it matter so like if we normalize here or like here chart is showing is it not so like this is saying that it's about you know here's 100 and it's still about a hundred at both sides so does it matter where we started the normalization like if there was a ten fold the clients and it spend on taxis this black line would be heading towards the mob saying the dollar spent on the Green Line is still less than the dollars spent on the taxi line the incursion of the right search service wouldn't be significant enough to cause movement change since the time scale showed inappropriately so yeah we have a data point from this case lawyer Cohen Trump's fixer so which would suggest that this is wrong in some way you know it suggests that the future value of the taxi shields the future value of the taxi in your city has fallen based on what people expect yeah they also could be wrong yeah right but based on five suicides of taxi NSW is moose s is five more or less than the average like how does this compare the dentist suicides in New York City okay but this is New South Wales correct in Australia and yeah is it yeah million so New South Wales is a very particular environment it has a very low population density and it has a particular employment pattern yeah in a particular economic space which is radically different from New York City I mean you could not find to differ to more dissimilar places and something else in Sydney New South Wales is the state in which Melbourne is no Sydney is a minister well so I think what I'm hearing is I should probably take this slide out of my nose it's not so this slide is not particularly important to the points to make overall cuz then you can tell the truth because you

Channel: stanfordonline

Views: 35,589

Rating: 4.8576269 out of 5

Keywords: Stanford, Stanford University, Seminar, ee380, Scott Phoenix, Vicarious, future AI systems, Artificial Intelligence, deep learning, Future Paradigms, AI Implications

Id: wJqf17bZQsY

Length: 74min 42sec (4482 seconds)

Published: Thu May 31 2018