Brains Vs Machines: The Future of Work

I have Here Blake Richards a computer scientist and neuroscientist who studied under the Godfather of AI Jeffrey Hinton we're going to talk about Hinton where you agree and disagree his influence on you uh we're also going to talk about why scientists who build and program AI can't figure out what it's doing the interpretability problem or explainability we're going to talk about AI safety and Alignment conscious AI the future of work lots of other things but first you say um we might be able to accomplish more with AI by not designing it to mirror human brains can you talk about that right to go from the computer science to the biology it was very overwhelming at first there's so much in the biology and biology is so complicated I learned though the more and the more I tried to incorporate biological facts into my models is that generally speaking they made things more difficult it was it was usually the case that like you'd have a neural network that worked that did stuff and then you try to incorporate some extra biological realism into it and all it did was make it work less well do you think it might be confounding now but there might be ultimately an important reason buried in the biology about why it makes it work less well um yeah I think that there's there is a very important reason and that reason is that if you think about all of the facts that exist in biology some subset of them are probably there because they helped our ability to learn and behave and perform in the environment but a probably larger set of them are there because they were necessary for minimizing energy usage they were necessary for the way that proteins get phosphorylated they were necessary for you know genetic reasons they were just holdovers from our phylogenetic ancestry if you're just sort of randomly adding biological details to your model nine times out of ten they're going to be things that have nothing to do with the algorithmic capacity of the model and instead everything to do with these other extraneous factors that don't daughter for a neural network so they're like vestigial or people yeah or other things that matter for the biology but not for a neural network right a neural network doesn't have to worry about uh what it's going to do with the byproducts of the Krebs cycle for example right so like it doesn't have to have these additional mechanisms in place for dealing with that the exercise ends up becoming can you figure out which biological details actually matter algorithmically versus not it's not the case that biology hasn't helped at all I am not someone who makes that argument there have been very specific instances where the biology has helped us and indeed neural networks themselves were brought about by people thinking about how to model the brain but there's sometimes a tendency for people to think well if it's in the biology then it's helpful and that has in my experience definitely not been the case so the AI interpretability problem or explainability is this problem where even the scientists are built and program these things don't understand and can't predict AI decisions why is it so hard to figure out what these things are doing or to predict it so the reason is that these systems are engineered using techniques for optimization rather than engineered for specific purposes if you build a car every piece of the car has been designed to achieve a certain function but in what we call neural networks we give it an objective function or a loss function so we say here's what we want you to get good at so in the case of generative pre-trained Transformer models so they just say like I'm going to give you a sequence of words I want you to predict the next word how you're going to do that I'm not going to tell you we're going to give you this objective function of predicting the next word and we're going to allow gradient descent which is the algorithm that we use for training these networks to find the right setting of the network that successfully lets you do that task and as a result when you go and you look in the system there aren't little bits that are doing things that you've designed them to do because you didn't design any part of the system except the overall system you just had said this is overall what I want you to do so it's got all these different little Parts Each of which is doing something that you never designed it to do necessarily and it's hard to read those parts are there too many of the parts is it just volumes of code that you can't look what does that look like in terms of challenging yeah so it's it's exactly that it's hard to interpret those parts so if you think of there being all these little neurons that you're simulating really they're just numbers they're vectors that you're pushing around your computer but whatever the problem is that it's very rare that you could look at the way that that neuron activates and say ah this neuron activates when the system is going to do X or when the system sees this word or when the system is planning a sentence on birds or something like that it never happens in a nice concrete way like that instead it's very complex whatever it's doing it'll activate in response to the word bird in this one context but not in this other context and then in this different context it'll seem to be representing whether or not the system is going to say something snarky or whatever and it'll be highly contextually dependent highly unstructured and even though we know the mathematical function that leads to that so you can fully describe the mathematical function there is no set of human language words that we can give to the unit to say this is what it does so it's not just a matter of too much data that we can't possibly like comb through it all and piece it together is that we we can't anticipate what it's going to do that's right yeah exactly exactly and oh problem because there's no there's nothing you can throw at it then you can't throw Manpower at it no there's not it's not just a case of like oh if you threw enough manpower at it you can figure out how these systems are going to behave when you give AI a goal does it always just like hyper focus on executing it yeah necessarily so um this is the reason that it can be potentially dangerous some people take it too far you know Nick bostrom's paper clip Factory argument is like a sort of fun taking it to the Absurd extreme but it's not totally crazy what we do see with these neural networks is that the only thing we're actively designing them with is this loss function or objective function that we give them so as far as the math that we're using is concerned it's just going to do what it can to minimize that loss function or maximize that objective function and so if you don't have a well-specified objective function it's entirely possible that it's going to come up with bizarre techniques for doing it right things that might end up being not what you want since you straddle computer science and Neuroscience what's at least like one key difference between human goals and machine goals well I'm not sure there's a big difference this way I think humans suffer from all the same oils here we see this in people all the time people will get hyper focused on specific goals that they have to the exclusion of a lot of other things that they maybe should be considering and they will act in potentially terrible ways if it furthers them achieving certain goals per your other question about interpretability humans aren't interpretable when we drop electrodes in humans brains we have no idea what's going on it's a mess it's really hard to interpret what the different neurons are doing so I think we're very similar I think we're these huge systems that have been optimized by a combination of life experience and evolution we have certain objectives that we evolved to have and we will often try to achieve those objectives at the detriment of other potentially important things so if our goals are the same what about other functions like do you agree that large language models at least at this point are synthesis machines it's not unreasonable to call them text synthesis machines I think that's perfectly appropriate what's remarkable is that when you actually give the system this task of synthesizing text that is indistinguishable from what humans generate it necessitates it actually developing some understanding of the world as human language reflects it there's a wonderful example I really like from Blaise ahead of Google brain in Seattle he spent a lot of time interacting with Lambda their previous large language model he said I dropped my bowling ball on my violin and the language model replied oh that's unfortunate I guess it broke badly and he says which the bowling ball or the violin says well the violin because it's fragile now that would seem to imply that the system has an understanding of the difference in fragility between a bowling ball and a violin and I think the answer is it does because to actually predict all of the sentences that it had seen throughout the course of its training data when it encountered sentences that involve bowling balls or violins sometimes those sentences you could only predict the next word if you understood that a violin was breakable and a bowling ball was not so it's a it's a funny thing they are text synthesis machines text generation machines but that task of text generation requires a remarkable level of understanding of the world and so are we right sort of yeah exactly I would say actually that there's a lot of evidence in Neuroscience that we are not radically different in terms of how we operate it's just that we're not only predicting text we're predicting our full sensory motor experience so we are predicting all the things we see all the things we feel all the things we hear that's what our brain is constantly doing every second of the day and this is why we have an understanding of the world is courtesy of the fact that to predict what's coming up you have to understand the structure of the world so the next generation are multimodal models are not just trained on text they're trained on audio and video do you think that's going to change the game oh yeah we'll see massive advances in terms of their understanding of the world is my guess and a much more human-like understanding of the world than the large language models this gets back to your question about do they do like anything other than just try to achieve the goal as we very specifically said for them the answer is no would they just try to do that so the large language models will only develop an understanding of whatever is required to successfully predict the next word but there's surely many ineffable things in our existence that aren't captured by our linguistic Communications that these models therefore don't understand but which a model trained on video data very well might understand more of and is it true that people are working on trying to integrate the sort of brains of AI models trained on language and video data into carbon Soft Robotics so putting AI multimodal intelligence into robot bodies um so people are working on this but it's actually a remarkable engineering challenge one of the reasons that this stuff first took off with large language models is because it's so easy to work with linguistic data there's such large data sets of it the literally just the RAM and memory requirements of operating with text are so much less than those with operating with full video and once you tie in robotics then it becomes really hard to collect enough data of the right sort potentially there are maybe ways around this using simulation and other stuff like that it's a much greater challenge them training on text something just occurred to me you know the hard problem of Consciousness is that you can see microscopically neurons firing in the brain uh when a person sees the color red but the the microscopic neural activity is a fundamentally different thing than the experience the person is having of actually seeing the color red and that the experience is called qualia it's the qualitativeness of what it feels like to see red but science has never been able to like objectively like point to it and be like ah there it is that this this thing is red this is the redness of qualia um so hallucinations sensory deprivation can cause hallucinations in humans I just wonder if AI hallucinations might be more closely related to human hallucinations than we're considering like maybe there's some sort of AI language based detached from the world qualia to AI hallucinations well yeah I think it's a it's an interesting question this gets back though to the problem of it maybe not actually being ever clear what's going on in these models and we have to accept that for example with the question of hallucinations like I think there's a tendency for some researchers to try to downplay what these models accomplish I think that tendency comes from a mixture of exhaustion with some of the over-the-top hype coupled with some less Fair kind of just like bias against any success and I think that there is a desire therefore to downplay some of the accomplishments uh that these models have made and and in part to then also say things that in my opinion are not justifiable so even though I agree these models are clearly text synthesis machines I would also say they hold some understanding of the world and they are intelligent I have no problem saying that by any reasonable use of the words understanding and intelligence it applies to these models no not the same way we are they don't understand the world the same way we do they don't have the same type of intelligence we have but I think they're clearly intelligent systems that understand the world that being said this whole question of like well we can go and we can analyze their behavior and see ways in which they behave oddly and hallucinations quote unquote is one of the examples that is given but I think the hallucinations example is really interesting and you raised an interesting point there do humans not do that I think they do like even if we don't want to subscribe that to qualia you can just see that human beings generate false things all the time like this is not an unusual thing in human beings if anything we have to train ourselves not to do that we spend I don't know if you have kids but like you know I spend my time with my kids in part you train your kids not to generate confabulations and instead to always stick to the truth and that's a commitment that you have to train them on so I don't think that where we differ from these models is in our tendency to engage in these hallucinations or confabulations or anything my guess with respect to something like qualia though and this is what I was getting at is that we can't say whether these systems have qualia I think that's an impossible question to answer for them all we can say is that they surely don't have quality like us because our polio is necessarily tied to all the things that that we experience with our bodies the red that we see I don't think can be separated from the way that our eyes work and the mixture of receptors photoreceptors we have in our retina so does you know Chad gbt see red does it experience red no I don't think so there's there's no way it does it's not like we do anyway if it experiences anything I don't know I think it's an unanswerable question my tendency is to think it doesn't because I think probably we can say that it has missing from it all of these raw phenomenological things that that embed us within the world that that make our quality of what they are but could or you know robot have these things sure maybe and and the more the robots like us the more that we would probably be inclined to say that it might have quality like ours so I just want to clearly capture your thought on whether AI is now at some level or could become unconscious this is an area where Jeff had a big influence on me but also uh the philosopher Ludwig wickenstein the the hard truth that most people are not going to be able to fully accept is that the dividing line between that which is conscious and that which is unconscious is never going to be clear or scientifically articulable it's it's akin to the dividing line between that which is alive and that which is not alive we can't actually draw a scientific line concretely we can't say whether a virus is alive or not as an example that's not to say that there isn't such a thing as life some things are alive and some things are not alive our ability to say this is conscious and this is not depends on there being a stark difference something that is like us we can say is conscious something that is clearly not like us we can say for sure it's not conscious but something that exists in that interstitial Zone like a virus is gonna be really hard and I think that's where AI is going to end up sitting so we're going to have to make all these moral and legal decisions about AI without ever knowing for sure that's exactly it I think we're gonna have to make moral legal decisions that will not be able to be guided by like yes this system is conscious know this system is not conscious question I haven't asked you yet that I should be asking you one of the most interesting questions is where a lot of the data is going to come from for doing alignment so like the reinforcement learning through human feedback has been key to making chat GPT not be an absolute toxic mess moving forward where exactly all that data is going to come from is a real issue you know my understanding is that openai paid a bunch of Kenyan dudes to interact with it to do that for proper alignment it's not clear to me that just subcontracting out to the lowest bidder is the right way to do it so really thinking hard about our data sources and and regulating data sources as well is going to be part of the picture that needs to happen here I think you're referring to the Time magazine article it was a bit of a harrowing read where a Kenyan Workforce earning somewhere in the range of two dollars an hour I had to label dark web content to make AI less toxic it's like it's it's too horrific to say out loud and I can't imagine the impact on the people labeling this stuff um let me think of how to phrase this as we're building these intelligent things whether or not they're conscious whether or not they're alien intelligence if they're being exposed to the dark web as they're being born do you think that's going to shape them well we'll definitely shape them um so it's not that they should be uh as it were sheltered from Bad data quite the contrary we need to figure out ways of training them to understand to recognize what is that data which we do already to some extent okay but aside from making sure it doesn't just regurgitate toxic text do you think it's going to change how it's I don't know if you can use the word thinking but how it's yeah awesome okay well so how it how it's thinking about optimizing these problems when it's been exposed to all this and we don't have it aligned yet do you think that's going to pose any problems yeah I think that will Pro pose problems potentially and I think there's interesting questions then that we'll have to ask about and this gets to the point of like how we select data and how we design curricula for it again I think to Children right like it's not the case that what we don't do with kids is show them everything and then post talk say okay this is not so good this is good this is not success rather like as they develop the understanding of the world then you start to introduce like some of the darker stuff and help them to understand it so there might be ways to better structure it than just train out everything and then post talk alter your value functions so you're making me think I've been thinking about this backwards because when I've been talking to researchers I've been thinking we sort of have to get an understanding of what's going on inside AI black boxes before we tackle alignment but it looks like we might never solve the interpretability problem especially if there's some supplements of Consciousness in there and and frankly we don't have interpretability with humans and we align or try to align Society with our goals and values without ever understanding what's going on inside us like what's your take on the order of alignment and interpretability yeah it's a good question yeah my take is definitely what you just articulated I don't think that alignment should depend on interpretability I think interpretability research is cool I think it's fun in the same way that Neuroscience is fun we would like to understand how these systems work and we do a little bit of interpretability work in my lab and I see lots of potential in that area moving into the future do I think it's necessary for alignment no not at all I don't get why anyone ties those things together because as you say we do alignment with other human beings all the time with zero interpretability the key to alignment is just ensuring that we train the models on the things we care about and and that's where her what Jeff has been saying what yahshua has been saying what others have been saying we do need some real regulations in this sphere because what we need is it to be that you can't train up and release a model if you have not done the due diligence of ensuring that you've considered really all of the objective functions that this system needs to be given to be deployed safely in the areas that's going to be deployed so it can't be the wild west anymore there has to be as there are with drugs with food whatever you know very clear regulations that say all right before you can release this into the wild you've got to prove that you've met this that and the other conditions and that your objective functions have been well aligned to the possible failure modes and even still things will slip through and cause damage just as they do with drugs you know the FDA is not perfect in its approval process but we certainly it does better than if there was no FDA and people were just pumping out drugs willy-nilly and giving them the populace right do you think ai's fatal flaw is optimization um I think the the honest truth is that uh that is the Fatal flaw of all human endeavor so like you know I think of this with respect to capitalism versus socialism as well like to some extent capitalism is the economic approach of using objective functions right rather than having an economy where someone comes in and centrally plans everything and says okay these people are going to do this these people are going to do this these people are going to do this we're all going to work towards the collective good um we instead just set up the objective function of make money and then you let things emerge as they do and the reason we do that and the reason we don't engage in socialism is because as with AI it turns out that that approach of like saying okay this piece does this and this piece does this and this piece does this is not really manageable for anything but simple machines if you're trying to structure a really complex system that approach rapidly becomes untenable so you need to take this optimization approach to do anything complex um but as we've seen with capitalism when you just maximize that objective function you're going to get all sorts of weird other things that emerge that you don't necessarily want whether it's ecological damage or massive inequality Etc I don't think anyone has ever discovered a way to avoid this necessary um trade-off regulation like isn't it yes I agree so quite regulation is the the only way we have right now within capitalism to avoid that trade-off and that's the kind of thing we need in AI as well so you need systems in place not only to regulate the release of AI but even you can think of Regulation within the AI itself it says like okay this is your objective but there are certain things that you're not allowed to do certain places you're not allowed to go with this solution space well that sounds almost manageable although have you heard the new buzzword Malik no what's this molec it's um it's not really a new word but it's sort of newly Buzzy um it's about how we want to attribute all the evils of us like hurtling towards the cliff edge of catastrophic climate change and nuclear proliferation and now ai the AI arms race um to like evil heads of Corporations and governments but really it's just misaligned incentives this Cascade mislined incentives which is like if you're beholden to shareholders and maximizing profits and you're going to do things that put your very existence and the existence of your children and your species in like existential crisis no no this sounds like exactly it's that misaligned incentives that's it yeah so that's the danger with AI too so Jeffrey Hinton cognitive psychologist computer scientist and Godfather of AI he won the Turing award for deep learning and this is the award considered the Nobel Prize of computing how did studying under him influence you had a massive impact on me I mean Jeff is uh the kind of person who has a big impact on people I use the phrase uh unironically that I totally drank his Kool-Aid I had found my degree in artificial intelligence really frustrating actually at that time it was largely class after class of things that didn't work I was feeling really unimpressed with with artificial intelligence and it felt like it wasn't going anywhere but Jeff's class was the first time I felt really excited by the ideas I was learning I fully got on board with the idea that neural networks were the way to go not only for artificial intelligence but also for understanding the brain so so what year was this then uh so I took Jeff's neural networks course in 2 2003 and then I was uh R.A in his lab in 2004 to 2005. so this was right before some of his ideas uh about deep belief Nets started to make waves in fact I was working with the model that they released that helped to kind of Usher in the modern deep learning era the the one on stacked uh restricted bulletin machines to create deep belief networks can you define deep belief networks so a belief network is a network where you've got each node representing a stochastic variable that can be on or off and it has a probability of being on that's sort of your belief as to whether or not this thing is true so you can make a prediction by activating this unit with a different probability and um a deep belief network is a network where you've got multiple layers of these variables that can be on or off stacked one on top of each other given that Hinton was such a big influence on you what did you think about him sort of breaking the cone of silence and coming out as a whistleblower about the existential threat of AI the day after quitting Google um I I'll fundamentally I think Jeff's doing it for the right reasons uh he's ultimately a very ethical guy you know I don't know if you know this but like one of the reasons he came to the University of Toronto was that he was uncomfortable with all the DARPA funding that was pouring through uh the universities he was out in the states he's a lifelong socialist and he's a very ethical guy so I think he's coming from a very honest place with this I think he's also right that there are some real potential dangers from AI systems I don't agree with everything he said in every interview it's not that he's saying anything wrong but I think he's maybe over emphasizing some things like for me I'm much less worried about the dangers of a super intelligence than I am about the broader dangers of breakdown of our social cohesion and and democracy that could come about as a result of these things but Jeff's also talking about that so to be clear this is why I don't disagree with him but some of the concerns that are being raised about super intelligent systems they're not crazy but I I see them as still less concerning than these more concrete things that that I am much more directly concerned about so you're an independent researcher have you run into the problems with AI experiments and impartiality because you have to trust big tech companies who are by definition conflicted and might distort unfavorable outputs the specific scenario you just described is less a problem for us because we build our own models what is the case is that we're in a weird place in Academia right now where a lot of the most interesting models are not accessible to us and we don't have the compute Firepower to make versions of them for ourselves as it stands there was a petition that I signed called Leon l-a-i-o-n which called on governments to invest in something like a publicly funded large compute resource for independent researchers sort of akin to the Large Hadron Collider but for AI where researchers could build larger models using these um communal compute resources I would love to see something like that exist because as it stands we necessarily work a smaller scale than the industry guys do and the industry scale is important for achieving the state of the art you need scale to get there did you see the Google memo that was either intentionally or accidentally leaked saying that open source AI not open AI but open source actual open source AI models are going to bury big Tech like Google and open AI slash Microsoft do you think open source is an opportunity for independent researchers definitely definitely that's part of what that petition I mentioned called for is the creation of not just communal resources but then that the results of the those communal resources would be open source because I think that's the way to prevent this from all being controlled by small groups within industry the thing that's not quite true yet is that as it stands no one yet has built open source models at scale uh but I think they're probably right that that's gonna happen and I hope it does and if it does independent researchers will be back in the game what do you see as the role of Academia in the AI era the whole idea was that the marks you get in undergrad are supposed to be some reflection of your skills and that employers or whatever Downstream could use that as a signal about how intelligent you were these systems necessarily break that model but maybe this model needed to be broken because it's never been that clear that that was the best signal anyway it might just be liberating to not have to be trying to learn stuff that you don't actually want to learn and which in my opinion if you don't want to learn it you're going to forget it once you leave the class anyway I mean speaking about training data I've been curious about this because I've been fact-checking sacred tomes in the humanities and many aren't passing a basic fact check and I I know these models are training on this stuff and this is considered the Pinnacle of quality training data so I really wonder if there's a future where where academics and students can fact check texts across liberal arts the other question for Academia though is then the role of academic institutions visit is AI research and so this is where I think we will get back to AI research being done also importantly within academic Labs I think the particular situation we find ourselves in now is a unique historical juncture where we are at a point where putting a lot of resources just engineering resources into it have helped a lot and it'll take a while for Academia to catch up to that and in the meantime we're gonna see the end of the same level of basic research being done in Industry moving forward because things work out so they can engineer things and make products and there will be less incentive for them to just pour money into basic research and it'll mean that there will still be very open doors for Academia in basic research so predictions about what the future of work looks like are we going to be AI babysitters and teachers and parents is that what the workforce is going to look like some people will be definitely I think 100 there will be AI psychologists AI teachers uh AI babysitters all these things I'm sure of that and it's part of the reason that I'm I'm in the long run less concerned about employment I think that'll be really disruptive for employment in the short run but like a hundred years from now I don't think it'll be so bad that way any recommendations about what people should do in the short term to upskill yeah uh well learn how to use these models and make yourself more productive with them so that you can take advantage of it rather than be left behind by it or go into the trades or social work or something like that which are totally safe and for anyone interested in studying human and machine Minds like you what was your path uh so I did my undergraduate in computer science and and in fact in a program at the University of Toronto that was specifically called cognitive science and artificial intelligence so it was mostly computer science with a little bit of psychology and philosophy but like 80 computer science and then I did my graduate work in Neuroscience I was able to make that switch because I got into Oxford's four-year refill program which is designed to take students from different disciplines and the first year is just like a catch-up year where they teach you the basics of Neuroscience so that even if you're coming from computer science like I was you learn what a neuroscience student would know or at least they try to do that and they do actually a pretty good job of that uh so so I did my graduate work in Neuroscience uh then my postdoc was also largely Neuroscience but since starting my lab it's been a mix of the two and lastly I've been corresponding with a lovely open AI rep trying to get an opportunity to speak with them so do you have any questions you'd like me to ask if I get a chance if you are keeping things under wraps as you're doing now so you're no longer releasing the model parameters you're not being super clear on how you trained it up on exactly what objective functions you used how you collected your rlhf data all these other things fundamentally like why should anyone trust that you're doing this in a responsible way do you not think that more transparency is needed moving forward to have people trust that you're not gonna release something that's really bad for our society so so you reject the Eleazar utkowski claim that this is a good move because it reduces the not the existential threat but the a bad actor could get a hold of it and do something nefarious with it yeah I don't agree with that I think this I think the concern about the bad actor is getting a hold of it like it's not that bad actors can't couldn't get a hold of some of this stuff but I even if they feel like opening the model parameters is dangerous for that reason and I think there's an argument to be made that way I'm less concerned about that but let's even just go with it for a second let's say yeah you don't want to release the model parameters I don't see any reason not to be transparent about how you've trained it up what the exact data set is and stuff like that because Bad actors are going to be able to if they're really committed figure out how to build this anyway and as it stands we can't trust that you've done due diligence on alignment even if you're not going to open it publicly publicly there needs to be ally the FDA a regulatory board that you do give full access to that can go in and make sure that you're not doing anything that could lead to fundamental misalignment Thank You Blake

Channel: Variable Minds

Views: 769

Keywords: explainable ai, ai news, artificial intelligence, geoffrey hinton, what is explainable artificial intelligence, chat gpt, ai neuroscience, open source ai, consciousness, Continual learning, Embodied cognition, The alignment problem (, Science journalism (, science and tech, science and tech current affairs, large language models, technology news, explainable artificial intelligence, what is explainable ai, chatgpt, interpretable machine learning, variable minds

Id: K_Yqs68J5IE

Length: 34min 39sec (2079 seconds)

Published: Thu Jun 15 2023