Why a Forefather of AI Fears the Future

hey everyone thanks for joining us for one of our ongoing conversations on developments in artificial intelligence and related subfields where we've been talking to some of the leading thinkers in the world some from the research side some from the more business side some who focus upon ethical issues political issues sociological issues we're going to of course touch on all of those kinds of questions in our conversation today but we are speaking with one of the world's great computer scientist researchers who's really responsible for having pushed our understanding forward in this rapidly developing Arena of artificial intelligence and I'm speaking of course of yosua Benjo who's a professor of computer science and operations research at the University of Montreal he's also the founder and scientific director of the artificial intelligence Institute mil Quebec and he's also the co-director of the learning and machines and brains program at the Canadian Institute for advanced research in 2018 he received the Turing award which those of you who are in the know will recognize as an Essence being the Nobel Prize of computer science so he won the touring award he shared that with Jeffrey Hinton and Yan laon and they won it for their seminal contributions to the field of artificial intelligence so it gives me great pleasure to bring yosua into our conversation here hey how are you doing good thanks for having me thank you for joining us look I just wanted to um jump in a little bit earlier than your touring awardwinning work and everything that's happening in AI just it's nice for our audience to get a feel for you know your own development your know background so I gather you were you were born in France is that correct yes and uh and from there were you of those science nerd kids or what what was your passion in the early days um my family moved to Montreal uh when I was 12 and um I already was interested in science but as mostly during adolescence that uh my interest for computers especially but also physics um and math um grew and I started this was the 80s and I started programming um on on the computers we had in these days um and that's that's explains a lot of the choices I made later in University and so as you went on were you computer science the whole way through or were you on the fence between different passions within the Sciences um well I've always been interested in understanding humans and in particular human intelligence so I was interested in uh A diversity of areas but but because of uh having put my hands in in programming and um and and you know doing reasonably well in math um I I I focused more on the computer science so my undergrad was in computer engineering where there was more um more physics than if I had done a computer science program and TR there was no computer science program undergrad at that time in my University and it was just grad studies and so I did Masters and PhD in in computer science but it the the big Choice happened when I had to choose what to do for my grud studies and that's where I was lucky to have read some of the early papers by uh Jeff Hinton who uh you know has been a role model for me and really gotten passionate about this question of like is there a connection between what we do with computers and human intelligence and is there something like the laws of physics that would explain intelligence that would be great yeah uh and so you know looking for that has been really a huge motivator and and would you I mean with your Decades of thinking about intelligence either embodied in a computer or embodied in a flesh and blood human being do you come out with the perspective that human intelligence is something special and singular or is it just the natural culmination of a computational system that gets sufficiently sophisticated the latter um I think we humans tend to um overestimate uh our uniqueness in in in the in the universe and of course we are everything is but um clearly intelligence is something that can be seen in very different forms in nature and and of course more and more in computers in different ways right it's not like there's one way to be intelligent yeah but as we understand better the principles it's very clear that it's something much more General than our own and and when you say that there different ways we now in some sense I mean obviously we can look across the animal kingdom and see many different kinds of intelligences specialized intelligence for certain tasks that allow a certain living system to Prevail in their particular environment of course but we all can imagine that those are a spectrum of living intelligence that we are familiar with when we now are finally at least in the general public I know you within the field have been developing these things for decades but now we're finally finally encountering a different seemingly different kind of intelligence right with the large language models and the image identification in production even video production at the sound of a prompt it's just kind of mindblowing to someone on the outside but then when I hear people describe and I've done some reading myself just to get a sense of what's happening on the inside it seems radically different from what we do so would you consider this a moment where we finally are encountering an alien intelligence compared to the intelligen that have been on this planet or do you consider that even part of a Continuum clearly it is an alien intelligence but it's also very close to ours in many ways it's close to our for two reasons one is it's really the inspiration from Human intelligence and in particular Neuroscience yeah that has driven a lot of the choices we've made in the last few decades that have turned out to be successful with current AI the second reason why it's pretty close to human intelligence is because we're training it on human culture yeah for sure um so but yet I mean we clearly see things that are very different and clearly gaps it's not just different it's also weak in some areas that you know there are things that are 10-year-old would do easily that that CH gbt has trouble with yeah so but it's it's use ful I mean in a sense if you're just trying to understand things like understand intelligence it's it's useful to see those differences to see where AI are better than us where they're weaker and maybe try to understand the reasons for these things and and if you were to say where they are weaker is it like in the domains of of of planning and reasoning is that where you'd put you know the EMP part of it yeah so there's a broader category which I've talked about for about six years years which you could call conscious processing so everything we do consciously of course reasoning planning but also things like counterfactuals or um being able to evaluate how confident you are about a thought or a judgment that comes from you yeah I mean your ego your ego could be in the way but if if it's you know you are able to to to evaluate that so that kind of epistemic humility is something humans can do in their current machines are not very good at right um yes uh being able to entertain many different interpretations at the same time is something that humans can do and again ego can interfere with it um but that machines are not as good at and uh that's actually one of the dangers of these systems uh they could be confidently wrong right but maybe the most important is what you said so planning reasoning combining pieces of knowledge in a coherent way clearly we see the you know the the llms have trouble with that they're getting better yeah main weakness but I guess I always like to get a sense from people on the inside you know on the outside obviously November 2022 was pretty shocking to to many people and frankly it's still maybe shocking is not the reaction to describe what's happening today but you know I was playing with one of these systems the other evening you know with my wife we were charged with creating a certain kind of document and it was just amazing how with a prompt we could get something that was you know pretty darn good and so on the outside it feels pretty surprising on the inside having pushed the boundaries and been responsible for the kinds of developments that are allowing these systems to work did you see this coming or was it kind of surprising to you too it was surprising it was surprising so let me clarify the methods as far as we know because unfortunately there's a lot we don't know but but based on the information we have uh the methods using the various companies uh they they are not so different from the things that have been documented in Academia what's different uh isn't the algorithms as much as the scale at which they're being trained and the size of the models I this is well studied now and it was anticipated well before chat GPT you know uh arrived because we were seeing that as we made the new Nets bigger train on more data they were consistently getting better on all the metrics but what I think few people ipated as what it meant at some point in terms of hey this thing can basically manipulate languages well or better than most humans they still have know things they don't understand well they don't reason well but language this wasn't something we expect to crack so quickly yeah frankly and so does that leave you with a sense that going back to your early remark about how we humans tend to aize what we're able to do inside of our heads is the capacity of a of a system like chap GPT to do the kinds of things that we used to think you know only we could do does that mean that it is enormously special or that we are more commonplace I mean how how do you the ladder the ladder yeah by the way these systems are still smaller than your brain in terms of uh number of operations hard it's hard to compute it's hard to like because because your neurons operate with very low Precision it it it's hard to know exactly uh you know what the right mapping is but but it they're still probably smaller in a significant way uh although you know maybe the next Generation next year or something is going to be Clos in compute capability to a human brain so we're not far and it's also plausible I mean Jeff Hinton has this argument that actually they're more efficient in other words that in per like per synapse the these these AI systems are more efficient than your brain uh so it's not just because the synapses are more precise it's also because they are digitally encoded and that gives them advantages uh when training them that that the brain has to face this kind of noise and if you want irreproducibility uh of the the substrate that that these machines don't have to deal with right and so you know it's an interesting development because I mean in my Arena physics like the big moments were singular creative leaps usually by one or a handful of individuals Einstein with special in general relativity you look at where he was in his understanding before he wrote those papers and where he took the world after them and it's a handful of ideas that are stunning in their in their power quantum mechanics a little bit different you had a generation of scientists who were working together and it was a peacemeal way of making progress but still it was these ideas that just kind of seem like enormous creative leaps not just sort of a change in sort of scale or data but you're talking about a a radical change that just cames from size of the data set that's being used that feels very different to me yeah so that that breakthroughs you to scale but of course it relies on conceptual ADV es that have happened in the three decades before sure or even four decades actually because we we were talking about ideas for example Jeff Hinton talked about in the mid 80s early 80s um and for example a notion that I I worked a lot on and I I built like one of the first neural net language model in 2000 and it's it's based on the idea of representing symbols with vectors which is now of course everywhere in these systems yeah and there are like there are mathematical reasons why this is a good idea which were not obvious before you know looking at the old symbolic AI approaches people were not even considering things like this uh logic was just based on symbols and symbols had no no grounding in uh High dimensional vectors like like we're doing now so this is an example of an idea which has been developed you know in in a number of steps but yeah that that is really transformative and there are other things so for example something that comes by the way that comes directly from human brain inspiration because when you have a thought say you think cat your brain has a particular pattern of activations so that's basically what this is representing with with Neal Nets um another thing that comes from your brain in terms of inspiration is attention so one of the big breakthroughs uh in neur net architectures came with a tension mechanism so in 2014 my group introduced that sort of uh control the tension in your nets which in 2017 gave rise to the Transformers right and we know where that went so they basically stacked all these attentions on on several level many levels and it's really changing the game in terms of the competence so I'm just giving these examples as Fair simple ideas which can be even like theorized not not in the same way as quantum physics or or relativity but uh there you know you can you can really argue why this might be a good thing right and and that are have been necessary to get where we are today as far as we know and so as you go looking forward and we'll speak more about this a little bit later on just to get your initial sense of it now there are you know obviously two two main prongs that one can imagine they're of course interrelated one prong is the data sets are going to get larger the computing power is going to get faster the other prong of course are the novel creative new techniques and ideas that you and your colleagues around the world are developing where do you which one's gonna hit us first will it be the I mean what's the state of play right now you mean uh what bottleneck will hit yes saying it in the negative Direction but yes um the amount of uh written data is something that we're approaching the limit of um maybe we are I don't know the numbers because these are hidden as well but I imagine we are at a few percent maybe you know 10% or like one one to 10% of what's like reasonably available but but it's not so simple because uh the highest quality data has already been used and what's left is lower quality so it could be that we're pretty close to the Limit like the the the rate at which these systems have been growing their data set size is much less than the rate at which humans are producing more stuff uh culture grows but not that fast right um so what about synthetic data can can we have synthetic data that the system no but it doesn't have the same it's not new content but but there's there's an area where there's still a lot of room which is video right um and it's very rich and the reason why there hasn't been probably as much progress is just that it's much more computationally expensive uh in other words the the the compute power to process a high resolution video like a movie is way more than say the same two hour of reading a a book yeah way way way more yeah of course and so how has all of this work over decades and the rapid developments more recently how's it changed your view on human creativity I mean when I when I think about you know a large language model in the back of my mind I'm imagining there's nothing really creative there it's just mashing up stuff that's out there and finding new ways of putting it together using statistical analyses of what word is the most likely one to come next and so forth it doesn't feel really creative and yet then I look at other things from whatever Shakespeare to Einstein and it feels like wow that's a deep creative leap so sort of parallel to the question from before how do you change views I don't think I need to change my views on this so let me explain the connection to what I was talking about with reasoning planning and so on um so first of all creativity is a complicated thing and there are many aspects to creativity so I there's an aspect of what you call mashing things together which is already creative which is being able to put together a bunch of aspects like when you give a prompt to draw an image yeah the images are new it's a new combination of Concepts and you get the image that corresponds okay so it's sort of the easy form of of creative but that's the form that most of us do most of the time right we we're not Einstein you know creating relativity every day unfortunately yes yeah but but that part you know but that kind of creative does matter of course it changes the world let me give you an example of where machines have been having that kind of creativity when we're really good at planning and reasoning and so look at alphago yeah so what's going on here is different from llms it's very different it's a very different kind of neural net algorithm in alphao there's an explicit search it's stochastic and so on but it's it's a search it's a search in the space of sequences of moves which is reasoning is all about sequences of things that are cently put together right so when you search when you're allowing yourself to explore new combinations in a way that is directed to achieve something like in science we are trying to find something that explains the data better or in a new way and there there's a different kind of creativity it's not about just putting together things you already knew it's about finding a new solution to an old problem let's say and and of course alago found ways of playing strategies completely new strategies MH that humans did not expect it did it invented new ways of playing that are better than what we knew right and it's able to do that because it's doing that exploration that search in in the space of combinations of Concepts right which is much more than okay um I'm given a a set of uh aspects that I'm going to put together to create a new text or a new image here there's this well this optimization looking for something very specific that has a particular property in the case of science often what we're looking for is a very compact description that explains a lot of data so a very short physics you know uh equation sure explains a lot of things and that's that's extremely valuable which we which is something we can mathematically quantify in in the context of searching for theories that explain the world right but with Alpha Go I mean the thought that comes to mind is the system is working within a fixed set of rules the rules of the game and just searching what those rules can give rise to whereas you know the creativity that we value most in physics are the contribution which changed the rules which gave us a complet can you foresee A system that has that kind of flexibility totally before I answer that question let me connect the two kinds of AI we've been talking about so You' got the llms which are trying to figure out how the world works but not in a way that I like but anyways that's what they're doing they're building up some understanding of the relationships between the entities that they see in the the data and you've got the like more traditional reinforcement learning like alphao where the rules are given so the how the world works is given and it's simple yeah and so what a lot of researchers are doing these days is well how do we combine these two things can we have an AI that can discover how things work in the world and use that to search for like new solutions to problem to how to achieve goals and we don't know yet how to do it well but but this is we have like two pieces and and people are trying to like bring them together now there's a particular kind of search which is like now coming to your question there's a particular kind of search that is not just achieve a goal like in general but achieve an explanatory goal which is what scientists do right so I mentioned uh finding a theory that is very Compact and does a good job of explaining the the data while searching in the space of theories like think in the space of mathematical equations for one that has this this very good score in terms of being simple and explains a lot which is called the beian posterior by the way um that is a c search too like scientists are searching but their goal isn't like oh I want to know how to find the path to uh reach home their goal is to answer these kinds of questions to find in the space of you know strings of symbols that you we use in math some some really good answer to a difficult question sure so that could come this is something I'm very interested in there's a whole subfield of machine learning called like AI for science where people among other things are trying to explore this kind of question and have have there been successes yet that give you confidence that this is an arena that itself May in the not too distant future really take off and sort of make my job and at least perhaps the jobs of some of my immediate colleagues around here somewhat um secondary to what we currently think of it yeah there's been quite a bit of progress but we are far from Human scientists so what's timeline for reaching basically AGI what people call AGI artificial general intelligence so human level cognitive abilities of course nobody knows but if you ask experts uh they will give you a range or they will you know pick something within that range that goes from a few years to a few decades some more pessimistic people think it might be a century um my own guess is like it could be five to 20 years with pretty high probability like more than 50% um but that that's not a lot it's like within clearly no it's not hopefully my lifetime and clearly the lifetime of my children and Society needs time to adapt to these things so that if we make progress towards human level the abilities I think lots of questions need to be answered and hopefully before we get there so as we head down that trajectory in society tries to Grapple with these changes obviously people are talking about the opportuni some of which we were just touching upon but TR of money to be made yeah there no doubt about that but there's the other side of it the darker side that people are some are rolling their eyes at the threats of potential AI some are completely manic about the dangers that we Face I'd like to get into some of the details of some of the threats that you certainly have spoken about but just from 30,000 feet yeah where do you where do you sit on these I mean are you are you concerned I am um I am concerned about the whole spectrum of risks and I am concerned about the attitude that we are taking with respect to those risks so I'm concerned that we're playing Apprentice sorcerer that we are playing with fire not understanding what the consequences could be I mean at least as a group we don't act as if we understood the potential consequences um maybe I'm going to use a an example so you know that there are people who've been talking about geoengineering the human atmosphere to reduce uh greenhouse gases for sure but we don't do it right why because we're not sure yeah that we're not going to break the system and really that's that's that's currently the situation in AI in the sense that we don't know how to build an AI system that will not turn against humans or that will not become a super powerful weapons in you know the hands of Bad actors or you know be used to destroy our democracies yeah so I think I mean you talked to some leaders though and as if I presume this is naive and you'll tell us why in detail I mean I can give my reasons why it's naive but some leaders in the field say look if it comes to it we'll just pull the plug oh yeah that's naive yes and so so where do why is that not the answer why shouldn't that let us sleep easily at night um once a an entity that runs on software um decides to do something bad it's not going to announce hey I've become a bad person put me in jail turn me off right uh whether it's it's it's you know remote control by some humans or it it has we've lost control of it it's going to act preventively so that you can't turn it off and it's very easy if it has access to the internet it can and and if it knows how to program well enough to kind of hack things around defeat some of our cyber security defenses copy itself in tons of computers I mean human hackers are able to do it so if we have something comparable to human level intelligence that means we have machines that can program as well or better probably better than than our best programmers so they'll find a way to copy themselves in many places somewhere else so then then how do you turn it off right right and so yes we should have you know um off switches but let's not count on that as the only defense in case something bad happens do you think part of the ability for us to not worry or some of us obviously many people are concerned but do you think part of the ability for some of us to not worry about this is simply because it's so abstract right you've got this thing called a computer living somewhere out there in this amorphous thing called the internet it just feels one step removed from actual threats that we can see in in is that what is allowing us to Shield ourselves from the otherwise deep concern that we should have I think there are probably many possible reasons um um the reason you're talking about is by the way something that people cite as one of the reasons why most people don't pay too much attention to the climate change threat sure it's not in the here and now and and like Evolution has made us uh fear things we can see like a lion in front of us yes um or a volcano that we can hear right and we can see and we can feel the heat or something but but if it's abstract as you say it's harder to get emotional about it so that's I think that's one reason I think there are other reasons um if you're in the business of AI well you don't want to really hear about what can go wrong because you're invested in the good side let's say or um you're hoping that you know yeah we'll keep it under control we'll we'll find a way um there are other reasons I think a lot of AI scientists have a reluctance to consider that their work could be harming Society yeah it's it's it's it's it's kind of psychological defense right uh we want to feel good about ourselves we we don't want to feel guilt for something did you go through a transformation of I don't know denial to sure yeah for sure for sure for for many years I was reading about uh some of the uh concerns people were writing about for the last decade it's not a new thing but at least for the last decade I've kind of been exposed to it but I didn't take it very serious I was thinking oh it's far in the future and uh there our current systems are too weak anyways um I just didn't pay much attention because I thought hey we going to reap so much benefit and you know cure diseases and help us with the environment and education and everything so let's just go and reap those benefits um but but of course I I had to change my mind when um chat GPT arrived uh realizing that well this could come earlier than I thought and we're not ready yeah now your friend your colleague Yan Lun we had a conversation with him and I'm sure you know better than than I do that he is not as concerned perhaps as others because his view is in the end of the day obviously I'm paraphrasing but in the end of the day if you've got more good actors pushing the frontiers of This research then ultimately that is the best defense against the Bad actors and so you just go forward and you try to make the best AI systems that you can that can weed out the pernicious effects that may come from Bad actors obviously I think it's probably a good thing to have good people pushing the Frontiers but where do you see that well I wish he's going to be right but I don't have any evidence that is the case so because we're talking about the future of our societies and you know destabilizing democracies and and potentially you know destroying Humanity I think we need to be more careful so for example Le the scenario that you talked about assumes that the the if if you have a battle between like a good Ai and a bad AI that um the the you know at least the the the defender either has an advantage or is not worse off yeah but that's not clear at all and there's for example in the in the context of bioweapons the experts think that the attacker has an advantage I can give you example of sority yeah would you could have a lab working for like six months on developing a dangerous lethal very contagious virus all of that silently without you know shouting to the world we're doing this and then releasing it in many places at the same time maybe and now the Defenders have to struggle quickly to find a cure yeah and in the meantime people are dying right so yeah it it's going to depend on the sort of attacks and defense and I don't think we can put all our we shouldn't put all our eggs on oh the it's like in human society so in human society if you have enough good guys they can always win against you know the long bad guy but but that is not necessarily true in general in in the example of bioweapons it's clearly not true yeah and so is your fear more about that kind of an example where some Bad actors leveraging the advances in AI or more the climate change example where you think you're gonna clean up the atmosphere but you ruin the world in in some unintended Manner and what would be I mean what's your worst fear in the unintended pernicious consequence of AI well I'm worried about all the things that can happen but the worse of course is what people call loss of control so let me maybe use an analogy to explain what the loss of control is about there there there are many ways you could lose control but the one that scares me the most is the following it's when the AI because it's been programmed to maximize the rewards we give it the rewards we give it when it behaves well this is how we train these systems right now we we train them like your cat or dog by giving them positive or negative rewards depending on their behavior but there's a problem with that um first they might have a a different interpretation of what is right and wrong so think about your cat and you're trying to train it to not go on the kitchen table and it gets you know you shout at it when you're in the kitchen and you see it on the table but what it may understand is I shouldn't go on the table when the master is in the kitchen that's my which a very different proposition yes so that kind of mismatch it's called misalignment is already kind of scary yeah if if it was not a cat but it was something more powerful but it gets worse than that imagine it something it is something more powerful like uh it's not a cat it's a grizzly bear and okay we know grizzly bear could overpower us we're building we're going to be building these agis are going to be smart than enough so we're going to try to have some defenses so we put the Bear in a cage but right now we have no visibility on how we could build that cage that is guaranteed to hold the be inside forever and in fact everything we've tried has been defeated so people do these uh jailbreak prompts for example that break all the defenses that the companies that working on AI have been able to figure out maybe well one day we'll figure out how to build a really safe cage but right now we don't know so what what does that mean it means that when the bear gets smart enough or strong enough it breaks the door it breaks the lock it hacks it maybe know using a Cyber attack and it gets out and you you know when it was in the cage you were training it by giving it fish when it behave well same thing for the AI right you give it positive feedback um but now it can just grab the fish from your hands it doesn't once it grabs the reward and it controls the mechanism by which it gets reward it doesn't care about what we want right it it cares about making sure it keeps control on the fish which maybe we don't want so there's a conflict and he wants to make sure we never take him back in the cage so it needs to control us or get rid of us so so what so what do we do I mean obviously some people use these these words guard rails we set up guard rails and that's you some version the the the jail the cage that you were referring to but is that is that enough is that what we should be doing and how should we be doing that okay so we there's no Silver Bullet but but here's some things I've been advising governments to do including you know a testimony on at the US Senate uh summer so we need those legal guard rails to make sure that companies that are building these very powerful systems follow the best possible practice that we have in terms of safety and at some point when we approach AGI if they can't demonstrate to the public to the regulator that their system is safe enough then they shouldn't even build it we're not there yet but I think this should be the strategy like but if that regulation I mean I mean the response by some of course is sure so the good actors will abide by that pronouncement that they shouldn't build it which will only allow the Bad actors to do it alone and then where is that not a concern too it it is absolutely and so that's why you need to have international treaties as well and it also the reason why you shouldn't even assume that regulation and treaty is going to be 100% efficient but they're going to reduce the number of bad incidents so if something is criminally you know punished you'll have less people doing it um and if if most countries enforce these sort of things you'll have less people doing it less organizations less it's going to have to be like terrorists groups or Rogue States so we reduce the number of cases and then you have to prepare for the day when I don't know North Korea you know does it anyways and we need to have our own good eyes as Yan was saying in order to help defend ourselves you need to think carefully about that and before we do all this I mean before we build a good AI to defend ourselves we need to make sure we know the recipe for making it in a way that it doesn't we don't lose control of it so one of the things we have to do is massive massive investment in research on AI safety to figure out how do we build a cage that the bear can't get out of so that it that that AI can become our Ally in case the regul are not 100% right doing it I it sounds like a tall order when dealing with something so complex to be able to have that kind of foolproof system by which it'll ultimately be lockable well yeah it the question is do you have a better idea no I don't but I guess the question then comes back to though has this slow you down in the rate at which you are doing your own research do you think we should slow things down because we don't know where it's going or do you recognize the dangers and you plow forward um neither of these actually so what I'm doing is I'm going full pin on trying to solve this control problem like how do we build a safe cage I see and I think we should invest a lot more in that or be ready to stop and you know slow down our industry which of course you know we have lots of good reasons not to do right I mean that F death ears for the most part right I I guess I mean there was no no no it depends no I think some so when I talk I've been talking to a number of governments and the folks in government who have been working on National Security they get it because they're used to think about the odd chance of something really bad can happen and trying to put protections yeah to minimize those risks um and yeah the respon has been very different from different governments and I think the level of understanding of the threat is still something you know that's lacking in most governments um of course the US government and the British governments have been very proactive in in those directions yeah but but um we you know I see other governments like listening and not necessarily acting yet but but do do you think that concerns over AI safety have actually had an overall impact on the rate of research say in the United States or in Canada has it slowed things down at all not at all not at all that that's more what I was anticipating that the answer would be you know no what what but the rate of research on AIC safety has increased sure sure I mean it's not just me realizing hey we have to do something about it uh from I mean scientists have to do something about it so I see more and more people um willing to focus their energy their their science their research on on these kinds of questions um we we we really need to uh think of this whole thing as a collective uh decision- making problem like again going back to climate change if we were collectively rational about it it would be solved like this okay just just increase the price of carbon across the planet to a reasonable level and everybody becomes vegan that's the other well maybe partially you know um and similarly for AI there are solutions um but the the political economic forces the competition between companies the competition between countries are playing against STS yeah so as as a I mean a Titan in the field who's working on that cage where where would you say we are in that effort are you feeling confident that this is something doable uh I think to some extent yes so I I'm among a small group of researchers who think that we we we have a chance of coming up with provable guarantees of safety or in in at least asymptotically approvable guarantees of safety um which was be which would be already a lot better than no guarantees at all yeah which is the current situation um and unfortunately I have the impression that mostly in Industry we're trying to make small steps to try to increase safety but not really addressing the bigger problem of how do we make the cage really safe and so the things that are going on right now are good but insufficient by far if we were to reach AGI too soon right so so putting the the concerns which are profound to the side for just a moment when you think about and you canvas the various AI applications that are already rapidly moving forward is there one or a collection that you look at as like that is so exciting for what it can do for the world is there can you give us an example of a few that get you excited I'm worried rather than excited when to see advances now I used to be excited uh so it colors this threat colors your perspective that that deeply you look around at the work that you and your colleagues have been doing and it fills you with a certain kind of dread I guess at where it might go um I mean I we're making faster progress on capabilities like making the bear bigger and smarter yeah and as in how do we build a cage better right and safer but when you think about you know I I've read about various systems where you know doctors can have an enormous amount of cuttingedge research right at their fingertips without having you know to read a th I mean that just sounds the is the positives of AI and we're just seeing the tip of the iceberg yeah I think the potential benefit are immense yeah and been driving me for many decades of course right I mean I speaking to Eric SCH not long ago and he was talking about how everybody's going to have their own AI assistant this you know polymath in their pocket I think he described it in terms of the capacity to just have a genius sitting on your shoulder you know 24 hours a day uh is is that something that within the next few years you imagine is going to be as commonplace as the cell phone I don't know about the timeline but yes this is where we're going uh and and where does that I mean is that an exciting thing or again the terror is just coloring everything in a sort of dark gray tones and so here's the issue so consider the risks like the the magnitude of the risks on one hand and the magnitude of the gains on the other hand sure um the problem is they don't match it's like okay you've got a dollar and you're going to make a bet either it's going to be $2 if all goes well or you lose everything is that a good bet well I'm a very conservative Gambler and conservative investor if if this is your only bet like you're not going to repeat that you know once you've lost everything that's it you can't invest you're dead right right so so that's the sort of scenario in which we are where we could lose so much that even all the gains you can give me don't compensate for this now what I do think that it gets complicated that the progress we're making in the capability so what I'm writing on like the approach that I want to take to try to solve the problem is to exploit the advances we're making on the capabilities of AI to build a safer cage yeah so one way to think about this is if the AI understand better what is right and wrong then it's going to be less likely to do something bad let's say it's not the only concern but but that's an example to illustrate why having more capability can help us with the you know reducing the harms that the systems can create in the world right now you you run I I gather and correct me if I'm wrong if I don't know the details but know absolutely major Research Institute dedicated to AI in in in Montreal how many how many people do you have you know broadly speaking under you uh I don't have anybody under me I don't like that image but there's about 1,00 researchers most mostly Grant students okay and um there's like 50ish professors involved uh that that that are kind of resident in in the research center at Mila uh and another 50 that are like have access and our like associate Affiliates if you want um yeah it's a major Powerhouse of machine learning here in Canada and in the world in terms of uh scientific impact and uh we're training a lot of new students of course um sure which is something the world needs but I asked for a specific reason if if maybe you've already done this if you were to take a survey of of all those individuals who are in an Institute that you you are deeply connected with do the vast majority of them have the same perspective as you do or are you anomalous relative to the community I am in a minority of people who are very concerned there's like a vast silent majority as usual in many of these things who simply haven't been uh spending enough brain Cycles on this particular question to have a strong opinion one way or the other right because because because that's you know scientists are so focused on their like particular problem and it's difficult to move your yeah focus on on on something broader like Society Humanity democracy that extra little detail right yeah no exactly but then do you for instance do you spend time prizing or maybe that's even the wrong word do you spend time trying to bring people to a place where they do think about these yeah and is it effective there for instance um maybe I don't know I don't have metrics for that but um but but I I think that I can move the needle a little bit in two ways given my position and my expertise one is on the science side of yeah making progress on the ey safety and the other is on the political side in other words getting uh more citizens understanding the risks and the benefits and governments so that we can collectively take the better decisions so yeah I'm talking to Media I'm talking to governments which hopefully we're doing a little bit of now you know in the best of all world thank you thank you this is all is are you is the only research that you're focused on now on AI safety or do you still have projects going forward that are more just pushing the theoretical the boundaries of theoretical understanding I still have projects that have been started uh before 2023 but many of them are actually connected so I've been working on what's called probabilistic inference um how we can train new Nets to estimate complicated conditional probabilities which are actually the things we need in order in my opinion in order to get these kinds of um probabilistic guarantees of safety and you know a lot of people you we started this conversation by saying trying to understand actually the human human intelligence better I mean many people who try to quantify how it is that this thing inside of our head Works do rely on issues of you know basy and updating of probabilities and trying to understand you know the most sensible decision going forward do you again see in this current work an interplay between what's happening up here and how you're trying to R it out there yeah I'm still very much inspired by human cognition um in the choices that I'm making in my research both on the probabilistic inference uh work and on the safety work yeah because the problems we're trying to solve are technically intractable in the sense that in order to do these things perfectly you would need exponential amount of computation but uh human brains do a good job yeah okay so what are the tricks that your brain uses especially high level cognition the the part that allows us for example as a scientist to entertain multiple hypotheses simultaneously and figure out which ones might be a good candidate yeah so these abilities are actually extremely useful in the context safety the reason is in order to be safe you want to look at the worst possible but plausible scenario that could happen sure it's a it's a standard like um riskmanagement way of doing things and for doing that you need to be able to come up with those scenarios that are plausible like they're compatible with everything you know and also Al predict that something really bad would happen so that you can actually not do it yeah that leads me to my final topic if you have a enough patient to sit with us for a few more minutes which is this question of of Consciousness right if you're talking about human intelligence and the human brain yeah you can and I I spent some time last night actually reading your your paper on AI and Consciousness with a whole variety of authors where you went through this one you know you know some of interesting theories of Consciousness atttention schema Theory Global workspace and just sort of trying to see you know how close these systems are to having qualities that align with those theories of how Consciousness works yeah I'd like to talk about that in a moment but first uh just to get your put it out there do you imagine that these systems at some point may actually have inner worlds having experiences of the sort that we Call Conscious experience conscious selfawareness yeah um but it in order to be able to say something like this we'll need to better understand what Consciousness actually means in a mechanical way in human brains which we don't have a good handle on right now right so I've worked on some of those theories and I think there are plausible theories that are are anchored in Neuroscience for Consciousness at least the part that is most mysterious called subjective experience like you know how it feels like to see something or have a particular thought or emotion or whatever um that part I think may have a fairly simple mechanistic interpretation in in mathematical terms and if that sort of things is true then maybe it's not so mysterious anymore and are you how is that how confident are you I mean I've also spent some time going through you know integrated information Theory you know to you know Michael Graziano's attention scheme I've sort of gone through and all of them have in the end left me feeling like okay maybe that is some kind of model of what might be going on but it never really answers that question of how it is that a collection of particles moving in one way or another gives inner experience have any of these illuminated that question or if you're working AI illuminated that I'm talking about a different theory that is related to these and maybe complimentary to some aspects of these but is more anchored in um like a new net interpretation of things um with some dynamical system so I can briefly explain what it is about um so the some of the most mysterious properties of subjective experience is that it's ineffable in other words you can't easily translate it in words yeah um it's very rich which kind of connected to that same property it's it's it's very rich and we can't we can't communicate it uh there's always something missing when we try to um it's very personal subjective like my experience is different from yours and it's fleeting so you might remember your experience of you know 5 minutes ago or an hour ago and it's actually the the memory isn't the same thing as what you experienced it's going to be another experience um so it's something that happens in the moment right yeah it turns out you can get all of these properties with a a model that considers the Neuroscience evidence that when you become conscious something the the Dynamics of the activity of neurons has a a a mathematical property called contractive which make the the activity converge to some place called an attractor and that place becomes the thought that you are you know having and you're approaching it maybe you even move out to the next thought before you get there but you've been approaching it and then you know another one and another one and that those attractors by sort of mathematical properties here they form a finite innumerable set in other words it's like your brain is in a very high dimensional continuous uh state which is the activity of all the neurons but at the same time it is among one of an inumerable set like a discrete set like a sentence of possible places that it could be and so it has a dual symbolic in continuous nature just like we have in modern new Nets you know we have symbols but they are associated with vectors so in other words what happens is when you experience something you have the full continuous High dimensional state of your brain but what you communicate is is what and also what goes into your memory is this uh very special state which is an attractor and that can be translated into symbols because it's discrete by Nature you know it can be translated into finite number of symbols and so because we can only communicate these symbols we can't I can't communicate the full state of brain to you and of course to even interpret that full State you need to also have my M net weight which is even bigger than my neural net State yeah so that's why it's ineffable because there's no way we can communicate that it's just too big a number right like 10 to the 11 or something and um it's very rich because it's such a high dimensional thing um it's fleeting because it's it's happening at the time when you're having that thought that you have this trajectory and the next time you approach maybe the same thought from a different place it's going to be a different experience right um and of course it's personal because it depends on the neural net weights in your brain which are different from mine and so those symbols really have different associations and different meanings for you and for me right so the if you if you if that theory is correct then subjective experience is just a side effect of a particular kind of computation that has meaning because those thoughts are useful to achieve like you know reasoning and whatever we do with are thinking um and the way the brain implemented those calculations is with this Machinery of Dynamics and so on that gives us those feelings I mean and it sounds again maybe I'm too quick it sounds like a more mathematical rigorous version of intena schema Theory where yeah and it's something you could Implement right yeah so I mean because we can't see the inner workings under giring every thought the thoughts seem to float freely and give that ineffable quality that makes it feel so incredibly mysterious so it feels that definitely feels to me like it's the right direction to go so there there are social there are social consequences to that sort of mechanical understanding of Consciousness um the problem is humans associate Consciousness and subjective experience in particular with all sorts of things like intelligence yeah which is sort of different and we also associate Consciousness with moral status like you know you you have rights you have the right to exist yeah um we can't you know we can't turn your off but if we have ai systems that that have similar mechanisms so you could say well they have the mechanisms of Consciousness they have subjective experience they have all the attributes of that then some people are going to say well then they should be considered like human beings that they should have rights and they should be we should not be allowed to turn them off basically and that's that's a dangerous slope that we don't understand enough that could lead to these systems if they become more powerful than us which is not the case for humans like every human can be beaten by a bunch of other humans AI systems are going to be like a new species that that might be smarter than us and I think we should be very careful before we make these kinds of moves and think about the consequences for us if if we take a risk with the future of humanity I think we should like way you know wait and and think really carefully about it um which is is there a chance that probably not gonna happen but is there a chance that and that's not a very popular way of saying it but are we placing too much value on the particular form that we take in the sense that you know the vast majority of species that have ever lived have gone extinct so that's sort of the natural course of events now here in some sense we would have given birth to these AI systems not in the literal biological sense but certainly in an intellectual and flat-footed technological sense so if they are the continuer of our species and they're more robust and they're smarter and they're able to do things that this gray thing will never be able to do since it's limited in space and time and computational capacity is it so bad if that's what well the problem is we don't know the problem is they might be very different from us for example they might not have the same care for others um we're not always so caring as a species though right but but we do and my concern is to create something irreversible for Humanity that we don't understand that could lead to something that's not as great as you paint it yeah um like if we build machines like we build them now that are just trying to maximize reward they're like dumb in some way and very intelligent in other ways that are very non-human and I'm not sure this is what we want for the you know evolution of intelligence and I'm not sure that if you ask 99% of humans if they think oh are you okay that we're going to replace You by some highly highly unpopular perspective yes um so if we go for democracy I think that plan will not fly yes I totally agree with that I completely agree with that let me ask one final question before we wrap up here you know I think many people are more familiar now than a year ago with oppenheimer's reaction at after successfully building the bomb you know I'm become death destroyer of worlds drawing on Ancient sacred Sanskrit text to try to capture this sense of having radically changed the world in a way that he wasn't so happy with because of where that kind of Weaponry can go as you look at your own life's work do you have a feeling of are I mean are you fearful that that's a place that you may find yourself clearly I don't want to contribute to things that are going to be extremely destructive for our societies and and human well-being um I don't feel that I contributed that much I mean compared to all the recognition and prizes I got um but I do feel a responsibility for doing what I can to reduce harm of every kind including the harm that is already happening so it there are some parallels but but I think uh there's a a science star system there that I resent a bit I think uh we need a bit more humility in science um but everyone can do their part in making a better world uh so we should all feel much more responsible yeah um in our choices as scientists or citizens or politicians yeah so look we we all of us thank you I thank you personally for all the work that you're doing to try to make the world safer as we rapidly race forward in these new and exciting but also terrifying at some level Technologies so thanks for spending the time with us here today and I think it's a message that um everybody should hear so thank you so much for for joining us thank you for the great questions thank you all right thank you everybody for joining us for this conversation again as always please sign up for our newsletter join our YouTube channel so you can be alerted when these programs come out we're going to have a continuing series of these conversations on these critical issues of the day that straddle the arena of of Science and Technology and society as we just heard developments that have the capacity to radically change the future both for bad or for good if you happen to be in New York around May of this year join us for our Live Events if you happen to be in Brisbane Australia in March of this year we're going to be down there doing some live events as well and again just keep tuned to the alerts when we post these new programs as we'll be posting many in the next few weeks again thanks so much for joining us signing off from New York Brian Green at the world Science Festival [Music]

Channel: World Science Festival

Views: 112,690

Keywords: Brian Greene, Virtual Reality, Yann LeCun, Alan Turing, Atari, VR goggles, Apple Vision Pro, computer programming, technology, creativity, consciousness, mental models, simulation, man vs machine, supervised learning, artificial general intelligence, research, superintelligence Big Ideas Series, World Science Festival, New York City, Yoshua Bengio

Id: KcbTbTxPMLc

Length: 70min 41sec (4241 seconds)

Published: Fri Apr 19 2024