#90 - Prof. DAVID CHALMERS - Consciousness in LLMs [Special Edition]

David Chalmers is a professor of philosophy and Neuroscience at New York University and an honorary professor of philosophy at the Australian National University he's co-director of the center for mind brain and Consciousness as well as the Phil papers Foundation his research focuses on the philosophy of Mind especially consciousness of course and its connection to Fields such as cognitive science physics and Technology he also investigates areas such as the philosophy of language metaphysics and epistemology with his impressive breadth of knowledge and experience David Chalmers is a leader in the philosophical community so there has been all this controversy and debate recently about whether language models might be conscious and I've actually been especially interested in not so much people's gut intuitions one way or the other but the reasons and the evidence on both sides first what is the evidence that these language models might be conscious and second what is the evidence of the reasons for saying they're not conscious when it comes to the reasons in favor there are a few things there uh well it's you can get these systems to say they're conscious on the other hand you can also get these systems to say they're not conscious so it's not clear what weight that carries for me actually the most strongest consideration in favor of Consciousness in these systems is they're showing signs of general intelligence the ability to do many things in the study of Consciousness we take the domain General use of information to be one marker of Consciousness maybe that's maybe one strike in favor when it comes to reasons against there are many many objections that people have raised one objection is that these language models aren't biological and biology is required to be conscious I mean that would be a very general reason that would prevent any artificial Consciousness so I end up setting that one aside but there are many other reasons I say what is the X such that you need X to be conscious and these large language models don't have X well besides biology there's maybe having senses and a body right now language models don't have sensors don't have a body on the other hand it's not terribly difficult people have already built extensions of these language models call them Vision language action models that deal with sensory information in the form of images and which control a body may be hooked up to a robot body or a virtual body so I think in this case it looks like even if the large language models May lack what's required for Consciousness these extended versions what I call LOM pluses extended large language models might overcome those objections and I think there are a series of objections to language models that take the same form people argue that they don't have World models or self models people argue they don't have recurrent processing or a global workspace maybe most deeply they don't seem right now to be unified agents with fundamental goals of their own but all of these seem to be to be temporary issues maybe today's language models lack them but tomorrow's extended language models May well have all of these properties they may have World models and self models they may have recurrent processing and a global workspace they may even have unified agency so I think there's a research program here of trying to build AI systems that have all those X's which are currently missing from large language models and then the question is just say we build within say 10 years or so a system which has all of those X's it needn't even be human level intelligence maybe just will have something like fish level cognition or intelligence because you know most people think fish are conscious just say we build a system like that that has the uh the sensors the body if only a virtual body the uh the world models and the self models the recurrent processing the global workspace the unified agency will it then be conscious we might have these systems by say 2032 10 years or so some people are going to say those systems are not conscious if so I ask them what's missing interesting but but for Charmers um this conscious experience it what's the cash value of it well exactly I've never managed to work that out uh and um that contradicts human first person accounts another maybe interesting question is one you prompted with the AGI comments so to what extent do you think Consciousness is a necessary condition for AGI assuming one can achieve it or could one get to AGI without the Consciousness path it's a really interesting question and this you know this interacts with the question of the philosopher zombie the system which has all the behavior of a human being but none of the Consciousness I've argued that zombies are at least conceivable we can imagine a system that behaves like us without Consciousness but that doesn't mean that such a system could actually exist in this world I'm inclined to think that in this world intelligent intelligence and Consciousness go very tightly together I mean maybe there are some systems my uh my colleague Ned block has considered the idea of a giant lookup table that stores every possible conversation passes the Turing test that would not possibly be conscious my own view is that uh is there any system with reasonable mechanisms that generates human-like intelligent Behavior will probably be conscious but if it turns out that you know that maybe massive scaling yields poor pure feed forward Transformers with Incredible capabilities but no consciousness I mean that's not something I can totally rule out I do think this is a potential concern if you're thinking about the role of AI in the future that we're going to have if it's possible that we're gonna have beings which are highly intelligent which lack Consciousness entirely then their role with respect to say moral social legal issues is going to be very delicate and hard to adjudicate philosophical zombies are imaginary beings that are physically and behaviorally indistinguishable from conscious humans yet lack conscious experience this concept has been used to explore the implications of certain thought experiments such as the Chinese room argument which posits that a computer program can pass the Turing test yet still lack consciousness to build a zombie one could imagine a cabinet full of raw materials from which to construct a perfect copy of a human all memory knowledge and personality would be embedded into the physical system and the Clone would behave as if it were conscious however our intuition tells us that when the parts were unassembled there would be no consciousness present the concept of a philosophical zombie has been widely discussed in the philosophical literature yet there is no consensus on the matter proponents of the idea of a philosophical zombie suggests that if such a being were possible it would suggest that Consciousness is an emerging property of certain physical systems and not an intrinsic property of the universe this would have implications for how we understand the nature of Consciousness and its relationship to physical systems opponents of the idea of a physical zombie point out that there's no empirical evidence that such a being could exist they argue that if there was a philosophical zombie then a physical system that is behaviorally and physically indistinguishable from a conscious human would still lack consciousness this would suggest that Consciousness is an intrinsic property of the universe and not an emergent property of certain physical systems the debate over the concept of a philosophical zombie continues to rage in the philosophical Community while it's unclear that this issue will ever be resolved anytime soon it's an intriguing concept that has been used to explore the relationship between Consciousness and physical systems the zombie is a system which has you know all the uh the intelligence and the outward behavior of a human being but not of the Consciousness and the thought is you know relevant here in New Orleans it's like wouldn't it be a drag to be a zombie of none of the uh none of the experience of a uh of a human being so it goes I act like you act I do what you do but I don't know what it's like to be you what Consciousness is I ain't got a clue I got the zombie Blues for a very long time philosophers and scientists have shared a common interest in and puzzlement over the hard problem of consciousness physicality seems insufficient to be the source of the subjective coherent experience that we call consciousness David Chalmers invented the phrase the hard problem in an attempt to describe this very phenomenon finding a way to link mental states to neurophysiological activity is the hard problem in a nutshell it's beyond the scope of human inquiry to determine why it is that any kind of physical action should and presumably does give rise to subjective experience consider the hypothetical situation of two sets of identical twins you have the same DNA upbringing and educational experiences the twins will have quite distinct subjective experiences of the world this exemplifies the challenge of explaining how subjective sensation may result from objective physical action now you might just counter and say this example is simply a matter of complexity however the Chinese room argument is another thought experiment which might shed light on the hard problem of consciousness in this hypothetical scenario a person is locked in a room and given a series of instructions and symbols and an external Observer is provided the output after the individual has been requested to modify the symbols in accordance with the instructions now despite the fact that the individual in the room is manipulating symbols the thought experiment shows that he doesn't understand since he does not have the requisite phenomenal experience Consciousness is a hard problem that has been discussed for Millennia but never resolved some say that it simply can't be done while others think that a mix of Neuroscience philosophy and psychology will eventually get us there no matter what the solution to the hard problem of Consciousness is it's clear that it's a difficult problem to solve I started this uh my original title for the talk is the word sentience in the end I decided that word is just too confusing even more confusing than the word Consciousness so I went with uh with Consciousness instead but as I use the terms those two things abruptly equivalent Consciousness is sentience which is subjective experience being is conscious in the philosopher Thomas Nagel's phrase if there's something it's like to be that being if that being has subjective experience like the experience of seeing of feeling of thinking wrote a famous article called what is it like to be a bat where he says that you know it's very hard to know exactly what a bad subjective experience is like when it's using Sonar to get around nevertheless most of us believe there is something it's like to be about it is conscious it has subjective experience on the other hand most people think there's nothing it's like to be let's say a water bottle the bottle does not have subjective experience it's not conscious now it's clear that the modern human has a huge problem with not knowing there'll always be a tension between having a theory of the world in hand which makes excellent scientific predictions but which bucks against our evolved intuitions John Locke Emanuel Kant and John Rose all have argued that an intelligible framework is necessary for humans to make moral and rational decisions such a framework provides us with a sense of stability and order as well as a sense of meaning and purpose Nietzsche however questioned the role of the personality of the philosopher in his own philosophy he argued that the innermost drives of a philosopher's nature can shape their morality suggesting that the search for a New Foundation of values knowledge and beliefs must come from within the individual in his essay what's it like to be a bat Thomas Nagel considers this subject from the vantage point of a bat since our experiences are so different he says we can never really comprehend what it's like to be a bat since bats use echolocation rather than vision Nagel claims that their perception of the world is fundamentally different from our own the bat he says has a much richer sensory experience than we do because it can pick up on a wide variety of objects and surfaces many more so than we can since our experiences are so different from our bats Nagel concludes that we can never know what it's like to be one according to him the only way to know what it's like to be a bat is to become one which is obviously not conceivable he believes that we can only do our best to Envision what it's like to be a bat and to fly around the world Nagel's inconceivability argument has its advantages and disadvantages on the one hand it's a great way to test the boundaries of our knowledge understanding the world is more complicated requires admitting that there are certain things that we just can't fathom however the inconceivability argument might be seen as a barrier to our human ability to comprehend the universe at all much like how Noam Chomsky said that rats would be unable to solve the prime number maze accepting that there are certain things which we will never be able to grasp May prevent us from understanding and expanding our frontiers of knowledge the most significant and defining quality of mental phenomena is consciousness making it impossible to solve the mind-body problem it's impossible to comprehend the material foundations of Mind fire reductionist theories since they ignore the subjective nature of things this subjective nature is linked to a particular point of view and cannot be represented by reductionist theories exploring the topic in the context of an example which highlights the difference between subjective and objective conceptions is useful for illustrating the relationship between subjectivity and making visible the relevance of subjective qualities the bat is used by Nagel because it exemplifies the difficulty of the subjective nature of experience by providing a spectrum of activity and sensory experience so different from ours that it's particularly striking now since the raw material of our imagination comes from our own experiences its scope is necessarily truncated we can't infer the bat's inner existence from our own and if we attempt to Envision what it's like to be a bat all we have is our own limited mental capacity nothing in our current makeup allows us to contemplate what it might be like to exist in the state of interpolated Badness even if such a transformation were theoretically possible so while we may make some assumptions about what it must be like to be a bat based on our own understanding of the animal's anatomy and behavior we recognize that each individual bats experience must have a unique subjective quality which is beyond our cognitive Horizon some scientific realities May defy description even in the broadest terms of Human Experience most people can't fathom the nature of a person's experience who's been deaf and blind from birth even sentient bats or martians from another planet would have a pretty hard time imagining what it's like to be human it's possible for us to have faith in the reality of things whose precise nature we can't yet fathom but reality may be known from just one vantage point it's unclear how the organism's physiology May somehow convey the underlying nature of experiences to simplify is to advance towards more objectivity a precise understanding of the true essence of things it's not clear what's meant by objective nature right the objective nature of an experience if it's not understood in relation to the subject's unique perspective even a scientist from Mars who's never seen a rainbow a bolt of lightning or a cloud Nagel said couldn't miss the fact that they're all examples of physical events Lightning's objective nature is not depleted by its outward form a more direct link seems to exist between one's experiences and their beliefs we don't characterize the world around us in terms of specific Sensations or our senses take away from it but rather in terms of the broad impacts it has and the traits which may be detected by methods other than the human senses we feel that the more our representation doesn't rely on a human or anthropocene trick perspective it kind of becomes more objective it's conceivable to forsake an anthropocentric perspective in favor of something more abstract while maintaining the same understanding Nagel said that not all experiences follow the pattern of relinquishing a subjective experience in favor of a more objective one in order to better comprehend the same events because it is fundamental to The Human Experience the uniqueness of the human Viewpoint is something which we can't ignore indefinitely modern psychology's behaviorist approach may be traced back to an attempt to replace subjective mental experience with something more scientifically rigorous like a theory of Mind there must be some essential similarity between going through certain physical processes and having a thought occur in your head if thoughts are actually physical processes Nagel said that the failure of physicalist ideas which rest on an incorrect kind of like objective examination of Mind proves nothing due to our lack of a conceptual framework physicalism remains an incomprehensible Viewpoint to him Nagel said we should try to develop an objective phenomenology not dependent on empathy or our imagination I mean if you were to describe what it's like to see to someone who's been blind their entire life you would use this phenomenology perception structural properties May lend themselves to a dispassionate analysis to get some insight of our own experience it's important to build Notions which are distinct from those which we acquire via first person learning or experience Nagel concluded that the insights into the physical substrate of experience may be made possible by an objective phenomenology so I'll just uh talk about language models in general which the topic I don't need to introduce to this crowd typically these are going to be with Transformer architecture multi-head self-attention gpt3 is still um the best known one but uh I take it gpt4 any day now people I'll also talk about uh extended large language models extensions of what I'll call llm classes these are models that add further capacities to the pure text or language capacities of a language model now there are very familiar Vision language models that add image processing language action models that add control of a physical or a virtual body there are large language models extended with things like code execution database queries simulations and so on and in many ways I'm interested in not so much not just the large language models today but the capacities of the systems that may be developed in the coming years which will include these llm plus models and my questions will be questions like well first our current large language models plausibly conscious but maybe even more importantly could future large language models and extensions thereof be conscious and what challenges need to be overcome on the path to conscious machine learning systems I really see this I mean there are challenges here and there are objections but I really want to think of this also as a constructive project one that might ultimately leave to a potential road map to Consciousness Nai systems so the thing I've sometimes seen characters in GPT models do is sometimes it's something strange happens in the story like you know you have some stories and characters and then suddenly saying you know the model the size in a you know refrigerator token is selected refrigerant characters will notice that as sometimes they'll question whether they are a simulation the the characters aerogeneity generate storm will be like this is just a simulation you know am I real and stuff like this in a way how does this relates to the idea of con stimulation are the characters inside of gbt potentially the generals are entities that have some amount of money or practice so um David I had dinner with Connolly he on Friday night and he's one of the founders of a life for AI and they famously recreated gpt3 type language models which is to say you know these statistical Auto regressive self-attention Transformers models which is a bit of a mouthful uh trained on a bunch of internet text and um some of their models are pretty large actually they just released a 22 billion parameter version called gbt Neo X now Conor really likes to think that gbt3 is an example of general intelligence he's a True Believer and therefore convinced that we should be concerned with AI alignment as a top priority he said something which really fascinated me which relates to your book reality plus he said that in many of the fictional stories that he's generated of gbt3 the characters sometimes becomes self-aware that they're in a simulation if you like so when they see an unexpected word in the sequence um you know they they kind of have that realization that that they're in a simulation of reality so it's kind of glitchy and the simulated residents are aware of the glitches so I think in all likelihood Conor has an overactive imagination but I thought it was a fascinating thought when I spoke with Keith about it yesterday he said it reminded him of the idea of being Fooled by Randomness there is a famous example with the ghosts in Pac-Man many years ago where people were convinced that there was some kind of intelligent Behavior and the ghosts were coordinating with each other which actually turned out not to be the case at all so I think one of the themes of our conversation today will be intention versus extension or causal structure versus correlation you know which is to say this idea that just because something appears to be conscious or intelligent in its physical configuration of the output doesn't mean it necessarily is so gpt3 conscious or not um I uh you know I'm open to the idea there's Consciousness even in very simple systems you know like even the uh the worm like one of these basic um C elegans it has uh 300 neurons uh it's sensitive tonight to light it engages in simple action and so on I'm at least open to the idea that that worm has some element very simple element of Consciousness and then you've got to ask yourself if a worm with a 300 neurons can possibly be conscious what about gbg3 with 100 billion plus uh parameters it's so much more complex than the worm there's so much going on it has so many capacities to do uh to communicate uh to talk to uh to reason even to uh to explain you might think okay well if a worm is conscious then gpt3 has got some element of Consciousness too on the other hand gpt3 is not very much like an agent you know it's not a it's not a doesn't correspond to a person that has consistent beliefs or desires or actions I mean it's more of a chameleon ride because you can you can set up your prompt engineering on gpt3 to get it to take on any kind of persona you like uh with any set of you know beliefs and desires that you like so maybe gpg3 is not so much like an agent or a person but a meta agent that can take on many different forms depending on exactly how it's engineered I mean I would not be inclined to think that right now gbt3 has anything like say the consciousness of a uh of a human being that would require a kind of coherence and unified intelligence right now I think is beyond gpt3 but does it have some basic element of Consciousness I certainly wouldn't rule that out well I so I I think we would agree though and and you said this explicitly in the book that there's no reason not to believe that it will be possible at some point for Consciousness to be in silico um is that true and so what I want to try and do is bring the the zomb The Perennial zombie question the philosophical zombie into the context of of reality Plus or a simulation and my question is this and it's really about trying to find the uh the explanation for Consciousness so suppose as a result of us posting this video here um you know the Architects whoever created this the simulation that we're in they watch the video and they say I think it's about time to have a little chat with uh with David and they come to you in a in a dream because they can do that and they present to you evidence that you're convinced that you are actually conversing with with the Architects they explain look uh you guys are actually running on this neutron star super computer that we built and and by the way we so happen to still use C code and they show you the the C code for for agents in this world for people here's all the C code that's running your brain and your neurons and everything and this module over here and they show you a module source code and they say this is the Consciousness module now not everybody's running this in fact you have one running but a lot of people we don't bother installing them and they're just actually zombies because we didn't attach you know this this conscious this module to them but you're you're running a Consciousness module would you accept that that bit of code you know however long it is this module 500 lines 10 000 lines would you accept that that code explains consciousness I would really need to know what their evidence is and what their what their science is I mean maybe uh these beings could convince me that I am actually part of a of a simulation if they presented the code for the simulation and they uh they did amazing stuff there were messages in the sky they turned the Empire State Building which is right outside my window if they turned it up they turned it upside down in the uh in the sky then maybe that would be pretty good evidence I'm in a simulation I mean it wouldn't be totally conclusive maybe they've just given me some great drugs or uh maybe my whole life had been in reality but just for a moment just for tonight they put me into a virtual reality that was totally convincing and presented me with all this fake evidence of turning the Empire State Building upside down so I don't know if it would be totally convincing but yeah I think I could probably get pretty good evidence that I'm in a simulation but then you're telling me okay now they've got some Theory Of Consciousness they think that they've put a uh a Consciousness module into me but not into uh into some other beings but then I interact with those beings and they're they're perfectly normal I don't know if those beings actually say tell me that they're conscious I'm gonna be inclined to uh to believe them just as I do with other human beings so if they tell me that they think that these beings are philosophical zombies beings that lacka that lack Consciousness I'm Gonna Wanna Know Why merely programming a merely programming the beings isn't going to tell us that we're going to need a theory of Consciousness for that I mean it's really important in these discussions maybe I should have done this to start with to distinguish Consciousness and let's say intelligence intelligence is all about you know behavioral capacities the things you can do carrying on a conversation taking place uh sophisticated goal directed action and so on but intelligence here is objective it's something you can measure from the objective point of view ultimately coming down to behavioral capacities whereas consciousness as I understand it is subjective Consciousness is a matter of how things feel from the inside when I carry on this conversation I can see you guys I have some visual subjective experience of you I can hear you I can hear the sound of my own voice I can feel my body I got thoughts running through my head that's all the subjective experience of Consciousness so you know it may well be that take gpt3 it's got rather sophisticated capacities for intelligence already but it is totally unclear whether it's conscious whether it has subjective experience maybe it has a bit and the trouble is of course well intelligence is very easy to operationalize and measure from the outside Consciousness is not and you can illustrate that with the with the idea of the uh the philosophical zombie which is behaving in extremely sophisticated ways but has no subjective experience on the inside I don't actually believe in philosophical zombies in the sense I don't think that there are uh intelligent beings without Consciousness around us but I do think it's a coherent hypothesis and that does mean that whenever we create say an artificial intelligence with uh with very sophistic even very sophisticated intelligence you're always going to be able to raise the question is it conscious of that question we need a theory of Consciousness okay but the scenario here though was you've you've been convinced by these you know beings that they are actually The Architects of the simulator and they're literally showing you the source code for the Consciousness module but I think what you're telling me is you still wouldn't believe it unless they presented you with a theory that you could personally understand yeah well I think that being a good simulator makes you a good theorist of Consciousness I could you know maybe potentially I could build a great Universe simulation eventually and simulate brains really well and so on that doesn't mean that I've suddenly solved the problem of Consciousness I'm still going to know you know which bit which processes in the brain or elsewhere correspond to Consciousness and what merely being a great simulation engineer doesn't yet turn you into a turn you into a scientist okay once we've got those simulations of brains end of the world and so on maybe they'll be amazing things we can do to try and you know put these systems in different circumstances see how they behave see when they respond see which kind of simulations actually report being conscious and maybe we could use that as really great data for a theory of Consciousness but I don't think it suddenly resolves a problem just being able to build a say a brain simulation introduce himself to me and he tells me you have a profound philosophical analysis of Consciousness and well yeah they've been doing at your school so I just want to very much encourage you to keep going with this project and see if you can help us figure out the nature of Consciousness I'm looking forward to your future career as a philosopher he brought in a thesis yes also from Ilias is this Twitter thing because either your daughter brought the uh the tweet from yeah yeah yeah because it's my thing from AI said why not start with this and that was the same kind of starting to think yeah and now it becomes really uh uh interesting for the philosophers because now it has a I will say AI it has some actuality and stuff I think it's a really cool topic now you and your daughter have found something and yeah yeah yeah yeah yeah yeah you can collectively figure out AI classes yeah I know we don't know how to approach it because it's it's really a hard topic to Define it because I'm was a math guy and it's not so well defined and therefore I don't like it uh what don't you like so definitions of qualia Consciousness and quality yeah and you need an operational definition yeah in terms of input and output yeah yeah give me something I can hand off to my engineer yeah yeah I would like it I understand the Instinct yes I did put forward a call for Consciousness benchmarks but yeah if we had some common bit I don't think you're going to find an uncontroversial Benchmark but even having a few different benchmarks for different kinds or aspects or theories no that's absolutely right yeah yeah I agree I agree then we can yeah train up more and more systems get closer and closer on those benchmarks and then we can argue about whether they're conscious what's the subjectiveness uh here uh you cannot look into another thing and that's uh you only it's a secondary uh measurements exactly that's make it makes it hard but it would be a lot easier if we had the Consciousness meter I'd just wave it in your head yeah searching for Consciousness yeah exactly searching for consciousness okay they're accepts Consciousness but we don't have this this but physics is the same you don't see atoms you don't see other stuff yeah service reasonable David can I just start with a question which is that um a lot of your arguments about the existence of intelligence are based on functionalism which is to say if it's similar to if you read a book by Peter norved it says oh I'm an intelligent system has planning it has reasoning it has sensing it has perception therefore it's intelligent and and I I feel that this is a form of behaviorism but a slightly more sophisticated form of behavior behaviorism and then I feel that some of this has been projected into um I think your your conversation the other day on Consciousness which is to say um first of all could we have a Turing test for Consciousness but also could we have a functional approach to define whether something is conscious and the thing is I'm I'm a little bit my intuition is to be skeptical of this but I have to be completely honest I have no principled way of describing why I'm skeptical about it so if you could play Devil's Advocate how would you attack your own position on that functional um view well I actually make a distinction between consciousness and intelligence um intelligence is a matter of the outputs you produce given any input especially goal directed Behavior the ability to solve problems Consciousness is the subjective experience so and you know when it comes to Consciousness the easy problems are explaining the behavior the reactions the hard problem is explaining subjective experience so actually start by drawing the distinction that you want to draw between say experience and function that said when it comes to Consciousness in other people our we don't have direct evidence of their Consciousness or we can go by is their functioning and so in general what another person says is evidence of their Consciousness if you tell me that you're conscious at a given time or even if you show a certain sophisticated reactive Behavior to a certain stimulus we'll usually take that as evidence that you are that you are conscious so I'd say that yeah the view I was arguing the other day was not that we should define consciousness as some function but we should look for those functions which are the best evidence of Consciousness in other people but also in animals and ultimately in AI systems would it be fair to say that you would argue that it's a sufficient condition but not a necessary condition function is a sufficient condition for experience yeah I mean not sufficient across the board and the famous philosophical zombie will be a creature which has all the functioning indeed but none of the uh none of the experience so that's at least conceivable but my own view is that those systems can't exist in the real world so in the real world certain kinds of functioning may be sufficient conditions for a Consciousness the the coin of the realm in this science of human consciousness is verbal report someone says they're conscious you believe them unless there's reason to believe otherwise or they say they're conscious of a given stimulus but humans we know are conscious makes it easier uh non-human animals are already tricky at least they're related to us so we think we've got some grounds for this but AI systems we start from the position of being totally unsure of whether they're conscious or not so you know Turing had the genius of proposing at least one sufficient condition behaving just like a human yes to me that would be good enough yes fascinating but what shorter that actually counts then we start to argue yes indeed indeed well um I mean with the functionalism it's it's really interesting how much of it is to do with human intelligibility rather than the actual thing which exists um I feel that a lot of it is taking something which is very complex and beyond our cognitive Horizon and creating or reducing it to a to a cognitive framework which is compatible with with the templates that we have in in our own brains Consciousness itself is yeah very mysterious and subjective and on the other hand our tools for understanding the brain are mostly cognitive and informational but they do seem to be of strong correlations at least between the two you replicate all that informational or cognitive structure you replicate the brain you will replicate Consciousness yes so the two seem to be tied together what we need to do is find the neural correlates of Consciousness those bits of the brain that seem to go along with being Consciousness being conscious and maybe even more fundamentally the you know the informational correlates of Consciousness the very general mechanisms that support Consciousness and then if we find the informational correlate say of consciousness in human beings maybe then we can project those to AI systems see what we do know very well the informational properties of an AI system try to build an AI system that has the right kind of informational properties to serve as an informational or a computational correlative Consciousness then we might have some evidence that it's conscious it still won't be conclusive because how do we know for sure whether that is in fact the computational or informational correlative Consciousness but there would be some evidence if you could say something to Ilia right now what would you say well I want him to define the scale of Consciousness for us if neural networks are going to be slightly conscious then fantastic that means we've got a we've got a scale and Ellie's a smart guy so I want to see his definition of the mathematical scale and the mathematical structure of Consciousness so that so that uh so that these uh neural networks come in at the a certain Epsilon I myself am quite sympathetic with the idea that Consciousness could run very deep in nature if pan sarcasm is true nothing is at zero yes even particles have got a little bit on that measure the today's neural networks even yesterday's neural networks were slightly conscious today's neural networks Maybe slightly and a bit unconscious the problem with pan psychism is it's such a parsimonious and beautiful Theory um we've just done a show on the Chinese room argument and in in the simplest possible terms what would you say to someone like so who says that um subjective experience requires biology I mean it's a coherent hypothesis that we don't know what the physical basis of conscious is one view is this biological another view is informational and then there are other views besides I'm very strongly inclined towards the informational View partly because the structure of Consciousness I find seems to be so deeply tied to information processing my visual Consciousness has a very complex structure of objects and qualities located in space it seems to very closely track the informational properties of the visual system and it's just hard to see for me how the biology is essential for example we can imagine gradually replacing the biological components of our visual system by say silicone chips that play the same role um keeping the information processing the same to me is very plausible that would actually preserve Consciousness we even have the thought experiment where you gradually replace every neuron in the brain by a silicon chip I suppose soil and others are gonna have to say that your Consciousness gradually degrades during this process but I think it was if it's a good enough if these silicon chips are good enough simulations of the neurons that they actually replicate all the information processing by far the most plausible thing is this will replicate the Consciousness too and if it's possible that any old silicon system can be conscious and it looks like that will refute the biological view that biology has required fascinating fascinating and could you also just touch on why you think Consciousness is one of the only examples of a strongly emergent phenomena which is to say not deducible from truths in the lower level domain I know we touched on this when we spoke last time but I would love to just get like a slightly fresher perspective it's not going to be fresh I'm sorry because this is a very old issue for me uh nothing obviously nothing I can say about this will be fresh okay but um um you know most of the things most of the phenomena we try to explain in the world are matters of structure and and function and when it comes to say explaining life we need to explain how it is a system can reproduce how it can adapt how it can metabolize how it can grow these are all kind of functional questions analogous to the easy problems of Consciousness and those are perfectly well suited for explanation and physical terms because physics gives you a complicated story of structure and of Dynamics but when it comes to Consciousness it looks like we have to explain something that goes beyond structure function Dynamics we've got this those are the easy problems when it comes to Consciousness we also have this hard problem subjective experience it seems even once you explain the structure of the function the Dynamics of the cognitive system there's still this further question why subjective experience I think what makes it unique is it's not really a question about structure and function at all so yeah I think some people who believe in a global workspace think that is somehow essential through Consciousness maybe that's what someone like Joshua bengio thinks um I heard you on the Clinton say once yeah we got to uh powerful enough systems that didn't need this low capacity workspace then maybe we wouldn't need Consciousness anymore okay and there I think the thought is they're somehow time conscience by definition to slow processing in a low capacity system to system but I'd be more inclined they would still have subjective experience it would just be very fast and Rich yeah yeah I mean there's already a debate between people about how fast and rich is conscious you know poorer and slow Consciousness poor and slow and fast and Rich I'm kind of on the side where it's already pretty fast and pretty rich it feels like that uh it feels like that for people yeah I suppose actually relates to one of my um well I guess the next question which is I just heard I heard recently talking about animal ethics and talking about insect Consciousness it just hadn't really occurred to me that any other animals might have a faster rate or even a richer Consciousness than people might but I I just heard it sort of speculated that well insects might have insects my you know but maybe it's it's the way that their brains are smaller and neurons are closer together that means they might actually have a richer and deeper experience than we do an individual land would have a human being I don't think I've heard that one before that's it was the first time I'd heard it what do you think about I've heard suddenly people saying by sheer weight of numbers the number of insects in the world means that insect Consciousness to making sure our insects don't suffer [Music] um yeah I haven't heard the uh the reasoning in favor of individual insects being more conscious it has been like this so simple and integrated maybe that there are Consciousness has a certain Purity yeah I guess I guess it is related to these Notions of integration and and speed and messages are being sent and singles being processed I I I know zero Neuroscience to to actually substantiate anything else my colleague Jeff sibo is tinyapple Center NYU on uh we saw the NYU talks you introduced it one of the special leads is going to be non-human countries in animals okay and his real specialty is insects yeah so I think there's gonna be a lot of focus on on insect Consciousness there great uh and I guess if I can ask one more um given your personal you know Credence probability estimates like do you think there are significant ramifications for studying Ai and and how how researchers go forward like to studying probing deploying language but you're talking about sort of ethical and safety issues yeah I mean yeah there's a million I mean there's a million very familiar issues that already arise for any human level AI whether it's conscious or not yeah um you know you've got to make very sure it's going to be aligned with our goals otherwise it's going to be so powerful that it's going to have the ability to bring about whatever it wants to bring about so we've got to make sure it wants to bring about the right things I guess one question is whether Consciousness changes that in any fundamental way right I'm not sure that it fundamentally changes the questions of say of our safety which may work the same with other the systems are conscious or not it does add a whole new moral Dimension which is worrying about the AI systems themselves it turns out that we're building AI systems already which are conscious yeah then we may need to consider well how are we training them does it turn out that every time they get I mean every time they back propagate they're actually suffering yeah yeah I certainly don't rule that out then yeah that would be a potentially a moral disaster and even if they're not conscious yet if they're going to be conscious in say 10 years could it be that in building these more sophisticated language models where producing an enormous amount of of suffering I mean I think this is the case where I think we want to have better theories with a better Theory Of Consciousness we might also have better theories of like of the basis of certain kinds of Consciousness like affective Consciousness suffering and so on perhaps that could be used to try to uh you know organize the way we use these systems certainly so you know the negative States they go through or or minimize but that's a bit of a pipe dream at the moment I think the uh you know the ethical consequences are fairly enormous I'm not myself and ethicist but um I do hope that as an emphasis think pretty seriously about these issues sure actually Jeff Sandra at NYU is going to be thinking a lot about exactly these issues the moral consequences of Consciousness in AI systems as well as as animals it sounds like that's an issue uh interested enough certainly I mean I care um a lot about safety of systems and it's mainly why I work on explainability is because I think this is instrumentally useful for what sort of thing are you doing with explainability yeah um a few there are really so many angles uh some of the work we were presenting here is focused on supervising model explanations for tasks so you might get a model's doing some complicated task it gives some kind of explanation of Its Behavior to you and what we're doing is is lining those explanations up with human explanations for the same task so you're trying to get the model to behave in a more human-like way and um it gets get roughly towards like aligning its own behavior and reasoning particularly with human reasoning yeah so this is actually changing the processing and the model that's right that's right more and more explainable yeah not just giving feedback at the the final output where it's like correct or incorrect but trying to supervise the features and reasoning process that it doesn't work too yeah and you take a hit in performance no no it improves performance it improves generalization because you specifically want to come up with tests where it's the only way you're going to generalize successfully is by using the correct reasoning process that a human would so it improves performance okay yeah yeah that's good is this true in general and uh explainability work or uh there's sort of a performance hip there's kind of a discussion um a lot of terms get thrown around you know there's this post-hoc explainability approach which the idea is to you know apply it to any model and you don't have to change the objective you don't have to change the training you don't have to worry about losing performance there's another basically camp at this point that is uh you know suggesting the postdoc isn't enough what we need to do is change the training and maybe you do list a bit of a little bit of performance because you're constraining the optimization you don't just care about tax performance you care about this additional objective the objectives have to trade off a little bit uh then maybe you lose performance um I I think I think the like debate at high level is like kind of interesting but for any particular problem um it's it's it's like well I guess what the the I guess the point I really want to make is like the the debate's like a little too high level like the actual details of the problems don't match up well to this like General accuracy intervalent trade-off so what's big and new and exciting and explainability work um I think some of the interesting stuff is getting really low level with representations and weights and figuring out for a given representation what is the encode for um the single unit level single neuron single layer yeah some other side further as like localization work so you have a specific behavior in mind you want to know what component in the model is responsible for this uh this this could I would feel it seems like it's a very very cool project but also them so limited because so much about the models are doing is out of a local level uh right right so you want to build up the understanding so you might start with smaller units and then ask this is a little bit of like Chrysalis circuits uh research trying to understand what the brain is doing by looking at what single neurons are doing what it helps a little bit you get inside yeah this is my neuroscience some voxels that are like a million neurons because you have to you might need a higher level of granularity to start with but it turns out to be really hard to do explainability work at that yeah so a single layer is somewhere in between things go in both directions because it's like okay maybe this entire layer is responsible for something the model only has 30 layers one of the layers is responsible for something so then you you you might meet in the middle where it's like these neurons but also this layer and and you're trying to you try to exactly Define the scope of the behavior or the function that is being carried out yeah it is it is like there's a lot of analogies to Neuroscience for because it is like uh reverse engineering It's a larger Stone cool that's exciting yeah

Channel: Machine Learning Street Talk

Views: 16,409

Keywords:

Id: T7aIxncLuWk

Length: 53min 48sec (3228 seconds)

Published: Sun Dec 18 2022

Reddit Comments

[u/FusionRocketsPlease]

He looks young and handsome in this thumb lol

[u/paconinja]

thank you, MLST is one of the most informative podcasts on theory around