Joscha Bach: Nature of Reality, Dreams, and Consciousness | Lex Fridman Podcast #212

the following is the conversation with yoshi bach his second time on the podcast yoshi is one of the most fascinating minds in the world exploring the nature of intelligence cognition computation and consciousness to support this podcast please check out our sponsors coinbase codecademy linode netsuite and expressvpn their links are in the description this is the lex friedman podcast and here is my conversation with yosha bach thank you for once again coming on to this particular russian program and sticking to the theme of a russian program let's start with the darkest of topics so this is inspired by one of your tweets you wrote that quote when life feels unbearable i remind myself that i'm not a person i am a piece of software running on the brain of a random ape for a few decades it's not the worst brain to run on have you experienced low points in your life have you experienced depression of course we all experience low points in our life and we get appalled by the things by the ugliness of stuff around us we might get desperate about our lack of self-regulation and sometimes life is hard and i suspect you don't get to your life nobody does to get through their life without low points and without moments where they're despairing and i thought that let's capture this state and how to deal with that state and i found that very often you realize that when you stop taking things personally when you realize that this notion of a person is a fiction similar as it is in westworld where the robots realize that their memories and desires are just stuff that keeps them in the loop and they don't have to act on those memories and desires that our memories and expectations is what make us unhappy and the present rarely does the day in which we are for the most part it's okay right when we are right sitting here right here right now we can choose how we feel and the thing that affects us is the expectation that something is going to be different from what we wanted to be or the memory that something was different from what you wanted it to be and once we basically zoom out from all this what's left is not a person what's left is this state of being conscious which is a software state and software doesn't have an identity it's a physical law and it's a law that acts in all of us and it's embedded in a suitable substrate and we didn't pick that substrate right we are mostly randomly instantiated on it and there all these individuals and everybody has to be one of them and uh eventually you're stuck on one of them and um have to deal with that so you're like a leaf floating down the river you just have to accept that there's a river and you just that you are an agent is a construct right what part of that is actually under your control and i think that our consciousness is largely a control model for our own attention so we notice where we are looking and we can influence what we are looking how we are disambiguating things how we put things together in our mind and the whole system that runs us is this big cybernetic motivational system so we're basically like a little monkey sitting on top of an elephant and we can put this elephant here and there to go this way or that way and we might have the illusion that we are the elephant or that we are telling it what to do and sometimes we notice that it walks into a completely different direction and we didn't set this thing up it just is the situation that we find ourselves in how much prodding can we actually do of the elephant a lot but i think that our uh consciousness cannot create the motive force is the elephant consciousness in this metaphor no the monkey is the consciousness the monkey is the attentional system that is observing things there is a large perceptual system combined with the motivational system that is actually providing the interface to everything and our own consciousness i think is a tool that directs the attention of that system which means it singles out features and performs conditional operations for which it needs an index memory but this index memory is what we perceive as our stream of consciousness but the consciousness is not in charge that's an illusion so everything outside of that consciousness is the elephant so it's the physics of the universe but it's also society that's outside of europe i would say the elephant is the agent so there is an environment which the agent is stomping and uh you are influencing a little part of that agent so uh can you is the agent a single human being what's what which object has agency that's an interesting question i think a way to think about an agent is that it's a controller with a set point generator the notion of a controller comes from cybernetics and control theory control system consists out of a system that is regulating some value and the deviation of that value from a set point and it has a sensor that measures the system's deviation from that set point and an effector that can be parametrized by the controller so the controller tells the effector to do a certain thing and the goal is to reduce the distance between the set point and the current value of the system and there's environment which disturbs the regulated system which brings it away from that set point so simplest case is the thermostat the thermostat is really simple because it doesn't have a model the thermostat is only trying to minimize the set point deviation in the next moment and if you want to minimize the set point deviation over a longer time span you need to integrate it you need to model what is going to happen so for instance when you think about that your set point is to be comfortable in life maybe you need to make yourself uncomfortable first right so you need to make a model of what's going to happen when and this is task of the controller is to use its sensors to measure the state of the environment and the system that is being regulated and figure out what to do and if the task is complex enough the set points are complicated enough and if the controller has enough capacity and enough sensor feedback then the task of the controller is to make a model of the entire universe that it's in the conditions under which it exists and of itself and this is a very complex agent and we are in that category and an agent is not necessarily a thing in the universe it's a class of models that we use to interpret aspects of the universe and be when we notice the around us a lot of things only make sense at the level that you are entangled with them is we interpret them as control systems that make models of the world and try to minimize their own set points so but the models are the agents the agent is a class of model and we notice that we are an agent ourself we are the agent that is using our own control model to perform actions we notice we uh produce a change in the model and things in the world change and this is how we discover the idea that we have a body that we are situated environment and that we have a first person perspective still don't understand what's the best way to think of which object has agency with with respect to human beings is is it the body is it the brain is it the contents of the brain that has agency like what's the actuators that you're referring to what is the controller and where does it reside or is it these impossible things like because i keep trying to ground it to space-time the three-dimensional space and the one dimension of time what's the agent in that for humans there is not just one it depends on the way in which you're looking at the thing in which you're framing it imagine that you are say angela merkel and you are acting on behalf of germany then you could say that germany is the agent and in the mind of angela merkel she is germany to some extent because in the way in which she acts the destiny of germany changes there are things that she can change that basically affect the behavior of that nation state okay so it's hierarchies of to go to another one of your tweets with uh i think your uh playfully mocking jeff hawkins with saying his brains all the way down so it's like it's agents all the way down it's agents made up of agents made up of agents like if phanja marco's germany and germany's made up a bunch of people and the people are themselves agents in in some kind of context and then people are made up of cells each individual so is it agents all the way down i suspect that has to be like this in a world where things are self-organizing most of the complexity that we are looking at everything in life is about self-organization yeah so i think up from the level of life you have agents and below life you rarely have agents because sometimes you have control systems that emerge randomly in nature and try to achieve a set point but they're not that interesting agents that make models and because to make an interesting model of the world you typically need a system that is true and complete can i ask you a personal question uh what's the line between life and non-life it's personal because you're a life form so what do you think in this emerging complexity at which point does the thing start being living and have agency personally i think that the simplest answer is that life is sales because life is what cells cells biological cells so it's a particular kind of principle that we have discovered to exist in nature it's modular stuff that consists out of basically this dna tape is a read write head on top of it that is able to perform arbitrary computations and state transitions within the cell and it's combined with a membrane that insulates the cell from its environment and there are chemical reactions inside of the cell that are in this equilibrium and the cell is running in such a way that this this equilibrium doesn't disappear and the cell goes if the cell goes into an equilibrium state it dies and it requires something like an neck entropy extractor to maintain this this equilibrium so it's able to harvest like entropy from its environment and keep itself running yeah so there's information and there's a wall to protect to to to maintain this disequilibrium but isn't this very earth-centric like what you're referring to as i'm not making a normative notion uh you could say that there are probably other things in the universe that are cell-like and life-like and you could also call them life but eventually it's just a willingness of to find an agreement of how to use the terms i like cells because it's completely co-extensional with the way that we used the word even before we knew about cells so people were pointing at some stuff and saying this is somehow animate and this is very different from the non-animated stuff and what's the difference between the living and the dead stuff and it's mostly whether the cells are working or not and uh also this boundary of life where we say that for instance a virus is basically an information packet that is subverting the cell and not life by itself that makes sense to me and it's somewhat arbitrary you could of course say that systems that permanently maintain a disequilibrium and can self-replicate are always life and maybe that's a useful definition too but this is eventually just how you want to use the word is it uh so useful for conversation but is it uh somehow fundamental to the universe do you think there's a actual line to eventually be drawn between life and non-life or is it all a kind of continuum i don't think it's a continuum but there's nothing magical that is happening um living systems are a certain type of machine what about non-living systems is it also a machine there are non-living machines but the question is at which point is the system able to un uh perform arbitrary state transitions in uh to make representations and living things can do this and of course we can also build non-living things that can do this but we don't know anything in nature that is not a cell and is not created by the lola life that is able to do that not not only do we not know i don't think we have the tools to see otherwise i always worry that we we look at the world too narrowly like we have there could be life of a very different kind right under our noses that we're just not seeing because we're not either limitations of our cognitive capacity or we're just not open-minded enough either with the tools of science or just the tools of our own mind yeah that's possible i find the thought very fascinating and i suspect that many of us ask ourselves since childhood what are the things that we are missing what kind of systems and interconnections exist that are outside of our gaze but the um we are looking for it and physics doesn't have much room at the moment for uh opening up something that would not violate the conservation of information as we know it yeah but i i wonder about time time scale and scale spatial scale whether we just need to um open up our idea of what like how life presents itself it could be operating in a much slower time scale yeah a much faster time scale and it's almost sad to think that there's all this life around us that we're not seeing because we're just not like thinking in terms of the right of the right scale both time and space what is your definition of life what do you understand this life [Music] entities of sufficiently high complexity that are full of surprises i don't know i don't have a free will so that just came out of my mouth i'm not sure that even makes sense there are certain characteristics so complexity seems to be an unnecessary property of life and i almost want to say it has ability to do something unexpected it seems to me that life is the main source of complexity on earth yes and complexity is basically a bridgehead that order builds into chaos by modeling by processing information in such a way that you can perform reactions that would not be possible for dump systems and this means that you can harvest neck entropy that dump systems cannot harvest and this is what complexity is mostly about yeah in some sense the purpose of life is to create complexity yeah increasing i mean there there's um there seems to be some kind of universal drive towards increasing pockets of complexity i don't know what that is that seems to be like a fundamental i don't know if it's a property of the universe or it's just the consequence of the way the universe works but there seems to be this small pockets of emerging complexity that builds on top of each other and starts having like greater and greater complexity by having like a hierarchy of complexity little organisms building up a little society that then operates almost as an individual organism itself and all of a sudden you have uh germany and merkel but that's not obvious to me everything that goes up has to come down at some point right so every if you see this big exponential curve somewhere it's usually the beginning of an s-curve where something eventually reaches saturation and the s-curve is the beginning of some kind of bump that goes down again and there is just the thing that when you are in sight of an evolution of life you are on top of a puddle of negentropy that is being sucked dry by life and during uh that happening you see an increase in complexity because life forms are competing with each other to get more and more and a finer and finer corner of that like entropy extraction but that i feel like that's a gradual beautiful process like that's almost you know follows a process akin to evolution and the way it comes down is not the same way it came up the way it comes down is usually harshly and quickly so usually there's some kind of catastrophic event well the roman empire took a long time uh but that's would that be would you classify this as a decrease in complexity though yes i think that this uh size of the cities that could be fed has decreased dramatically and you could see that the quality of the art decreased and it did so gradually and maybe future generations when they look at the history of the united states in the 21st century will also talk about the gradual decline not something that suddenly happens do you have a sense of where we are are we on the exponential rise are we at the peak or are we the downslope of the the united states empire it's very hard to say from a single human perspective but i it seems to me that we are probably at the peak i think that's probably the definition of like optimism and cynicism so my nature of optimism is i think we're on the rise but uh i think it's just all a matter of perspective nobody knows but i do think that erroring on the side of optimism like you need a sufficient number you need a minimum number of optimists in order to make that up thing actually work and so i tend to be on the side of the optimists i think that we are basically a species of grasshoppers that have turned into locusts and when you are in that locust mode you see an amazing rise of population numbers and of the complexity of the interactions between the individuals but it's ultimately the question is is it sustainable see i think we're a bunch of lions and tigers that have become domesticated cats to use a different metaphor as i'm not exactly sure we're so destructive or just softer and nicer and lazier but i think we have monkeys and not the cats and if you look at the monkeys they are very busy are the ones that have a lot of sex those monkeys not just the bonobos i think that all the monkeys are basically a discontent species that always needs to meddle well the gorillas seem to have a little bit more of a structure but it's a different different part of the tree [Laughter] okay uh you mentioned the elephant and the the monkey riding the elephant and uh consciousness is the monkey and there's some prodding that the monkey gets to do and sometimes the elephant listens i heard you got into some content maybe you can correct me but i heard you got into some contentious free will discussions uh is this with sam harris or something like that not that i know of some people on clubhouse told me you made a a bunch of uh um big debate points about free will well let me just then ask you where where in terms of the monkey and the elephant uh do you think we land in terms of the illusion of free will how much control does the monkey have we have to think about what the free will is in the first place we are not the machine we are not the thing that is making the decisions we are a model of that decision making process yeah and there is a difference between making your own decisions and predicting your own decisions yes and that difference is the first person perspective and what basically makes decision-making um and the conditions of free will distinct from just automatically doing the best thing is that uh we often don't know what the best thing is we make decisions under uncertainty we make informed bets using a betting algorithm that we don't yet understand because we haven't reverse engineered our own mind sufficiently we don't know the expected rewards we don't know the mechanism by which we estimate the rewards and so on but there is we observe ourselves performing where we see that uh we evade facts and factors and the future and then some kind of possibility some motive gets raised to an intention and that's informed bad that the system is making and that making of the open bet the representation of that is what we call free will and it seems to be paradoxical because we think that's the crucial thing is about it that it's somehow indeterministic and yet if it wasn't deterministic it would be random and of course it cannot be random because it was if it was random if just dice were being thrown in the universe randomly forces you to do things it would be meaningless so the important part of the decisions is always the deterministic stuff but it appears to be indeterministic to you because it's unpredictable because if it was predictable you wouldn't experience it as a free will decision you would experience it as just doing the necessary right thing and you see this continuum between the free will and the execution of automatic behavior when you're observing other people so for instance when you are observing your own children if you don't understand them you will use this agent model where you have a agent with a set point generator and uh the agent is doing the best it can to minimize the difference to the set point and it might be confused and uh sometimes impulsive or whatever but it's acting on its own free will and when you understand what happens in the mind of the child you see that is automatic and you can outmodel the child you can build things around the child that will lead the child to making exactly the decision that you are predicting and in under these circumstances like when you were a stage magician or somebody who is dealing uh with people that this you sell a car to and you completely understand the psychology and the impulses and the space of thoughts that this individual can have at that moment under these circumstances it makes no sense to attribute free will because it's no longer decision making under uncertainty you are already certain for them there is uncertainty but you already know what they are doing but what about for you so is this akin to like systems like cellular automata where it's deterministic but when you squint your eyes a little bit it starts look like there's agents making decisions at the higher so when you zoom out and look at the entities that are composed by the individual cells even though the there's underlying simple rules that make the system evolve in deterministic ways it looks like there's organisms making decisions is that where the illusion of free will emerges that jump and scale it's a particular type of model but this jump in scale is crucial the jump in scale happens whenever you have too many parts to count and you cannot make a model at that level and you try to find some higher level regularity and the higher level regularity is a pattern that you project into the world to make sense of it and agency is one of these patterns right you have all these cells that interact with each other and the cells in our body are set up in such a way that they benefit if their behavior is coherent which means that they act as if they were serving a common goal and which that means that they will evolve regulation mechanisms that act as if they were serving a common goal and now you can make sense of these all these cells by projecting the common goal into them right so for you then free will is an illusion no it's a model and it's a construct it's basically a model that the system is making of its own behavior and it's the best model that it can come up with under the circumstances and it can get replaced by a different model which is automatic behavior when you fully understand the mechanism under which you are acting yeah but the another word for model is what story so it's the story you're telling i mean you actually have control is there such a thing as a you and is there such a thing as you having control it's like are you manifesting your evolution as an entity in some sense the u is the model of the system that is in control it's a story that the system tells itself about somebody who is in control yeah and the contents of that model are being used to inform the behavior of the system okay so the system is completely mechanical and the system creates that story like a loom and then it uses the contents of that story to inform its actions and writes the results of that actions into the story so how's that not an illusion the story is written then or or rather we're not the writers of the story yes but we always knew that no we we don't know that when did we know that i think that's mostly a confusion about concepts the conceptual illusion in our culture comes from the idea that we live in physical reality and that we experience physical reality and that you have ideas about it and then you have this dual list interpretation where you have two substances res extensor the world that you can touch and that is made of extended things and res cognitions which is the world of ideas and in fact both of them are mental representations one is the representations of the world as a game engine that your mind generates to make sense of the perceptual data and the other one yes that's what we perceive as the physical world but we already know that the physical world is nothing like that right quantum mechanics is very different from what you and me perceive as the world the world that you and me perceive is a game engine yeah and there are no colors and sounds in the physical world they only exist in the game engine generated by your brain and then you have ideas that are not cannot be mapped onto extended regions right so the objects that have a spatial extension in the game engine are res extensor and the objects that don't have a physical extension in the game engine our ideas and they both interact in our mind to produce models of the world yep but you know when you play video games i understand that what's actually happening is zeros and ones inside of uh inside of a computer instead of a cpu and a gpu but you're still seeing like uh the rendering of that and you're still making decisions whether to shoot to turn left or to turn right if you're playing a shooter or every time you start thinking about skyrim and elder scrolls and walking around in beautiful nature and swinging a sword but it feels like you're making decisions inside that video game so even though you don't have direct access uh in terms of perception to the bits to the zeros and ones it still feels like you're making decisions and your decisions are actually feels like they're being applied all the way down to the zeros and ones yes it feels like you have control even though you don't direct access to reality so there is basically a special character in the video game that is being created by the video game engine yeah and this character is serving the aesthetics of the video game and that is you yes but i feel like you have control inside the video game like the all those like 12 year olds that kick my ass on the internet so uh for when you play the video game it doesn't really matter that they're zeros and once right you don't care about the vids of the bus you don't care about the nature of the cpu that it runs on what you care about are the properties of the game that you're playing and you hope that the cpu is good enough yes and a similar thing happens when we interact with physics the world that you and me are in is not the physical world the world that you and me are in is a dream world how close is it to the real world though we know that it's not very close but we know that the dynamics of the dream world match the dynamics of the physical world to a certain degree of resolution right the causal structure of the dreamworld is different so you see waves crashing on your feet right but there are no waves in the ocean there's only water molecules that have tangents uh between the molecules that are uh ex the result of electrons in the molecules interacting with each other aren't they like very consistent we're just seeing a very uh crude approximation isn't our dream world very consistent like to the point of being mapped directly one-to-one to the actual physical world as opposed to us being completely tricked is this is like where you have like that it's not a trick that's that's my point it's not an illusion it's a form of data compression yeah it's an attempt to deal with the dynamics of too many parts to count at the level at which we're entangled with the best model that you can find yeah so we can act in that dream world and our actions have impact in the in the real world in the physical world yes to which we don't have access yes but it's basically like accepting the fact that the software that we live in the dream that we live in is generated by something outside of this world that you and me are in so is the software deterministic and do we not have any control do we have so free will is uh having a conscious being the free will is the monkey being able to steer the elephant no it's slightly different basically in the same way as you are modeling the water molecules in the ocean that engulf your feet when you are walking on the beach as waves and there are no waves uh but only the atoms on more complicated stuff underneath the atoms and so on and you know that right you would accept yes there is a certain abstraction that happens here it's a simplification of what happens in simplification that is designed in such a way that your brain can deal with it temporarily and spatially in terms of resources and tuned for the predictive value so you can predict with some accuracy whether your feet are going to get wet or not but it's a really good approach it's a really good interface and approximation yes it's like equals mg squared is a good equations are good approximations for what they're much better approximation so to me waves is a really nice approximation what's all the complexity that's happening underneath basically it's a machine learning model that is constantly tuned to minimize surprises so it basically tries to predict as well as it can what you're going to perceive next are we talking about which is the machine learning our perception system or the dream world the machine world is a dream world is the result of the machine learning process of the perception system that's doing the compression yes and uh the model of you as an agent is not a different type of model or it's a different type but not uh not different as in its model like nature from the model of the ocean right some things are oceans some things are agents and one of these agents is using your own control model the output of your model the things that you perceive yourself as doing and that is you what about the fact that like when you're standing um and with the water on your feet and you're looking out into the vest like open water of the ocean and then there's a beautiful sunset and it well the fact that it's beautiful and then maybe you have like friends or a loved one with you and like you feel love what is that as the dream world what is that yes it's all uh happening inside of the dream okay but see the word dream makes it seem like it's not real yeah of course it's not real the physical universe is real but the physical universe is incomprehensible and it doesn't have any feeling of realness the feeling of realness that you experience gets attached to certain representations where your brain assesses this is the best model of reality that i have so the only thing that's real to you is the thing that's happening at the very base of reality like for something to be real it needs to be implemented so uh the model that you have of reality is a real in as far as it is a model right it's an appropriate description of the world to say that there are models that are being experienced but the world that you experience is not necessarily implemented there is a difference between a reality a simulation and a simulacrum the reality that we are talking about is something that fully emerges over a causally closed lowest layer the idea of physicalism is that we are in that layer that basically our world emerges over that every alternative to physicalism is a simulation theory which basically says that we are in some kind of simulation universe and the real world needs to be an apparent universe of that where the actual causal structure is right and when you look at the ocean and your own mind you are looking at a simulation that explains what you're going to see next and we are living in a simulation yes but the simulation generated by our own brains yeah and this simulation is different from the physical reality because the causal structure that is being produced what you are seeing is different from the causal structure of physics but consistent hopefully if not then you are going to end up in some kind of institution where people will take care of you because your behavior will be inconsistent right your uh behavior needs to work in such a way that it's interacting with an accurately predictive model of reality and if your brain is unable to make your model of reality predictive um you will need help so what uh what do you think about donald hoffman's argument that it doesn't have to be consistent the dream world to the the what he calls like the interface uh to the actual physical reality where there could be evolution i think he makes an evolutionary argument which is like it could be an evolutionary advantage to have the dream world drift away from physical reality i think that only works if you have tenure as long as you are still interacting with the ground tools your model needs to be somewhat predictive well in some sense humans have achieved a kind of tenure in the animal kingdom at some point we became too big to fail so we became postmodernist it all makes sense the version of reality that we like oh man okay yeah but basically you can do magic you can change your assessment of reality but eventually uh reality is going to come bite you in the s if it's not predictive do you have a sense of what is that base layer physical reality you have like uh so you have these attempts at the theories of everything the very very small of like strength theory or what um stephen wolfram talks about with a hyper grass these are these tiny tiny tiny tiny objects and then there is more like quantum mechanics that's talking about objects that are much larger but still very very very tiny do you have a sense of where the tiniest thing is that is like at the lowest level the turtle at the very bottom do you have a sense i don't think that you can talk about where it is because space is emergent over the activity of these things so space uh the coordinates only exist in relation to the things other things and so you could in some sense abstract it into locations that can hold information and trajectories that the information can take between the different locations and this is how we construct our notion of space yeah and uh physicists uh usually have a notion of space that is continuous and this is a point where i tend to agree with people like stephen warfram who are very skeptical of the geometric notions i think that geometry is the dynamics of too many parts to count and when there are no infinities if there were two infinities you would be running into contradictions which is in some sense what uh google and turing discovered in response to hilbert's call so there are no infinities there are no infinities fake there is unboundedness but if you have a language that talks about infinity at some point the language is going to contradict itself which means it's no longer valid in order to deal with infinities and mathematics you have to postulate the existence in uh initially you cannot construct the infinities and that's an issue right you cannot build up an infinity from zero but in practice you never do this right when you perform calculations you only look at the dynamics of too many parts to count and usually these numbers are not that large they're not googles or something the big the infinities that we are dealing with in our universe are mathematically speaking relatively small integers and um still what we're looking at is dynamics where um a trillion things behave similar to 100 trillion things or something that is very very large because they're converging and these convergent dynamics these operators this is what we deal with when we are doing the geometry right geometry is stuff where we can pretend that it's continuous because uh as if we subdivide the space sufficiently fine grained these things approach a certain dynamic and this approached dynamic that is what we mean by it but i don't think that infinity would work so to speak that you would know the last digit of pi and that you have a physical process that rests on knowing the last digit of pi yeah that that could be just a peculiar quark of human cognition that we like discrete discrete makes sense to us infinity doesn't so in terms of our intuitions no the issue is that uh everything that we think about uh needs to be expressed in some kind of mental language not not necessarily a natural language but some kind of mathematical language that your neurons can speak that refers to something in the world and what we have discovered is that uh we cannot construct a notion of infinity without running into contradictions which means that such a language is no longer valid and i suspect this is what made photographers so unhappy when somebody came up with the notion of irrational numbers before it was time right there's this miss that he had this person killed when he blapped out the secret that not everything can be expressed as a ratio between two numbers but there are there are numbers between the ratios the world was not ready for this and i think he was right that has confused mathematicians uh very seriously because these numbers are not values they're functions right so you can calculate these functions to a certain degree of approximation but you cannot pretend that pi has actually a value pi is a function that would generally approach this value to some degree but nothing in the world rests on knowing pie uh how much does how important is this distinction between discreet and continuous uh for you to get to the because there's a i mean in discussion of your favorite flavor of the theory of everything there's a few on the table so there's string theory there's a particular there's a loop quantum gravity which focus on one particular unification uh there's there's just a bunch of favorite flavors of different people trying to uh propose a theory of everything uh eric weinstein and a bunch of people throughout history and then of course stephen wolfram who i think is one of the only people doing a discrete no no there's a bunch of physicists who do this right now and okay like um topholy and tomasello and um the digital physics is something that is i think growing in popularity but uh the main reason why this is interesting is because it uh it's important sometimes to settle disagreements i don't think that you need infinities as or at all and you never needed them you can always deal with very large numbers and you can do it with limits right you're fine with doing that you don't need any kind of infinity you can build your computer algebra systems just as well without believing in infinity in the first place you're okay with limits yeah so basically a limit means that something is behaving pretty much the same if you make the number larger right because it's converging to a certain value and at some point the difference becomes measurable and you can no longer measure it and uh in this sense you have things that uh yeah if every ngon which is has enough corners then it's going to behave like a circle at some point right and it's only going to be in some kind of esoteric thing that cannot exist in the physical universe that you would be talking about this perfect circle and now it turns out that it also wouldn't work in mathematics because you cannot construct mathematics that has infinite resolution without running into contradictions so that is itself not that important because we never did that right it's just a thing that some people thought we could and this leads to confusion so for instance roger penrose uses this as an argument to say that there are certain things that mathematicians can do dealing with infinities and by extension our mind can do that computers cannot do yeah he he talks about that there's the human mind can do certain mathematical things that the computer as defined by the universal touring machine cannot yes what so that it has to do with infinity yes it's one of the things so he is basically pointing at the fact that there are things that are possible in the mathematical mind and in pure mathematics that are not possible in uh machines that can be constructed in the physical universe and because he's an honest guy he thinks this means that uh present physics cannot explain operations that happen in our mind do you think he's right and uh so let's let's leave his discussion of consciousness aside for the moment do you think he's right about just what he's basically referring to as intelligence so are is the human mind fundamentally more capable as a thinking machine than a universal touring machine no but so he's suggesting that right so our mind is actually less than a turing machine there can be no touring machine because it's defined as having an infinite tape and we always only have a finite tape but you can perform finally many operations yes it can do the kind of computation the yes the touring machine cannot and that's because he thinks that our minds can do operations that have infinite resolution in some sense and i don't think that's the case our minds are just able to discover these limit operators over too many parts to count what about his idea that consciousness is more uh more than a computation so it's more than something that uh a touring machine can can do so again saying that there's something special about our mind they cannot be replicated in the machine the issue is that i don't even know how to construct a language to express this statement correctly well the the the basic statement is there's a there's a human experience that includes intelligence that includes self-awareness that includes the hard problem of consciousness and the question is can that be fully simulated in the computer in the mathematical model of the computer as we understand it today rajapanos says no so the the uh universal turing machine cannot simulate the universe so the interesting question is uh and you have to ask him this is why not what is this specific thing that cannot be modeled and when i looked at his writings and i haven't read all of it but when i read for instance um the section that he writes in the introduction to and wrote to infinity the thing that he specifically refers to is the way in which human minds deal with infinities and that itself can i think easily be deconstructed a lot of uh people feel that our experience cannot be explained in a mechanical way and therefore it needs to be different and i concur our experience is not mechanical our experience is simulated it exists only in a simulation the only assimilation can be conscious physical systems cannot be conscious because they're only mechanical cells cannot be conscious neurons cannot be conscious brains cannot be conscious people cannot be conscious as far as you if you understand them as physical systems what can be conscious is the story of a system in the world where you write all these things into the story you have experiences for the same reason that a character novel has experiences because it's written into the story and now the system is acting on that story and it's not a story that is written in a natural language it's written in a perceptual language in this multimedia language of the game engine and in there you write in what kind of experience you have and what this means for the behavior of the system for your behavior tendencies for your focus for your attention for your experience of valence and so on and this is being used to inform the behavior of the system in the next step and then the story updates with the reactions of the system and the changes in the world and so on and you live inside of that model you don't live inside of the physical reality and i mean just just to linger on it like you see okay yeah it's in the perceptual language the multimodal perceptual language that's the experience that's what consciousness is within that within that model within that story but do you do you have agency when you play a video game you can turn left and you can turn right in that story so in that dream world how much control do you is there such a thing as you in that story like is it right to say the main character you know everybody's npcs and then there's the main character and you're controlling the main character or is that an illusion is there a main character that you're controlling i'm getting to the point of like the free will point imagine that you are building a robot that plays soccer yeah and you've been to mit computer science you basically know how to do that right and so uh you would say the robot is an agent that solves the control problem how to get the ball into the goal and it needs to perceive the world and the world is disturbing him and trying to do this right so he has to control many variables to make that happen and to project itself and the ball into the future and understand its position on the field relative to the ball and so on in the uh position of its limbs or in in the space around it and so on so it needs to have an adequate model that abstracting reality in a useful way and you could say that this robot does have agency over what it's doing in some sense and the model is going to be a control model and inside of that control model you can possibly get to a point where this thing is sufficiently abstract to discover its own agency our current robots don't do that they don't have a unified model of the universe but there is not a reason why we shouldn't be getting there at some point in the not too distant future and once that happens you will notice that the uh robot tells a story about the robot playing soccer so the robot will experience itself playing soccer in a simulation of the world that it uses to construct a model of the locations of it lacks on and limbs in space on the field with relationship to the ball and it's not going to be at the level of the molecules it will be an abstraction that is exactly at the level that is most suitable for past planning of the movements of the robot right it's going to be a high level abstraction but a very useful one that is as predictive as we can make it and in that side of that story there is a model of the agency of that system so this model can accurately predict that the contents of the model are going to be driving the behavior of the robot in the immediate future but there's the hard problem of consciousness which i would also there's a subjective experience of free will as well that i'm not sure where the robot gets that where that little leap is because for me right now everything i imagine with that robot as it gets more and more and more sophisticated the agency comes from the programmer of the robot still of what was programmed in you could probably do an end-to-end learning system you maybe need to give it a few prayers so you nudge the architecture in the right direction that it converges more quickly but ultimately uh discovering the suitable hyper parameters of the architecture is also only a search process right and as the search process was evolution that has informed our brain architecture so we can converge in a single lifetime on useful interaction with the world and if we define hyper parameters broadly so it's not just this the uh the parameters that control this end-to-end learning system but the entirety of the design of the robot like the there's you have to remove the human completely from the picture and then in order to build the robot you have to create an entire universe because you have to go you can't just shortcut evolution you have to go from the very beginning in order for it to have because i feel like there's always a human pulling the strings um and that makes it seem like the robot is cheating it's getting a shortcut to consciousness when you are looking at the current boston dynamics robots it doesn't look as if there is somebody pulling the strings it doesn't look like cheating anymore okay so let's go there because i gotta talk to you about this so obviously with the case of boston dynamics as you may or may not know it's always either hard coded or remote controlled there's no intelligence i don't know how the current generation of boston dynamics robots works but what i've been told about the previous ones was that it's basically all cybernetic control which means you still have uh feedback mechanisms and so on but it's not uh deep learning for the most part as it's currently done it's for the most part just identifying a control hierarchy that is congruent to the limbs that exist and the parameters that need to be optimized for the movement of these limbs and then there is a convergence progress so it's basically just regression that you would need to control this but again i don't know whether that's true that's just what i've been told about how they work we have to separate several levels of discussions here so the only thing they do is pretty sophisticated control no with no machine learning in order to be to maintain balance or to write itself it's a control problem in terms of using the actuators to when it's pushed or when it steps on a thing that's uneven how to always maintain balance yes and there's a tricky like set of heuristics around that but uh that's the only goal everything you see boston dynamics doing in terms of that to us humans is compelling which is any kind of um higher order movement like turning uh wiggling its butt uh like uh you know uh jumping back on its two feet dancing the dancing is even worse because dancing is hard coded in it's um it's choreographed by humans there's choreography software so like there is no of all that high level movement there's no anything that you can call certainly can't call ai but there's no uh even like basic heuristics it's all hard coded in and yet we humans immediately project agency onto them which is which is fascinating so the gap here is uh it doesn't necessarily have agency well it has a cybernetic control and the cybernetic control means you have a hierarchy of feedback loops that keep the behavior in certain boundaries so the robot doesn't fall over and it's able to perform the movements and the choreography cannot really happen with motion capture because the robot would fall over because the physics of the robot the weight distribution and so on is different from the weight distribution in the human body so if you were using the directly motion captured movements of the human body to project it into this robot it wouldn't work you can do this with the computer animation it will look a little bit off but who cares but if you want to correct for the physics you need to basically tell the robot where it should move its limbs and then the control algorithm is going to approximate a solution that makes it possible within the physics of the robot and you have to find um the basic solution for making that happen and there's probably going to be some regression necessary to get the control architecture to to make these movements but those two layers are separate yes the the the thing the higher level instruction of what how you should move and where you should move is that so i expect that the control level of these robots at some level is dumb this is just the physical control movement the motor architecture but uh it's a relatively smart motor architecture it's just that there is no high level deliberation about what decisions to make necessarily right but see it doesn't feel like um free will no that was not where i was trying to get to i think that in our own uh body we have that too so we have a certain thing that is basically just a cybernetic control architecture that is moving our limbs and deep learning can help in discovering such an architecture if you don't have it in the first place if you already know your hardware you can maybe handcraft it but if you don't know your hardware you can search for such an architecture and this work already existed in the 80s and 90s people were starting to search for control architectures by motor babbling and so on and just use reinforcement learning architectures to discover such a thing and now imagine that you have the cybernetic control architecture already inside of you and you extend this a little bit so you are seeking out food for instance or rest or and so on and you get to have a baby at some point and now you add more and more control layers to this and the system is reverse engineering its own control architecture and builds a high level model to synchronize the pursuit of very different conflicting goals and this is how i think you get to purposes purposes are models of your goals the goals may be intrinsic as the result of the different set point violations that you have hunger and thirst for very different things and rest and pain avoidance and so on and you put all these things together and eventually you need to come up with a strategy to synchronize them all and you don't need uh just to do this alone by yourself because we are state building organisms we cannot function in the isolation the way that homo sapiens is set up so our own behavior only makes sense when you zoom out very far into a society or even into ecosystemic intelligence on the planet and our place in it so the individual behavior only makes sense in these larger contexts and we have a number of priors built into us so we are behaving as if we are acting on these high level goals pretty much right from the start and eventually in the course of our life we can reverse engineer the goals that we are acting on what actually are our higher level purposes and the more we understand that the more our behavior makes sense but this is all at this point complex stories within stories that are driving our behavior yeah i just don't know how big of a leap it is to start uh create a system that's able to tell stories within stories like how big of a leap that is from where currently boston dynamics is or any robot that's operating in the physical space that and that leap might be big if it requires to solve the hard problem of consciousness which is telling a hell of a good story i suspect that um consciousness itself is relatively simple what's hard is perception and the interface between perception and reasoning that's for instance the idea of the consciousness prior that would be built into such a system by uh joshua bangio and uh what he describes and i think that's accurate is that our own model of the world can be described through something like an energy function the energy function is modeling the contradictions that exist within the model at any given point and you try to minimize these contradictions the tangents in the model and to do this you need to sometimes test things you need to conditionally disambiguate figure and ground you need to just distinguish whether this is true or that is true and so on eventually you get to an interpretation but you will need to manually depress a few points in your model to let it snap into a state that makes sense and this function that tries to get the biggest dip in the energy function in your model according to joshua bangio is related to consciousness it's a low dimensional discrete function that tries to maximize this dip in the energy function i yeah i think i would need to dig into details because i think the way he uses the word consciousness is more akin to like self-awareness like modeling yourself within the world as opposed to the subjective experience the hard problem no it's not even the self is in the world the self is the agent and you don't need to be aware of yourself in order to be conscious the self is just a particular content that you can have but you don't have to have right you can be conscious in uh for instance a dream at night or during a meditation state but you don't have a self right where you're just aware of the fact that you are aware and what we mean by consciousness and the colloquial sense is largely this reflexive self-awareness that we become aware of the fact that you're paying attention that we are the thing that pays attention we are the thing that pays attention right i don't see where uh the uh awareness that we're aware the the heart problem doesn't feel like it's solved i mean they they're they're it it's called a hard problem for a reason because it seems like there needs to be a major leap yeah i think the major leap is to understand how it is possible that a machine can dream that the physical system is able to create a representation that the physical system is acting on and that is spun force and so on but once you accept the fact that you are not in physics but that you exist inside of the story i think the mystery disappears everything is possible in a statement exists inside the story okay so your consciousness is being written into the story the fact that you experience things is written to the story you ask yourself is this real what i'm seeing and your brain writes into the story yes it's real so what about the perception of consciousness so to me you look conscious so um the illusion of consciousness the demonstration of consciousness i ask for the the legged robot how do we make this legged robot conscious so there's two things and maybe you can tell me if they're neighboring ideas one is actually make it conscious and the other is make it appear conscious to others are those related uh let's ask from the other direction what would it take to make you not conscious so when you are thinking about how you perceive the world can you decide to switch from looking at qualia to looking at representational states and it turns out you can yeah there is a particular way in which you can look at the world and recognize its machine nature including your own and in that state you don't have that conscious experience in this way anymore it becomes apparent as a representation everything becomes opaque and i think this thing that you recognize everything as a representation this is typically what we mean with enlightenment states and yeah you can't have a motivational level but it you can also do this on the experiential level and the perceptual level see but then i can come back to a conscious state okay i particularly i'm referring to the social aspect that the demonstration of consciousness is a really nice thing at a party when you're trying to meet a new person it's it's a nice thing to to to know that they're conscious and they can um how i don't know how fundamental consciousness is in human interaction but it seems like to be at least uh an important part and i i asked that in the same kind of way for robots you know in order to create a rich compelling human robot interaction it feels like there needs to be elements of consciousness within that interaction my cat is obviously conscious and so my cat can do this party trick she also knows that i am conscious be able to have feedback about the fact that we are both acting on models of our own awareness the question is how hard is it for uh the robot artificially created robot to achieve cat level and party tricks yes so the issue for me is currently not so much on how to build a system that creates a story about a robot that lives in the world but to make an adequate representation of the world and the model model that you and me have is a unified one it's verb one where you basically make sense of everything that you can perceive every feature in the world that enters your perception can be relationally mapped to a unified model of everything and we don't have an ai that is able to construct such a unified model yet so you need that unified model to do the party trick yes i think that uh you it doesn't make sense if this thing is conscious but not in the same universe as you because you could not relate to each other so what's the process would you say of engineering consciousness in the machine like what are the ideas here so uh you probably want to have some kind of perceptual system this perceptual system is a processing agent that is able to track sensory data and predict the next frame and the sensory data from the previous frames of the sensory data in the current state of the system so the current state of the system is perception instrumental to predicting what happens next and this means you build lots and lots of functions that take all the blips that you feel on your skin and that you see on your retina or that you hear and puts them into a set of relationships that allows you to predict what kind of sensory data what kind of sensor of blips your vector of blips you're going to perceive in the next frame right this is tuned and it's constantly tuned until it gets as accurate as it can you build a very accurate prediction mechanism that is step one of the perception so first you predict then you perceive and see the error in your prediction and you have to do two things to make that happen one is you have to build a network of relationships that are constraints that take all the variants in the world to put each of the variances into a variable variable that is connected with relationships to other variables and these relationships are computable functions that constrain each other so when you see a nose that points a certain direction in space you have a constraint that says there should be a face nearby that has the same direction right and if that is not the case you have some kind of contradiction that you need to resolve because it's probably not a nose what you're looking at it just looks like one so you have to reinterpret the data and until you get to a point where your model converges and this process of making the sensory data fit into your model structure is what prg calls the assimilation and accommodation is the change of the models where you change your model such a way that you can assimilate everything so you're you're talking about building a hell of an awesome perception system that's able to do prediction and perception and correct and improvement wait just uh if you had to wait there's more yes there's more so the first thing that we want to do is we want to minimize the contradictions in the model yes and of course it's very easy to make a model in which you minimize the contradictions just by allowing that it can be in many many possible states right so if you increase degrees of freedom you will have fewer contradictions but you also want to reduce the degrees of freedom because degrees of freedom mean uncertainty you want your model to reduce uncertainty as much as possible but reducing uncertainty is expensive so you have to have a trade-off between minimizing contradictions and reducing uncertainty and you have only finite amount of compute and experimental time and effort available to reduce uncertainty in the world so you need to assign value to what you observe so you need some kind of motivational system that is estimating what you should be looking at and what you should be thinking about it how you should be applying your resources to model what that is right so you need to have something like uh convergence links that tell you how to get from the present state of the model to the next one you need to have these compatibility links that tell you which constraints exist and which constraint violations exist and you need to have some kind of motivational system that tells you what to pay attention to so now we have a second agent next to the perceptual age we have a motivational agent this is a cybernetic system that is modeling what the system needs what's important for the system and that interacts with the perceptual system to maximize the expected reward and you're saying a motivational system is some kind of like what is it a higher level narrative over some lower level no it's just your brainstem stuff the limbic system stuff that tells you okay now you should get something to eat because i've just measured your dual blood sugar like motivational system like the lower levels yes like hungry yes but there's basically a physiological needs and some cognitive needs and some social needs and they all interact and they're all implemented at different parts in your nervous system as the motivational system but they're basically cybernetic feedback loops it's not that complicated it's just a lot of code and so you now have a motivational agent that makes your robot go for the ball or that makes your worm go to eat food and so on and you have the perceptual system that lets it predict that environment so it's able to solve that control problem to some degree and now what we learned is that it's very hard to build a machine learning system that looks at all the data simultaneously to see what kind of relationships could exist between them so you need to selectively model the world you need to figure out where can i make the biggest difference if i would put the following things together sometimes you find a gradient for that right when you have a gradient you don't need to remember where you came from you just follow the gradient until it doesn't get any better but if you have a word where the problems are discontinuous and the search spaces are discontinuous you need to retain memory of what you explored and you need to construct a plan of what to explore next and this thing that means that you have next to this perceptual construction system and the motivational cybernetics an agent that is paying attention to what it should select at any given moment to maximize reward and this scanning system this attention agent is required for consciousness and consciousness it is its control model so it's the index memories that this thing retains when it manipulates the perceptual representations to maximize the value and minimize the conflicts and it to increase coherence so the purpose of consciousness is to create coherence in your perception representations remove conflicts predict the future construct counterfactual representations so you can coordinate your actions and so on and in order to do this it needs to form memories these memories are partial binding states of the working memory contents that are being revisited later on to backtrack to undo certain states to look for alternatives and these index memories that you can recall that is what you perceive as your stream of consciousness and being able to recall these memories this is what makes you conscious if you could not remember what you paid attention to you wouldn't be conscious so consciousness is the index in the memory database okay uh but let me sneak up to the questions of consciousness a little further so we usually relate suffering to consciousness so the capacity to suffer i think to me that's a really strong sign of consciousness is a thing that can suffer how how is that useful suffering and like in your model what you just described which is indexing of memories and what is the coherence with the perception uh with this predictive thing that's going on the perception how how does suffering relate to any of that you know the higher level suffering that humans do basically pain is a reinforcement signal it pain is a signal that one part of your brain sends to another part of your brain or an abstract sense part of your mind sends to another part of the mind to regulate its behavior to tell it the behavior that you're currently exhibiting should be improved and this is the signal that i tell you uh to move away from what you're currently doing and push into a different direction so pain gives you a part of you an impulse to do something differently but sometimes this doesn't work because for the training part of your brain is talking to the wrong region or because it has the wrong model of the relationships in the world maybe you're mismodeling yourself or you're mismodding the relationship of yourself to the world or you're mismodeling the dynamics of the world so you're trying to improve something that cannot be improved by generating more pain but the system doesn't have any alternative so the uh it doesn't get better what do you do if something doesn't get better and you want it to get better you increase the strength of the signal and then the signal becomes chronic when it becomes permanent without a change inside this is what we call suffering and the purpose of consciousness is to deal with contradictions with things that cannot be resolved the purpose of consciousness i think is similar to a conductor in an orchestra when everything works well the orchestra doesn't need much of a conductor as long as it's coherent but when there is a lack of coherence or something is consistently producing disharmony and mismatches then the conductor becomes alert and interacts with it so suffering attracts the activity of our consciousness and the purpose of that is ideally that we bring new layers online new layers of modeling that are able to create a model of the dysregulation so we can deal with it and this means that we typically get higher level consciousness so to speak right we get some consciousness above our pay grade maybe if we have some suffering early in our life most of the interesting people had trauma early on in their childhood and trauma means that you are suffering an injury for which the system is not prepared which it cannot deal with which it cannot insulate itself from so something breaks and this means that the behavior of the system is permanently disturbed in the way that some mismatch exists now in the regulation that just by following your impulses by following the pain in the direction which it hurts the situation doesn't improve but get worse and so what needs to happen is that you grow up yeah that and that's part that has grown up is able to deal with the part that is stuck in this earlier phase yeah so it leads to growth adding extra layers to okay to your cognition uh let me ask you then because i got to stick on suffering uh the ethics of the whole thing so not our consciousness but the consciousness of others you've uh uh tweeted one of my biggest fears is that insects could be conscious the amount of suffering on earth would be unthinkable so when we think of other conscious beings is suffering a property of consciousness that we're most concerned about so i'm still thinking about robots how to make sense of other non-human things that appear to have the depth of experience that humans have and to me that means consciousness and the darkest side of that which is suffering the capacity to suffer and so i start thinking how much responsibility do we have for those other conscious beings that's where the um the definition of consciousness becomes most urgent like having to come up with a definition of consciousness becomes most urgent is who should we and should we not be torturing there's no general answer to this was genghis khan doing anything wrong it depends right on how you look at it well he he drew he drew a line somewhere where this is us and that's them it's the circle of empathy it's like these we don't have to use the word consciousness but these are the things that matter to me if they suffer or not and these are the things that don't matter yeah but when one of his commanders failed him he uh broke his spine and let him die in a horrible way and uh so in some sense i think he was indifferent to suffering or he was not different in the sense that he didn't see it as useful if he inflicted suffering but he did not see it as something that had to be avoided that was not the goal the question was how can i use suffering and the infliction of suffering to reach my goals from his perspective i i see so like different societies throughout history put different value on different individuals different psyches but also even the uh the objective of avoiding suffering like some society is probably i mean this is where like religious belief really helps that that afterlife that doesn't matter that you suffer or die what matters is you suffer honorably right so that you enter the afterlife it seems to be superstitious to me basically beliefs that assert things uh for which no evidence exists are incompatible with sound epistemology and i don't think that religion has to be superstitious otherwise it should be condemned okay in all cases you're somebody who's saying we live in a dream world we have zero evidence for anything so that's not the case there are limits to what languages can be constructed mathematics brings solid evidence for its own structure and once we have some idea of what languages exist and how a system can learn and what learning itself is in the first place and so on we can begin to realize that our intuitions that we are able to learn about the regularities of the world and minimize the appraisal and understand the nature of our own agency to some degree of abstraction that's not an illusion so useful approximation just because we live in a dream world doesn't mean mathematics can't uh give us a consistent glimpse of uh physical of objective reality we can basically distinguish useful encodings from useless encodings and when we apply our truth seeking to the world we know we usually cannot find out whether a certain thing is true what we typically do is we take the state vector of the universe separate it into separate objects that interact with each other so interfaces and this distinction that we are making is not completely arbitrary it's done to optimize the compression that we can apply to our models of the universe so we can predict what's happening with our limited resources in this sense is not arbitrary but the separation of the world into objects that are somehow discrete and interacting with each other is not the true reality right the boundaries between the objects are projected into the world not arbitrarily projected but still it's only an approximation of what's actually the case and we sometimes notice that we run into contradictions when we try to understand high-level things like economic aspects of the world and so on or political aspects or psychological aspects where we make simplifications and the objects that we are using to separate the world are just one of many possible projections of what's going on and so it's not in this postmodernist sense completely arbitrary and you're free to pick what you want or dismiss what you don't like because it's all stories no that's not true you have to show for every model of how well it predicts the world so the confidence that you should have in the entities of your models should correspond to the evidence that you have can i ask you in a small tangent uh to talk about your um favorite set of ideas and people which is post modernism what is what is post-modernism how would you define it and why to you is it not a useful framework of thought uh post-modernism is something that i'm really not an expert on and uh post-modernism is a a set of philosophical ideas that it's difficult to lamp together that is characterized by some useful thinkers some of them post structuralist and so on and i'm mostly not interested in it because i i think that it's not leading me anywhere that i find particularly useful it's mostly i think born out of the inside that the ontologies that we impose on the world are not literally true and that we can often get to a different interpretation by the world by using a different ontology that is different separation of the world into interacting objects but the idea that this makes the world a set of stories that are arbitrary i think is wrong and the people that are engaging in this type of philosophy are working in in an area that i largely don't find productive there's nothing useful coming out of this so this idea that truth is relative is not something that has in some sense informed physics or theory of relativity and there is no feedback between those there is no meaningful influence of this type of philosophy on the sciences or in engineering or in politics but there is a very strong information on of this on ideology because it basically has become an ideology that is justifying itself by the notion that truth is a relative concept and it's not being used in such a way that the the philosophers that or sociologists that take up these ideas say oh uh i should doubt my own ideas because maybe my separation of the world into objects is not completely valid and they should maybe use a different one and be open to a pluralism of ideas but it's mostly exists to dismiss the ideas of other people it becomes yeah it becomes a political weapon of sorts to achieve power basically this uh there's nothing wrong i think with uh developing a philosophy around this but to develop norms around the idea that truth is something that is completely negotiable is incompatible with the scientific project and i think if the uh if the academia has no defense against the ideological parts of the postmodernist movement it's doomed right you have to acknowledge the ideological part of any movement actually uh including post-modernism well the question is what an ideology is and to me an ideology is basically a viral memplex that is changing your mind in such a way that reality gets warped it gets warped in such a way that you're being cut off from the rest of human thought space and you cannot consider things outside of the range of ideas of your own ideology it was as possibly true right so i mean there's certain properties to an ideology that make it harmful one of them is that like dogmatism of just certainty dogged certainty in in that you're right you have the truth and nobody else but what is creating the certainty it's very interesting to look at the type of model that is being produced is it basically just to draw a strong prior and you tell people oh this idea that you consider to be very true the evidence for this is actually just much weaker than you thought and look here at some studies no this is not how it works it's usually normative which means some thoughts are unthinkable because they would change your identity into something that is no longer acceptable and this cuts you off from considering an alternative and many uh de facto religions use this trick to lock people into a certain mode of thought this removes agency over your own thoughts it's very ugly to me it's basically not just a process of domestication but it's actually an intellectual castration that happens it's an inability to think creatively and to bring forth new thoughts can i ask you about substances chemical substances that affect the video game the dream world so psychedelics that increasingly have been getting a lot of research done on them so in general psychedelics psilocybin mdma but also really interesting one the big one which is dmt what and where are the places that these substances take the mind that is operating in the dream world do you have an interesting sense how this throws a wrinkle into the prediction model is it just some weird little quirk or is there is there some fundamental expansion of the mind going on i suspect that a way to look at psychedelics is that they induce particular types of lucid dreaming states so it's a state in which certain connections are being severed in your mind when no longer active your mind basically gets free to move in a certain direction because some inhibition some particular inhibition doesn't work anymore and as a result you might stop having yourself or you might stop perceiving the world as three-dimensional and you can explore that state and i suppose that for every state that can be induced with psychedelics there are people that are naturally in that state so sometimes psychedelics that shift you through a range of possible mental states and they can also shift you out of the range of permissible mental states that is where you can make predictive models of reality and what i observe in people that use psychedelics a lot is that they tend to be over fitting overfitting means that you are using more bits for modeling the dynamics of a function than you should and so you can fit your curve to extremely detailed things in the past but this model is no longer predictive for the future what is it about psychedelics that forces that i thought it would be the opposite i thought i thought uh that it's a it's a good mechanism for uh uh for generalization for regularization so it feels like psychedelics expansion of the mind like taking you outside of like forcing your model to be uh non-predictive is a good thing meaning like uh it's almost like okay what i would uh say psychedelics are akin to is traveling to a totally different environment like going if you've never been to like india or something like that from the united states very different set of people different culture different food different roads and values and all those kinds of things yeah so psychedelics can for instance teleport people into a universe that is uh hyperbolic which means that if you imagine a room that you are in you can turn around 360 degrees and you didn't go full circle you need to go 720 degrees to go full circle exactly so the things that people learn in that state cannot be easily transferred in this universe that we are in it could be that if they're able to abstract and understand what happened to them that they understand that some part of their spatial cognition has been desynchronized and has found a different synchronization and this different synchronization happens to be a hyperbolic one right so you learn something interesting about your brain it's difficult to understand what exactly happened but we get a pretty good idea once we understand how the brain is representing geometry yeah but doesn't give you a fresh perspective on the physical reality who's making that sound is inside my head or is it external well there is no sound outside of your mind but uh it's making sense often on my nine physics uh yeah in the physical reality there's uh there's sound waves traveling through air okay that's our model of what happened tomorrow what happened right uh that doesn't uh don't psychedelics give you a fresh perspective on this physical reality like uh not this physical reality but this this more um [Applause] what do you call the dream world that's mapped directly to purpose of dreaming at night i think is yeah theater augmentation well exactly so that's very different that's a very similar just change parameters uh about the things that you have learned and uh for instance when you are young you have seen things from certain perspectives but not from others so your brain is generating new perspectives of objects that you already know which means they can learn to recognize them later from different perspectives and i suspect that's the reason why many of us remember to have flying dreams as children because it's just different perspectives of the world that we already know and that it it starts to generate these different perspective changes and then it fluidly turns this into a flying dream to make sense of what's happening right so you fill in the gaps and suddenly you see yourself flying and similar things can happen with semantic relationships so it's not just spatial relationships but it can also be the relationships between ideas that are being changed and it seems that the mechanisms that make that happen during dreaming um are interacting with these same receptors that are being stimulated by psychedelics so uh i suspect that there is a thing that i haven't read really about the way in which dreams are induced in the brain it's not just that the activity of the brain gets tuned down because you are somehow your eyes are closed and you no longer get enough data from your eyes but there is a particular type of neurotransmitter that is saturating your brain during these phases during the rm phases and you produce controlled hallucinations and psychedelics are linking into these mechanisms i suspect so there's an another trickier form of data augmentation yes but uh it's also data augmentation that can happen outside of the specification that your brain is tuned to so basically people are overclocking their brains and that that produces states that are subjectively extremely interesting yeah i just but from the outset very suspicious so i think i'm over applying the metaphor of a neural network in my own mind which i just think that doesn't lead to overfitting right but but you were just sort of anecdotally saying my experiences with people that have no psychedelics or that that kind of quality i think it typically happens so if you look at people like uh timothy leary and he has written beautiful manifestos about the effect of lsd on people he genuinely believed he writes in these manifestos that in the future science and art will only be done on psychedelics because it's so much more efficient and so much better and he gave lsd to children in this community of a few thousand people that he had near san francisco and basically he was losing touch with reality he did not understand the effects that the things that he was doing would have on the reception of psychedelics by society because he was unable to think critically about what happened what happened was that he got in an euphoric state that euphoric state happened because he was overfitting he was taking this sense of euphoria and translating it into a model of actual success in the world right he was feeling better limitations had disappeared that he exp appearance to be existing but he didn't get superpowers i understand what you mean by overfitting now there's a lot of interpretation to the term overfitting in this case but i i got you so he was getting he was getting uh positive rewards from a lot of actions this year but not just this so if you take for instance john lilly who um was studying dolphin languages and aliens and so on yeah a lot of people that use psychedelics became very loopy and the typical thing that you notice when people are in psychedelics is that they are in a state where they feel that everything can be explained now everything is clear everything is obvious yeah and uh sometimes they have indeed discovered a useful connection but not always very often these connections are over interpretations i wonder you know there's a question of uh correlation versus causation and also i wonder if it's the psychedelics or if it's more the social like being the outsider uh and having a strong community of outside and being having a leadership position in an outside occult-like community that could have a much stronger effect of overfitting than do psychedelics themselves the actual substances because it's a counter culture thing so it could be that as opposed to the actual substance if you're a boring person who wears a suit and tie and works at a bank and takes psychedelics that could be a very different effect of psychedelics on on your mind it i'm just sort of raising the point that the people you referenced are already weirdos i'm not sure exactly oh no not necessarily a lot of the people that uh tell me that they use psychedelics in a useful way started out as squares and were liberating themselves because they were stuck they were basically stuck in local optimum of their own self model of their relationship to the world and suddenly they had data augmentation they basically saw as an experience the space of possibilities they experienced what it would be like to be another person yeah and they took uh important lessons from that experience back home yeah i mean uh i love the the the metaphor of data augmentation because that's uh been the the primary driver of self-supervised learning in the vision computer vision domain is data augmentation so it's funny to think of data augment like like chemically induced data augmentation in the human mind there's also a very interesting effect that i uh noticed i've i know uh several people who are sphere to me that lsd has cured their migraines so severe cluster eight headaches or migraines that didn't respond to standard medication that disappeared after a single dose and i don't recommend anybody doing this especially not in the us where it's illegal uh and there are no studies on this for that reason but uh it seems that uh anecdotally that it basically can reset the serotonergic system so it's basically pushing them outside of their normal boundaries and as a result it needs to find a new equilibrium and in some people that equilibrium is better but it also follows that in other people it might be worse so if you have a brain that is already teetering on the boundary to psychosis it can be permanently pushed over that boundary well that's why you have to do good science which they're starting to do on all these different substances of how well it actually works for the different conditions like mdma seems to help with ptsd uh same with psilocybin that you know you need to do good science meaning large studies of large and yeah so based on the existing studies with mdma it seems that if you look at rick doblin's work and what he has published about this and talks about mdma seems to be a psychologically relatively safe drug but it's physiologically not very safe that is uh there is a neurotoxicity if you would use two large dose and if you uh combine this with alcohol which a lot of kids do in uh party settings during raves and so on it's very hepatotoxic so basically you can kill your liver and this means that it's probably something that is best and most productively used in clinical setting by people who really know what they're doing and i suspect that's also true for the other psychedelics that is uh while the other second eggs are probably not as toxic as say alcohol the effects on niseki can be much more profound and lasting yeah well as far as i know psilocybin so mushrooms magic mushrooms as far as i know in terms of the studies they're running i think have no over like they're allowed to do what they're calling heroic doses so that one does not have a toxicity so they could do like huge doses in a clinical setting when they're doing study on psilocybin which is kind of fun yeah it seems that most of the psychedelics work in extremely small doses which means that the effect on the rest of the body is relatively low and mdma is probably the exception maybe ketamine can be dangerous and larger doses because it can depress breathing and so on but the lsd and psilocybin work in very very small doses at least the active part of them of um lsd is only the active part and the but the effect that you it can have on your mental wiring can be very dangerous i think let's talk about ai a little bit what are your thoughts about gpt3 and language models trained with self-supervised learning it came out quite a bit ago but i wanted to get your thoughts on it yeah in the 90s i was in new zealand and i had an amazing professor ian witton who realized there was ward in class and put me in his lab and he gave me the task to discover grammatical structure in an unknown language and the unknown language that i picked was english because it was the easiest one to find corpus four construct one and he gave me the largest computer at the whole university it had two gigabytes of ram which was amazing and i wrote everything in c with some in-memory compression to do statistics over the language and first would create a dictionary of all the words which basically tokenizes everything and compresses things so they don't need to store the whole world but just a code for every word and then i was um taking this all apart in sentences and i was trying to find all the relationships between all the words in the sentences and do statistics over them and that proved to be impossible because the complexity is just too large so if you want to discover the relationship between an article and a noun and there are three adjectives in between you cannot do engram statistics and look at all the possibilities that can exist at least not with the resources that we had back then so i realized i need to make some statistics over what i need to make statistics over so i wrote something that was pretty much a hack that did this for um at least first order relationships and i came up with some kind of mutual information graph that was indeed discovering uh something that looks exactly like the grammatical structure of the sentence just by trying to encode the sentence in such a way that the words would be written in the optimal order inside of the model and what i also found is that if we would be able to increase the resolution of that and not just use this model to reproduce grammatically correct sentences we would also be able to correct stylistically correct sentences by just having more bits in these relationships and if we wanted to have meaning we would have to go much higher order and i didn't know how to make higher order models back then without spending way more years in research on how to make the statistics over what we need to make statistics over and this thing that we cannot look at the relationships between all the bits in your input is being solved in different domains in different ways so in computer graphics the computer vision standard methods for many years now is convolutional neural networks convolutional neural networks are hierarchies of filters that exploit the fact that neighboring pixels in images are usually semantically related and distance pixels and images are usually not semantically related so you can just by grouping the pixels that are next to each other hierarchically together reconstruct the shape of objects and this is an important prior that we built into these models so they can converge quickly but this doesn't work in language for the reason that adjacent words are often but not always related and distant words are sometimes related while the words in between are not right so how can you learn the topology of language and i think for for this reason that this difficulty existed the transformer was invented in natural language processing not in vision and what the transformer is doing it's a hierarchy of layers where every layer learns what to pay attention to in the given context in the previous layer so what to make the statistics over and the context is significantly larger than the adjacent word yes so the context that this um that gpt3 has been using the transformer itself is from 2017 and it's uh wasn't using that large of a context openai has basically scaled up this idea as far as they could at the time and the context is about 2048 symbols tokens in the language these symbols are not characters but they take the words and project them into a vector space where words that are statistically co-occurring a lot are neighbors already so it's already a simplification of the problem a little bit and so every word is basically a set of coordinates in a high dimensional space and then they use some kind of trick to also encode the order of the words in a sentence or in the not just sentence but 2048 tokens is about couple pages of text or two and a half pages of text and so they managed to do pretty exhaustive statistics over the potential relationships between two pages of text which is tremendous right i was just using a single sentence back then and i was only looking for first order relationships and there were really looking for much much higher level relationships and what they discover after they fed this with an enormous amount of treatment in their pretty much the written internet or a subset of it that had some quality but substantial portion of the common draw that uh they're not only able to reproduce style but they're also able to reproduce some pretty detailed semantics like being able to um add three digit numbers and multiply two digit numbers or to translate between programming languages and things like that so the results that gbt3 got i think were amazing by the way um i actually didn't check carefully it's funny you just mentioned how you coupled semantics to the multiplication is it able to do some basic math on two uh uh two digit numbers yes okay interesting i thought i thought there's a lot of failure cases yeah but basically it fails if you take larger digit numbers so four digit numbers and so on uh makes carrying mistakes and so on and if you take large larger numbers you don't get useful results at all and this could be an issue of the training set down.net many examples of successful long-form edition and standard uh human written text and humans aren't very good at doing three-digit numbers either yeah they're not you're not writing a lot about it yeah and the other thing is that the loss function that is being used is only minimizing surprises so it's predicting what comes next in the typical text it's not trying to go for causal closure first as we do yeah and but the fact that that kind of prediction works to generate text that's semantically rich and consistent is interesting yeah so yeah so it's amazing that it's able to uh generate symmetrically consistent text it's not consistent so the problem is that it loses coherence at some point but it's also i think not correct to say that gptc3 is unable to deal with semantics at all because you ask it to perform certain transformations in text and it performs this transformation and text and the kind of additions that it's able to perform are transformations and text right and there are proper semantics involved you can also do more there was a paper that was generating lots and lots of mathematically correct text and was feeding this into a transformer and as a result it was able to learn how to do differentiation integration in race that according to the authors mathematica could not to which some of the people in mathematica responded that uh they were not using the mathematica in the right way and so on i have not really followed the this resolution of this conflict this this part as a small tangent i really don't like in machine learning papers which they often do um anecdotal evidence they'll find like one example in some kind of specific use of mathematica and demonstrate look here's they'll show successes and failures but they won't have a very clear representation of how many cases this actually represents yes but i think as a first paper this is a pretty good start and yeah so uh the whole message is that the authors could get better results from this in their experiments than they could get from the vein which they were using computer ultra systems which means uh that was not nothing yeah and uh it's able to perform substantially better uh than gptsv can based on a much larger amount of training data using the same underlying algorithm well let me let me ask again so i'm using your tweets as if this is like plato right as if this is well thought out novels that you've written uh you tweeted gpt4 is listening to us now um this is one way of asking what are the limitations of gpt-3 when it scales so what do you think will be the capabilities of gpt-4 gpt-5 and so on what are the limits of this approach so uh obviously when we are writing things right now uh everything that we are writing now is going to be training data for the next generation of machine learning models so yes of course tbt4 is listening to us and i think the tweet is already a little bit older and the we now have wu dao and we have a number of other systems that basically are placeholders for gpt4 don't know what open ais plans are in the card i read that tweet in several ways so one is obviously everything you put on the internet is used as training data but in the second way i read it is in a uh we talked about agency i read it as almost like gpt4 is intelligent enough to be choosing to listen so not only like did a programmer tell it to collect this data and use it for training i almost saw the humorous angle which is like it has achieved agi kind of thing well the thing is um could we be already be living in gpt5 [Laughter] so gpt4 is listening and dpt5 actually constructed the entirety of the reality we in some sense the what everybody is trying to do right now in ai is to extend the transformer to be able to deal with video and uh there are very promising extensions where there is a work by google that is called perceiver and that is overcoming some of the limitations of the transformer by letting it learn the topology of the different modalities separately and by training it to find better input features so these basically feature abstractions that are being used by this um successor or to gpt3 are chosen such a way that it's able to deal with video input and there is more to be done so i one of the limitations of gpt three is that it's uh amnesiac so it forgets everything beyond the two pages that it currently reads also during generation not just during learning do you think that's fixable within the space of deep learning can you just make a bigger bigger bigger input no uh i don't think that our own uh working memory is infinitely large it's probably also just a few thousand bits but uh what you can do is you can structure this working memory so instead of just force feeding this thing a certain thing that it has to focus on and it's not allowed to focus on anything else with its network you allow it to construct its own working memory as we do right when we are reading a book it's not that we are focusing our attention in such a way that we can only remember the current page we will also try to remember other pages and try to undo what we learned from them or modify what we learned from them we might get up and take another book from the shelf we might go out and ask somebody and we can edit our working memory in any way that is useful to put a context together allows us to draw the right inferences and to learn the right things so this ability to perform experiments on the world based on an attempt to become fully coherent and to achieve causal closure to achieve a certain aesthetic of your modeling that is something that eventually needs to be done and at the moment we are skirting this in some sense by building systems that are larger and faster so they can use dramatically larger resources and human beings can do and much more training data to get to models that in some sense are already very superhuman and in other ways are laughingly incoherent so do you think uh sort of making um the systems like what would you say multi-resolutional so like some uh some of the language models are focused on two pages some are focused on uh two books some are focused on two years of reading some are focused on a lifetime like so it's like stacks of it's the gpt3s all the way down you want to have gaps in between them so it's not necessarily two years there's no gaps it stinks out of two years or out of twenty years or two thousand years or two billion years yeah where you are just selecting those bits that are predicted to be the most useful ones to understand what you're currently doing and this prediction itself requires a very complicated model that's the actual model that you need to be making it's not just that you are trying to understand the relationships between things but what you need to make relationships or discover relationships over i wonder what that thing looks like with the architecture for that for the thing that's able to have that kind of model that i think it needs more degrees of freedom than the current models have so it starts out with the fact that you possibly don't just want to have a feed forward model but you want it to be fully recurrent and to make it fully recurrent you probably need to loop it back into itself and allow it to skip connections once you do this right when you are predicting the next frame and your internal next frame in every moment and you are able to skip connection it means that signals can travel from the output of the uh network into the middle of the network faster than the inputs do you think it can still be differentiable do you think it still could be a neural network sometimes it can and sometimes it cannot so it it can still be a neural network but not a fully differentiable one and when you want to deal with non-differentiable ones you need to have an attention system that is discrete and dual dimensional and can perform grammatical operations you need to be able to perform program synthesis you need to be able to backtrack in this operations that you perform on this thing this thing needs a model of what it's currently doing and i think this is exactly the purpose of our own consciousness yeah the program things that triculo networks so let me ask you it's not quite program synthesis but uh the application of these language models to generation to program synthesis but generation of programs so if you look at github open pilot which is based on openai's codex i don't know if you got a chance to look at it but it's the system that's able to generate code once you uh prompt it with what is it like the header of a function with some comments and it seems to do an incredibly good job or not a perfect job is very important but an incredibly good job of generating functions what do you make of that are you is this exciting or is this just a party trick a demo or is this revolutionary i haven't worked with it yet so it's difficult for me to judge it but i would not be surprised if it turns out to be revolutionary that's because the majority of programming tasks that are being done in the industry right now are not creative yeah people are writing code that other people have written or they're putting things together from code fragments that others have had and a lot of the work that program has done practice is to figure out how to overcome the gaps in their current knowledge and the things that people have already done how to copy and paste from stuck over that's right and so of course we can automate that yeah to uh make it much faster to copy and paste from stack overflow yes but it's not just copying and pasting it's also basically learning which parts you need to modify to make them fit together yeah uh like literally sometimes as simple as just changing the variable names so it fits into the rest of your code yes but this requires that you understand the semantics of what you're doing to some degree yeah and you can automate some of those things yes the the thing that makes people nervous of course is that a little bit wrong in a program can have a dramatic effect on the actual final operation of that program so it's one little error which in in the space of language it doesn't really matter but in the space of programs can matter a lot yes but this is already what is happening when humans program code yeah this is so we have a technology to deal with this somehow it becomes scarier when you know that a program generated code that's running a nuclear power plant it becomes scarier you know humans have errors too exactly but it's scarier when a program is doing it because why why i mean there's a there's a fear that a program like a program may not be as good as humans to know when stuff is important to not mess up like there's a misalignment of priorities of values that's potential that maybe that's the source of the worry i mean okay if i give you code generated by uh github open pilot and code generated by a human and say here use use one of these which which how do you select today and in the next 10 years which code to use wouldn't you still be comfortable with the human at the moment when you go to stanford to get an mri they will write a bill to the insurance over twenty thousand dollars and of this maybe half of that gets paid by the insurance and the quarter gets paid by you yeah and the mri cost them six hundred dollars to make maybe probably less and what are the values of the person that writes the software and deploys this process it's very difficult for me to say whether i trust people i think that what happens there is a mixture of proper anglo-saxon protestant values where somebody is trying to serve an abstract greater whole and organized crime well that's a very harsh you're you're um i think that's a harsh view of humanity there's a lot of bad people whether incompetent or just malevolent in this world yes but it feels like the more malevolent you so the more damage you do to the world uh the more resistance you have in your own human like but cannot explain with malevolence or stupidity what can be explained by just people acting on their incentives right so what happens in stanford is not that somebody is evil it's just that they do what they're being paid for no and this is that's not evil that's i i tend to so no i see that as malevolence i see uh as i uh even like being a good german as i told you offline is some it's not it's not absolute malevolence but it's a small amount it's cowardice i mean when you see there's something wrong with the world um it's either incompetence that you're not able to see it or it's cowardice that you're not able to stand up not it's not necessarily in a big way but in a small way so i i do think that is in a bit of malevolence i'm not sure the example you're describing is the question is what is it that you are aiming for and if you don't believe in the future if you for instance think that the dollar is going to crash by what you try to save dollars if you uh don't think that humanity will be around in 100 years from now because uh global warming will wipe out civilization why would you need to act as if it were right so the question is is there an overarching aesthetics that is projecting you and the world into the future which i think is the basic idea of religion that you understand the interactions that we have with each other as some kind of civilization level agent that is projecting itself into the future if you if you don't have that shared purpose uh right what is there to be ethical for so i think when we talk about essex and ai we need to go beyond the insane bias discussions and so on where people are just measuring the distance between a statistic to their uh preferred um current world model but the optimism i was a little confused by the previous things just to clarify it's there is a kind of underlying morality to having an optimism that human civilization will persist for longer than a hundred years like that like uh i think a lot of people believe that it's it's a good thing for us to keep living yeah of course and thriving morality itself is not an end to itself it's instrumental to people living in 100 years from now right or 500 years from now right so uh it's only justifiable if you actually think that it will lead to people uh or increase the probability of people being around in that time frame and a lot of people don't actually believe that at least not actively but i believe what exactly so most people don't believe that they can afford to act on such a model basically what happens in in the u.s is i think that the healthcare system is for a lot of people no longer sustainable which means that if they need the help of the healthcare system they're often not able to afford it yeah and when they cannot help it they are often going bankrupt it's i think the leading course cause of personal bankruptcy in the us is the healthcare system yeah and uh that would not be necessary it's not because people are consuming more and more uh medical services and are achieving a much much longer life as a result that's not actually the story that is happening because you can compare to other countries and life expectancy in the u.s is currently not increasing and it's not as high as in all the other industrialized countries so some industrialized countries are doing better with a much cheaper healthcare system and what you can see is for instance administrative bloat the healthcare system has maybe to some degree deliberately set up as a job placement program to allow people to continue living in middle class existence despite not having useful and use case in productivity so they are being paid to push paper around and the number of administrators in the healthcare system has been increasing much faster than the number of practitioners and this is something that you have to pay for right and also the revenues that are being generated in the healthcare system are relatively large and somebody has to pay for them and the result why they are so large is because market mechanisms are not working right the fda is largely not protecting people from malpractice of healthcare providers the fda is protecting healthcare providers from competition right okay so this is a thing that is has to do with values and this is not because people are malicious on all levels it's because they are not incentivized to act on a greater whole on this idea that you treat somebody who comes to you as a patient like you would treat a family member yeah but yeah but we're trying i mean you're highlighting a lot of the flaws of the different institutions the systems we're operating under but i think there's a continuum throughout history mechanism design of trying to design incentives in such a way that these systems behave better and better and better i mean it's a very difficult thing to operate a society of hundreds of millions of people effectively with yes so do we live in a society that is ever correcting is this uh right do we observe that our models of what we are doing are predictive of the future and when they are not we improve them our laws adjudicated with clauses that you put into every law what is meant to be achieved by that law and the law will be automatically repealed if it's not achieving that right if you are optimizing your own laws if you're writing your own source code you probably make an estimate of what is the thing that's currently wrong in my life what is it that i should change about my own policies what is the expected outcome and if that outcome does manifest i will change the policy back right or i will change it to something different are we doing this on a societal level i think so i think i think it's easy to sort of highlight the i think we're doing it in the way that um like i operate my current life i didn't sleep much last night uh you would say that lex the way you need to operate your life is you need to always get sleep the fact that you didn't sleep last night is is totally the wrong way to operate in your life like you should have gotten all your done in time and gotten to sleep because sleep is very important for health and you're highlighting look this person is not sleeping look the the medical the health care system is operating poor but the point is that we just it seems like this is the way especially in the capital society we operate we keep running into trouble and last minute we we try to get our our way out through innovation and it seems to work you have a lot of people that ultimately are trying to build a better world and and get uh urgency about them when it's the problem becomes more and more imminent and that's the way this operates but if you look at the the the history the the long arc of history it seems like that operating on deadlines produces progress and builds better and better systems you probably agree with me that the us should have engaged in mass production in january 2020 and that we should have shut down the airports early on and that we should have made it mandatory that the people that work in nursery homes are living on campus rather than living at home and then uh coming in and infecting uh people in the nursing homes that had no immune response to covet and uh that is something that was i think visible back then the correct decisions haven't been made we would have the same situation again how do we know that these wrong decisions are not being made again have the people that made the decisions to not protect the nursing homes been punished has have the people that made the wrong decisions with respect to testing that prevented the development of testing by startup companies and the importing of tests from countries that already had them have these people been held responsible well first of all so what do you what do you want to like put before the firing squad i think they are no just make sure that this doesn't happen again no but you see but it's not that yes they're being held responsible by many voices by people being frustrated there's new leaders being born now they're going to see rise to the top in 10 years this moves slower than there's obviously a lot of uh older incompetence and bureaucracy and these systems move slowly they move you know like science one death at a time so like yes i think the the pain has been felt in the previous year is reverberating throughout the world maybe i'm getting old i suspect that every generation in the us after the war has lost the plot even more i don't see this development the war world war ii yeah so basically there was a time when uh we were modernist and in this modernist time the u.s felt actively threatened by the things that happened in the world the us was worried about possibility of failure and this imminence of possible failure led to decisions where there was a time when the government would listen to physicists about how to do things yeah and the physicists were actually concerned about what the government should be doing so they would be writing letters to the government and so for instance the decision for the manhattan project was something that was driven in the conversation between physicists and the government i don't think such a discussion would take place today i i disagree i think if the virus was much deadlier we would see a very different response i think the virus was not sufficiently deadly and instead because it wasn't very deadly what happened is uh the the current system started to politicize it the mask this is what i realized with masks early on they were not very quickly became not as a solution but they became a thing that politicians used to divide the country so that that same thing is happening with vaccines same thing so like nobody's really people weren't talking about solutions to this problem because i don't think the problem was bad enough when you talk about the world the war i think what i think our lives are too comfortable i think in uh in the developed world things are too good and we have not faced severe dangers one the dangers the severe dangers existential threats are faced that's when we step up on a small scale and a large scale now i don't i that's sort of my argument here but i did think the virus is i was hoping that it was actually sufficiently uh dangerous for us to step up because especially in the early days it was unclear it still is unclear because of mutations how bad it might be right and so i thought we would step up and even so the masks point is is uh it's a tricky one because to me the manufacturer of masks isn't isn't even the problem i'm still to this day and i was involved with with a bunch of this work have not seen good signs done on whether masks work or not like there still has not been a large-scale study to me that should be there should be large-scale studies and every possible solution like aggressive in the same way that the vaccine development was aggressive there should be masks which test what kind of tests work really well uh what kind of uh like even the question of how the virus spreads there should be aggressive studies on that to understand i'm still as far as far as i know there's still a lot of uncertainty about that nobody wants to see this as an engineering problem that needs to be solved it's it's uh that i was surprised about but i find that our views are largely convergent but not completely so i agree with the thing that uh because our society in some sense perceives itself as too big to fail right and uh the virus did not alert people to the fact that we are facing possible failure uh that basically put us into the post-modernist mode and i don't mean in the philosophical sense but in a societal sense the difference between a post-modern society and the modern society is that the modernist society has to deal with the ground truth and the postmodernist society has to deal with appearances politics becomes a performance and the performance is done for an audience and the organized audience is the media and the media evaluates itself via other media right so you have an audience of critics that evaluate themselves and i don't think it's so much the failure of the politicians because to get in power and to stay in power you need to be able to deal with the published opinion well i think it goes in cycles because the what's going to happen is all of the small business owners all the people who truly are suffering and will suffer more because the effects of the closure of the economy and the lack of solutions to the virus they're going to uprise and hopefully i mean this is this is where charismatic leaders can get the the world in trouble but hopefully we'll elect great leaders that will break through this post-modernist idea of uh of uh the media and the perception and the drama on twitter and all that kind of stuff but you know this can go either way yeah when the weimar republic was unable to deal with the economic crisis that germany was facing there was an option to go back but there were people which thought let's get back to a constitutional monarchy and let's get this this to work because democracy doesn't work and eventually uh there was no way back where people decided there was no way back they needed to go forward and the only options for going forward was to become a stalinist communist basically an option to completely expropriate the factories and so on and nationalize them and to reorganize germany and communist terms and ally itself with stalin and fascism and both options were obviously very bad and the one that the germans picked led to a catastrophe that was that devastated europe and i'm not sure if the us has an immune response against that i think that the far right is currently very weak in the u.s but this can easily change do you think uh from a historical perspective hitler could have been stopped from within germany or or from outside or this well depends on who you want to focus whether you want to focus on stalin or hitler but he feels like hitler was the one as a political movement that could have been stopped i think that uh the point was that a lot of people wanted hitler so he got support from a lot of quarters it was a number of industrialists who supported him because they thought that the democracy is obviously not working and unstable and you need a strong man and he was willing to play that part there were also people in the us who thought that hitler would stop uh stalin and would act as a balrack against bolshevism which he probably would have done right but uh at which cost and then many of the things that he was going to do like the holocaust was something where people thought this is rhetoric he's not actually going to do this right especially many of the jews themselves which were humanist and for them this was outside of the scope that was thinkable right i mean i i i wonder if hitler is uniquely i i want to carefully use this term but uniquely evil so if hitler was never born if somebody else would come in this place so like just thinking about the progress of history how important are those singular figures that that lead to mass destruction and cruelty because my senses hitler was unique the it wasn't just about the environment and the context that gave him like like another person would not come in his place to do as destructive of the things that he did there was a combination of uh charisma of madness of psychopathy of just uh ego all those things which are very unlikely to come together in one person in the right time it also depends on the context of the country that you're operating in if you uh tell the germans that they have a historical destiny uh in this romantic country the effect is probably different than it is in other countries but uh the uh stalin has killed a few more people than hitler did and if you look at the probability that you're survived under stalin uh right uh hitler killed people if uh they if he thought they were not worth living or if they were harmful to his uh racist uh project right he basically felt that the jews would be too cosmopolitan and would not be willing to uh participate in the racist redefinition of society and the value of society and an isno state in this way that as he wanted it to have it so they he saw them as a harmful danger especially since they played such an important role in the economy and culture of germany and uh it so we had basically had some radical but rational reason to murder them and stalin just killed everyone basically the stalinist purchase were such a random thing where he said that there's a certain possibility that this particular part of the population has a number of german collaborators or something and we just kill them all yeah right well if you look at what mao did the number of uh people that were killed absolute in absolute numbers were much higher under mao that they were under stalin so it's super hard to say the the the other thing is that you look at jinga's khan and so on how many people he killed where do you see there are a number of of things that happen in human history that actually really put a substantial dent in the existing population or napoleon and it's it's very difficult to eventually measure it because what's happening is basically evolution uh on a human scale where uh one monkey figures out a way to become viral and uh is using this viral technology to change the patterns of society at the very very large scale and what we find so abhorrent about these changes is the complexity that is being destroyed by this that it's basically like a big fire that burns out a lot of the existing culture and structure that existed before yeah and it all just starts with one monkey one one cares mata gabe and there's a bunch of them throughout history yeah but it's in a given environment it's basically similar to wildfires in california right the temperature is rising there is less rain falling and then certainly a single spark can have an effect that in other times would be contained okay speaking of which i i love when we went to hitler and stalin from 20 30 minutes ago uh gpt 3 generating doing program synthesis the argument was about morality of ai versus human so um and specifically in the context of writing programs specifically in the context of programs that can be destructive so running nuclear power plants or autonomous weapons systems for example and i think your inclination was to say that it's not so obvious that ai would be less moral than humans or less effective at making a a world that would make humans happy so i i'm not talking about self-directed systems that are making their own goals at a global scale if you just talk about the deployment of technological systems that are able to see order and patterns and use this as control models to act on the goals that we give them then if we have the correct incentives to set the correct incentives for these systems i'm quite optimistic so but so humans versus ai let me give you an example autonomous weapon system let's say there's a city somewhere in the middle east that has a number of terrorists and the question is what's currently done with with drone technologies you have information about the location of a particular terrorist and you have a targeted attack you have a bombing of that particular building and that's all directed by humans at the high level strategy and also at the deployment of individual bombs and missiles like that the actual everything is done by human except the the final targeting and the the like the country so like with spot similar thing like control the flight okay what if you give ai control and saying um write a program that says here's the best information i have available about the location of these five terrorists here's the city make sure it's cons all the bombing you do is constrained to the city make sure it's precision based but you take care of it so you do one level of abstraction out and saying take care of the terrorists in the city which are you more comfortable with the humans or the javascript gpt-3 generated code that's doing the deployment i mean that's this is the the kind of question i'm asking is the kind of bugs that we see in human nature are they better or worse than the kind of bugs we see in ai they're different bugs there is an issue that if people are creating an in imperfect automation of a process that normally requires a moral judgment and this mobile judgment is the reason why it cannot be automated often is not because the computation is too expensive but because the model that you give the ai is not an adequate model of the dynamics of the world because the ai does not understand the context that it's operating in in the right way and this is something that already happens with excel right you don't need to have an ai system to do this if you have an automated process in place where humans decide using automated criteria whom to kill when and whom to target when which already happens right and you have no way to get off the kill list once that happens once you have been targeted according to some automatic criterion by people right in a bureaucracy that that is the issue the issue is it's not the ai it's the automation so there's something about right it's automation but there's something about the there's a certain level of abstraction where you give control to ai to do the automation there's a scale that can be achieved that it feels like the scale of bug and scale mistake and scale of destruction that can be achieved of the kind that humans cannot achieve so ai is much more able to destroy an entire country accidentally versus humans it feels like the more civilians die as they react or suffer as the consequences of your decisions the more weight there is on the human mind to make that decision and so like it becomes more and more unlikely to make that decision for humans for ai it feels like it's harder to encode that kind of weight in a way the ai that we're currently building is automating statistics right intelligence is the ability to make models so you can act on them and ai is the tool to make better models so in principle if you're using ai wisely you're able to prevent more harm and i think that the main issue is is not on the side of the ai it's on the side of the human command hierarchy that is using technology irresponsibly so the question is how hard is it uh to encode to to pro to properly encode the right incentives into the ai so for instance there's this idea what happens if we let our airplanes be flown with ai systems and the neural network is a black box and so on and it turns out our neural networks are actually not black boxes anymore there are uh function approximators using linear algebra and uh there are performing things that we can understand but we can also instead of letting the neural network fly the airplane use the neural network to generate approval of the correct program with the degree of accuracy of the proof that a human could not achieve right so we can use our ai by combining different technologies to build systems that are much more reliable than the systems that a human being could create and so in in this sense i would say that if you use an early stage of technology to save labor and don't employ employ competent people but just to hack something together because you can that is very dangerous and if people are acting under these incentives that they get away with delivering shoddy work more cheaply using ai is less human oversight than before that's very dangerous the thing is though ai is still going to be unreliable perhaps less so than humans but it'll be unreliable in novel ways and yeah but this is an empirical question and it's something that we can figure out and work with so the issue is do we trust the systems the social systems that we have in place and the social systems that we can build and maintain that they're able to use ai responsibly if they can then ai is good news if they cannot then it's going to make the existing problems worse well and also who creates the ai who controls it who makes money from it because it's ultimately humans and then you start talking about how much you trust the humans so the question is what does who mean i don't think that we have identity per se i think that these the story of a human being is somewhat random what happens is more or less that everybody is acting on their local incentives but they're perceived to be their incentives and the question is what are the incentives that the one that is pressing the button is operating under yeah it's nice for those incentives to be transparent so for example i'll give i'll give you examples there seems to be a significant uh distrust of um a tech like entrepreneurs in the tech space or or people that run for example social media companies like mark zuckerberg there's not a complete transparency of incentives under which that particular human being operates the tren you know we can listen to the words he says or what the marketing team says for a company but we don't know and that's that's becomes a problem when that the algorithms and the systems created by uh him and other people in that company start having more and more impact on society and that it starts you know if if the incentives were somehow the definition and the explainability of the incentives was um decentralized such that no nobody can manipulate it no propaganda uh type manipulation of like how these systems actually operate could be done then yes it'd be not it i think i think ai could achieve um much fairer much more effective uh sort of uh like solutions to to to difficult ethical problems but when there's like humans in the loop manipulating the the the dissemination the communication of how the system actually works that feels like you can run to a lot of trouble and that's why there's currently a lot of distrust for for people at the heads of companies that have increasingly powerful ai systems i suspect what happened traditionally in the u.s was that since our decision making is much more central a decentralized than in authoritarian state right people are making decisions autonomously at many many levels in a society what happened that was uh we created coherence and cohesion in society by controlling what people thought and what information they had right the media synchronized public opinion and social media have disrupted this it's not i think so much russian influence or something it's everybody's influence it's that a random person can come up with a conspiracy theory and disrupt what people think and if that conspiracy theory is more compelling or more attractive than the standardized public conspiracy theory that we give people as a default then it might get more traction right you suddenly have the situation that the single individual somewhere on a farm in texas has more listeners than cnn which particular farmer you're referring to in texas [Laughter] i probably know yes i had dinner with him a couple times okay right this is an interesting situation because you cannot get to be an anchor and cnn if you don't go through a complete uh complicated gatekeeping process and suddenly you have random people without that gatekeeping process uh just optimizing for attention not necessarily with a lot of responsibility for the long-term effects of projecting these theories into the public and now uh there is a push of making social media more like traditional media which means that the opinion that is being projected in social media is more limited to an acceptable range with the goal of getting society into safe waters and increase the stability and cohesion of society again which i think is a laudable goal but of course it also is an opportunity to seize the means of indoctrination and the incentives that people are under when they do this are in such a way that the ai ethics that we would need becomes very often something like ai politics which is basically partisan and ideological and this means that whatever one side says another side is going to be disagreeing with right in the same way as when you turn masks or the vaccine into a political issue if you say that it is politically virtuous to get vaccinated it will mean that the people that don't like you will not want to get vaccinated right and as soon as you have this partisan discourse it's going to be very hard to make the right decisions because the incentives get to be the wrong ones ai ethics needs to be super boring it needs to be done by people who do statistics all the time and have extremely boring long-winded discussions that most people cannot follow because they are too complicated but that are that serious these people need to be able to to be better at statistics than the leading machine learning researchers and at the moment the ia ethics the debate is the one but you don't have any barrier to entry right everybody who has a strong opinion and is able to signal that opinion in the right strong words and to me that is a very frustrating thing because the field is so crucially important to us so crucially important but uh the only qualification you currently need is to be outraged by the injustice in the world it's more complicated right everybody seems to be outraged but so let's just say that the incentives are not always the right ones so basically i i suspect that a lot of people that enter this debate don't have a vision for what society should be looking like in a way that is non-violent where we preserve liberal democracy where we make sure that we all get along and we are around in a few hundred years from now preferably with a comfortable technological civilization around us i i generally have a very foggy view of that world but i tend to try to follow and i think society should in some degree follow the gradient of love increasing the amount of love in the world and whenever i see different policies or algorithms or ideas that are not doing so obviously that that's the ones that kind of resist so the thing that terrifies me about this notion is i think that german fascism was driven by love it was just a very selective love it was a love that now you're just manipulating i mean that's i it's you have to be very careful uh you're talking to the wrong person in this way about love so let's talk about what love is and i think that love is the discovery of shared purpose it's the recognition of the sacred and the other and this enables non-transactional interactions but the the size of the other that you include needs to be maximized so it's it's basically appreciation like deep appreciation of uh the world around you fully like um including the people that are very different than you people that disagree with you completely including people including living creatures outside of just people include including ideas and it's like appreciation of the full mess of it and also it has to do with like empathy which is coupled with a lack of confidence uncertainty about of your own rightness it's like an open a radical open-mindedness to the way forward i agree with every part of what you said and now if you scale it up what you recognize is that life is in some sense the service to a next level agency to the highest level agency that you can recognize it could be for instance life on earth or beyond that really you could say intelligent complexity in the universe that you try to maximize in a certain way but when you think it's true it basically means a certain aesthetic and there is not one possible aesthetic there are many possible aesthetics and once you project an aesthetic into the future you can see that there are some which uh defect from it which are in conflict with it that are corrupt that are evil right you and me would probably agree that hitler was evil because uh the aesthetic of the world that he wanted is in conflict with the aesthetic of the world that you and me have in mind yeah and so they think that he destroyed we we want to keep them in the world there there's a kind of uh there's kind of ways to deal i mean hitler is an easier case but perhaps he wasn't so easy in the 30s right to understand who is hitler and who is not no there was no consensus that the aesthetics that he had in mind were unacceptable yeah i mean it's difficult a lot love is complicated because you can't just be uh so open-minded that you let eva walk into the door but you can't be so self-self-assured that um you you can always identify evil perfectly because that's what leads to nazi germany having a certainty of what isn't wasn't evil like always drawing lines of good versus evil there seems to be a there has to be a dance between um like hard stances extending up against what is wrong and at the same time empathy and open-mindedness of towards not knowing what is right and wrong and like a dance between those i found that when i watch the miyazaki movies that there is nobody who captures my spirituality as well as he does it's very interesting and just wishes right there is something going on in his movies that is very interesting so for instance mononoker is discussing not only an answer to disney's simplistic notion of mowgli the jungle boy was raised by wolves and as soon as he sees people realizes that he's one of them and the way in which the moral life and nature is simplified and romanticized and turned into kitsch right it's disgusting in the disney movie and he answers to this you see he's replaced by mononoker this wolf girl who was raised by wolves and who is fierce and dangerous and who cannot be socialized because he cannot be tamed it cannot be part of human society and you see human society it's something that is very very complicated you see people extracting resources and destroying nature but the purpose is not to be evil uh but to be able to have a life that is free from for instance oppression and violence and uh to curb death and disease and you basically see this conflict which cannot be resolved in a certain way you see this moment when nature is turned into a garden and it loses most of what it actually is and humans no longer submitting to life and death and nature and to these questions there is no easy answer so it just turns it into something that is being observed as a journey that happens and that happens with a certain degree of inevitability and the nice thing about all his movies is there's a certain main character and it's the same in all movies it's this little girl that is basically heidi and i suspect that happened because he when he did um field work for working on the heidi movies back then the heidi animations before he did his own movies he traveled to switzerland and south uh western southeastern europe and adriatic and so on and got an idea about a certain aesthetic and a certain way of life that informed is his future thinking and heidi has a very interesting relationship to herself and to the world there is nothing that she takes for herself she is in a way fearless because she is committed to a service to a greater whole basically she is completely committed to serving god it is not an institutionalized god it has nothing to do with the roman catholic church or uh something like this but in some sense heidi is an embodiment of the spirit of europea protestantism it's this idea of a being that is completely perfect and pure and it's not a feminist vision because she is not a girl boss or something like this she is the justification for the men in the audience to protect her to build the civilization around her that makes her possible right so she is not uh just the sacrifice of jesus who is innocent and therefore nailed to the cross she is not being sacrificed he is being protected by everybody around her who recognizes that she is sacred and there are enough around her to to see that right so this is a very interesting perspective there is a certain notion of innocence and this notion of innocence is not universal it's not in all cultures right hitler was an innocent his idea of it of germany was not that there is a innocence that is being protected there was a predator that was going to triumph yeah and it's also something that is not at the core of every religion there are many religions which don't care about innocence they might care about um increasing the the status of something right and that's a very interesting notion that is quite unique and not claiming it's the optimal one it's just a particular kind of aesthetic which i think makes miyazaki into the most relevant protestant philosopher today and you're saying in terms of of all the of all the ways that society can operate perhaps the preservation of innocence um might be one might be one of the best no it's just uh my aesthetic so you're aesthetic it's it's a particular way in which i feel that i relate to the world that is natural to my own socialization and maybe it's not an accident that i uh have cultural roots in europe yeah in a particular world and yeah so maybe it's a natural convergence point and it's not something that you will find in all other times in history i'd like to ask you about solzhenitsyn and our individual role as ants in this very large society so he says that uh some version of the line between good and evil runs to the heart of every man do you think all of us are capable of good and evil like what's our role in this play uh in this game we're all playing is all of us capable to play any role like is there an ultimate responsibility to um you mentioned maintaining innocence or whatever the whatever the highest ideal for a society you want are all of us capable of living up to that and that's our responsibility or or is there significant limitations to what we're able to do in terms of good and evil so there is a certain way if you are not terrible if you are committed to some kind of civilizational agency a next level agent that you are serving some kind of transcendent principle uh in the eyes of that transcendental principle you are able to discern good from evil otherwise you cannot otherwise you have just individual aesthetics right the cat that is torturing a mouse is not evil because the cat does not envision or no part of the world of the cat is envisioning a world where there is no violence and nobody is suffering yeah right if you have an aesthetic where you want to protect innocence then torturing somebody needlessly is evil but only then no but within i guess the question is within the aesthetic like within your sense of what is good and evil are we still it seems like we're still able to uh commit evil yes so basically if you are committing to this next level agent you are not necessarily are this next level agent right you are part of it you have a relationship to it like a cell does to its organism it's hyper organism and it only exists to the degree that it's being implemented by you and others and that means that you're not completely fully serving it you have freedom in what you decide whether you are acting on your impulses and local incentives on your feral impulses so to speak or whether you're committing to it and what you perceive then is a tension between what you would be doing with respect to the thing that you uh recognize as the sacred if you do and what you're actually doing and this is the line between good and evil right where you see oh i'm here acting on my local incentives or impulses and here i'm acting on what i consider to be sacred and there's a tension between those and this is the line between good and evil that might run through your heart and if you don't have that if you don't have this relationship to a transcendental agent you could call this relationship to the next level agent soul right it's not a thing it's not an immortal thing that is intrinsically valuable it's a certain kind of relationship that you project to understand what's happening somebody is serving this transcendental sacredness or they're not if you don't have your soul you cannot be evil you're just a complex natural phenomenon so if you look at life like starting today or starting tomorrow when we leave here today there's a bunch of trajectories that you can take through life may be countless do you think some of these trajectories in your own conception of yourself some of those trajectories are the ideal life a life that uh if you were to be the hero of your life story you would want to be like is there some joshua bach that you're striving to be like this is the question i ask myself as an individual trying to make a better world in the best way that i could conceive of what is my responsibility there and how much am i responsible for the failure to do so because i'm a i'm i'm lazy and incompetent too often in my own perception in my own worldview i'm not very important so it's uh i don't have place for me as a hero in my own world i'm trying to do the best that i can which is often not very good and so it's not important for me to to have status or to be seen in a particular way it's helpful if others can see me or a few people can see me that can be my friends no sorry i i want to clarify that hero i didn't mean status or perception or uh like uh some kind of marketing thing but more in private in the quiet of your own mind is there the kind of man you want to be and would consider it a failure if you don't become that that's what i'm meant by hero yeah not really i don't perceive myself as having such an identity and it's also sometimes frustrating but it's basically a lack of of having this notion of father that i need to be emulating it's interesting it means the leaf floating down the river i worry that sometimes it's more like being the river i'm just a fat frog sitting in a leaf [Music] on a dirty muddy lake waiting for a princess to kiss me or the other way i forgot which way it goes somebody kisses somebody i can ask you i don't know if you know who michael malus is but um in terms of constructing systems of incentives it's it's uh interesting to ask i don't think i've talked to you about this before malice espouses anarchism so he sees all government as fundamentally getting in the way or even being destructive to collaborations between human beings thriving what do you think what's the role of government in a society that um that thrives is anarchism at all compelling to you as a system so like not just small government but no government at all yeah i don't see how this would work the government is an agent that imposes an offset on your reward function on your payout matrix so it your behavior becomes compatible with the common good so the argument there is that you can have collectives like governing organizations but not government like where you're born in a particular set of land and therefore you must follow this uh rule or else you're forced by what they call violence because there's there's an implied violence here so what government the key the key aspect of government is is it uh protects you from the rest of the world with an army and with with police right so there's this it has a monopoly on violence it's the only one that's able to do violence there are many forms of government not all governments do that right but we uh find that in the in successful countries the government has a monopoly on violence and uh that means that you cannot get ahead by starting your own army because the government will come down on you and destroy you if you try to do that and in countries where you can build your own army and get away with it some people will do it right in these countries is what we call failed countries in a way and you if you don't want to have violence the point is not to appeal to the moral intentions of people because some people will use strategies if they get ahead with them that feel a particular kind of ecological niche so you need to destroy that ecological niche and if a affective government has a monopoly on violence it can create a world where nobody is able to use violence and get ahead right so you want to use that monopoly on violence not to exert violence but to make violence impossible to raise the cost of violence so people need to get ahead with nonviolent means so the idea is that you might be able to achieve that in an anarchist state with companies so with the with the forces of capitalism is create security companies where the one that's most ethically sound rises to the top basically it would be much better representative of the people because there is uh less sort of stickiness to the the big military force sticking around even though it's long overlived outlet so you have groups of militants that are hopefully officially organized because otherwise they're going to lose against the other groups of militants and they are coordinating themselves with the rest of society until they are having a monopoly on violence how is that different from a government i listen i'm doing right so it's basically converging to the same thing so i think it always i did in my as i was trying to argue with malice i feel like it always converges towards government at scale but i think the idea is you can have a lot of collectives that are you basically never let anything scale too big so one of the problems with governments is it gets too big in terms of like the the the size of the group over which it has control my sense is that would happen anyway in in this so successful company like amazon or facebook i mean it starts forming a monopoly over over entire populations not over just a hundred hundreds of millions but billions of people so i don't know but there is something about the abuses of power the government can have when it has a monopoly on violence right and so that's that's a tension there but so the question is how can you set the incentives for government correctly and this mostly applies at the highest levels of government and we because we haven't found a way to set them correctly we made the highest levels of government relatively weak and this is i think part of the reason why we had difficulty to coordinate the pandemic response and china didn't have that much difficulty and there is of course a much higher risk of the abuse of power that exists in china because the power is largely unchecked and that's basically what happens in the next generation for instance imagine that we would agree that the current government of china is largely correct and but never limit and maybe we don't agree on this but if if he did how can we make sure that this stays like this and if you don't have checks and balances uh division of power it's hard to achieve we don't have a solution for that problem but the abolishment of government basically would remove the control structure from a cybernetic perspective there is an optimal point in the system that the regulation should be happening right that you can measure the current incentives and the regulator would be properly incentivized to make the right decisions and change the payout metrics of everything below it in such a way that the local prisoner's dilemmas get resolved right you cannot resolve the prisoner's dilemma without some kind of eternal control that emulates an infinite game in a way yeah i mean there there's a sense in which it seems like the reason government the parts of government that don't work well currently is because there's not good um mechanisms for through which to interact for the citizenry to interact with government is basically it it hasn't caught up in terms of technology and i think once you integrate some of the digital revolution of of being able to have a lot of access to data be able to vote on on different ideas at a local level at all levels at the at the optimal level like you're saying that can resolve the the prisoner dilemmas and to integrate ai to help you out automate things that are like um that don't require the human ingenuity i feel like that's that's where government could operate that well and can also break apart the inefficient bureaucracies if needed there'll be a strong incentive to to um to be efficient and successful so out human history we see an evolution and evolutionary competition of modes of government and of individual governments is in these modes and every nation state in some sense is some kind of organism that has found different solutions for the problem of government and you could look at all these different models and the different scales at which it exists as empirical attempts to validate the idea of how to build a better government and i suspect that the idea of anarchism similar to the idea of communism is the result of being disenchanted with the ugliness of the real existing solutions and the attempt to get to an utopia and i suspect that communism originally was not a utopia i think that's in the same way as original christianity it had a particular kind of vision and this vision is a society a mode of organization within the society in which humans can coexist at scale without coercion the same way as we do in a healthy family right in a good family you don't terrorize each other into compliance but you understand what everybody needs and what everybody can is able to contribute and what the intended future of the whole thing is and you everybody coordinates their behavior in the right way and informs each other about how to do this and all the interactions that happen are instrumental to making that happen right could this happen at scale and i think this is the idea of communism communism is opposed to the idea that we need economic terror or other forms of terror to make that happen but in practice what happened is that the proto-communist countries the real existing socialism replaced a part of the economic terror of his moral terror right so we were told to do the right thing for moral reasons and of course it didn't really work and the economy eventually collapsed and the immoral terror had actual real cause right people were in prison because they were morally non-compliant and that is the other thing is that the idea of communism became a utopia so it basically was projected to the afterlife we were told in in my childhood that communism was a hypothetical society to which we were in a permanent revolution that justified everything was presently wrong with society morally but it was something that our grandchildren probably would not ever see because it was too ideal and too far in the future to make it happen right now and people were just not there yet morally and the same thing happened with christianity right this notion of heaven was mythologized and projected into an afterlife and i think this was just the idea of god's kingdom of this world in which we instantiate the next level transcendental agent in the perfect form so everything goes smoothly and without violence and without conflict and without this human messiness on this economic messiness and the terror and coercion that existed in the present societies and the idea of that the humans can exist at scale in a harmonious way non-coercively is untested right a lot of people tested it but didn't get it to work so far and the utopia is a world in where you get all the good things without any of the bad things and you are i think very susceptible to believe in utopia when you are very young and don't understand that everything has to happen in causal patterns that there is always feedback loops that ultimately are closed there's nothing that just happens because it's good or bad good or bad don't exist in isolation they only exist with respect to larger systems so can you intuit why utopias fail as uh as systems so like having a utopia that's out there beyond the horizon is it because then um so it's not only because it's impossible to achieve utopias but it's because what certain humans certain small number of humans start to uh um sort of greedily attain power and money and control and influence as they become uh uh as they see the power in uh using this uh idea of a utopia it's probably like saying why is my garden not perfect it's because some evil weeds are overgrowing it and they always do yeah right but this is not how it works a good garden is a system that is in balance and requires minimal interactions by the gardener and so you need to create a system that is designed to self stabilize and the design of social systems requires not just the implementation of the desired functionality but the next level design but also in biological systems you need to create a system that wants to converge to the intended function right so instead of just creating an institution like the fda that is performing a particular kind of rule a role in society you need to make sure that the fda is actually driven by a system that wants to do this optimally that is incentivize the root optimally and then makes the performance that is actually enacted in every generation instrumental to that thing that actual goal right and that is much harder to design and to achieve so you have to design a system where i mean listen communism also was quote unquote incentivized to be a feedback loop system that achieves that utopia it's just it wasn't working given human naturally the incentives were not correct how do you uh incentivize people when they are getting coal off the ground to work as hard as possible because it's a terrible job and it's very bad for your health and right how do you do this and you can uh give them prices and medals and status to some degree right there's only so much status to give for that and most people will not fall for this yeah right or you can pay them or uh and you probably have to pay them an asymmetric way because if you pay everybody the same and they uh you nationalize the coal mines eventually people will figure out that they can game the system yes so you're you're describing capitalism so capitalism is the present solution to the system and what he also noticed that i think that marx was correct in saying that capitalism is prone to crisis that capitalism is a system that in its dynamics is not convergent but divergent it's not a stable system and that eventually it produces an enormous potential for productivity but it also is systematically misallocating resources so a lot of people cannot participate in the production and consumption anymore and this is what we observe we observe that the middle class in the us is tiny it's uh a lot of people think that they're middle class but if you are still flying economy you're not middle class right every class is a magnitude smaller class and right classes really like airline classes [Laughter] a lot of people are economy have we really business class and very few are first class and some are bridgette i mean some i i understand i i i think there's uh yeah maybe some people probably i would push back against that definition of the middle class it does feel like the middle class is pretty large but yes there's a discrepancy in terms of wealth um so there's a big gap in terms of the productivity that our society could have yeah there is no reason for anybody to fly economy right we would be able to let everybody travel in style well but also some people like to be frugal even when they're billionaires okay so like that let's take that into yes but uh i mean you probably don't need to be a traveling lavish but you also don't need to be tortured right there is a difference between frugal and uh subjecting yourself to torture listen i love economy i don't understand why you're comparing a flying economy to torture i don't although the the the fight here there's two crying babies next to me so that but that has nothing to do with the comments it has to do with crying babies they're very cute though so yeah i have two kids and sometimes i have to go back to visit the grandparents and back means going from the west coast to germany and it's a long flight is it true that so when you're a father you you grow immune to the crying and all that kind of stuff like you know because like me just not having kids it can be other people's kids can be quite annoying when they're crying and screaming and all that kind of stuff uh when you have children and you're wired up in the default natural way you're lucky in this regard you fall in love with them yeah and this falling in love with them means that you basically start to see the world through their eyes and you understand that in a given situation they cannot do anything but um being expressing despair and so it becomes more differentiated i noticed that for instance my son is typically acting on a pure experience of what things are like right now and has to do this right now and you have this a small child that is um if when he was a baby and so on where he was just immediately expressing what he felt and if you cannot regulate this from the outside well there's no point to be upset about it right it's like dealing with weather or something like this you all have to get through it and it's not easy for him either but uh if you also have a daughter maybe she is planning for that maybe she understands that you know she's sitting in the car behind you and she's screaming at the top of her lungs and you're almost doing an accident yeah and you really don't know what to do what should i have done to make you stop screaming you could have given me candy i think that's like a cat versus dog discussion i love it so but because you said the f like a fundamental aspect of that is is love that makes it all like worth it what in this monkey riding an elephant in a dream world what role does love play in the human condition i think that love is the facilitator of non-transsexual interaction you are um observing your own purposes some of these purposes go beyond your ego they go beyond the particular organism that you are and your local interests that's what you mean by non-transactional yes so basically when you are acting in transactional way it means that you are respecting something in return for you from the one that you're interacting with right you are interacting with a random stranger you buy something from them on ebay you expect a fair value for the money that you send them and vice versa because you don't know that person you don't have any kind of relationship to them but when you know this person a little bit better and you know the situation that they're in you understand what they try to achieve in their life and you approve because you you realize that they're in some sense serving the same human sacredness as you are and they need to think that you have maybe you give it to them as a present but the i mean the feeling itself of joy is a kind of benefit is the kind of transaction like yes but the joy is not the point the joy is the signal that you get it's the reinforcement signal that your brain sends to you because you are acting on the incentives of the agent that you're part of we are meant to be part of something larger right this is the way in which we out-competed other hominins [Music] uh take their neanderthals uh yeah right and also other humans uh right there was a population bottleneck for a human society that leads to an extreme lack of genetic diversity among humans if you look at bushmen in the kalahari that basically tribes that are not that far distant to each other have more genetic diversity that exists between europeans and chinese and it's because basically the out of africa population at some point had a bottleneck of just a few thousand individuals and what probably happened is not that at any time the number of people shrunk below a few hundred thousand what probably happened is that there was a small group that had a decisive mutation that produced an advantage and this group multiplied and killed everybody else and we are descendants of that group yeah i i wonder what uh the the the peculiar characteristics of that group yeah i mean we never know and a lot of people do and we can only we can only just listen to the echoes in our like the the ripples uh that are still within us so i suspect what eventually made a big difference was the ability to organize at scale people to program each other good ideas that we became programmable that we are willing to work on lockstep that we went below uh above the tribal level that we no longer were hoops over a few hundred individuals and uh acted on direct deportation systems transactionally but that we basically evolved an adaptation to become state building yeah to do to form collectives outside of the direct collective yes and that's basically a part of us became committed to serving something outside of what we know yeah then that that's kind of what love is and it's terrifying because it meant that we eradicated the others right it's a force it's an adaptive force that gets us ahead in evolution which means we displace something else that doesn't have that oh so we had to murder a lot of people that weren't about love so love led to destruction have the same strong love as we did right that's why i i mentioned this thing with fascism when you see this uh these speeches do you want total war and everybody says yes right this is this big uh oh my god we are part of something that is more important than me that gives meaning to my existence fair enough [Laughter] uh do you have advice for young people today in high school in college that are thinking about what to do uh with their career with their life so that uh at the end of the whole thing they could be proud of what they did don't cheat have integrity integrity so what does integrity look like when you're the river or the leaf or the fat frog in a lake it basically means that you try to figure out what the thing is that is the most right and this doesn't mean that you don't uh that you have to look for what other people tell you what's right but uh you have to aim for moral autonomy so things need to be right independently of what other people say i always felt that um when people told me to listen to what others say like um read the room build your uh ideas of what's true based on the high status people of your in-group that does not protect me from fascism the only way to protect yourself from fascism is to decide is the world that is being built here the world that i want to be in yeah and so in some sense uh try to make your behavior sustainable act in such a way that you would feel comfortable on all sides of the transaction realize that everybody is you in a different timeline but is seeing things differently and has reasons to do so yeah there's uh i've come to realize this recently there is an inner voice that tells you what's right and wrong and speaking of reading the room there's times what integrity looks like is there's times when a lot of people are doing something wrong and what integrity looks like is not going on twitter and tweeting about it but not participating quietly not doing so it's not like signaling or not all this kind of stuff but actually living your what you think is right like living it now that's also sometimes this expectation that others are like us so imagine the possibility that some of the people around you are space aliens that only look human right so they don't have the same price as you do they don't have don't have the same impulses that's what's right and wrong there is a large diversity in these basic impulses that people can have in a given situation and now realize that you are a space alien right you are not actually human you you think that you're human but you don't know what it means like what it's like to be human you just make it up as you go along like everybody else and you have to figure that out what it means that you are a full human being what it means to be human in the world and how to connect with others on that and there is also something don't be afraid in the sense that you if you do this you're not good enough because if you are acting on these incentives of integrity you become trustworthy that's the way in which you can recognize each other there is a particular place where you can meet you can figure out what that place is where you will give support to people because you realize that they act with integrity and they will also do that so in some sense you are safe if you do that you're not always protected there are people which will abuse you and that might that are bad actors in way that it's hard to imagine before you meet them but there is also people which will try to protect you yeah this that's such a thank you for saying that there's such a hopeful message that no matter what happens to you there'll be a place there's people you'll meet that also have what you have and you will find happiness there and safety there yeah but it doesn't need to end well it can also uh all go wrong so this there's no guarantees in this life so you can do everything right and you still can fail and you can still horrible things happening to you that traumatize you and mutilate you and you have to be grateful if it doesn't happen and ultimately be grateful no matter what happens because even just being alive is pretty damn nice yeah even that you know um the gratefulness in some sense is also just generated by your brain uh to keep you going it's all the trick speaking of which camus said i see many people die because they judge that life is not worth living i see others paradoxically getting killed for the ideas or illusions that give them a reason for living what is called the reason for living is also an excellent reason for dying i therefore conclude that the meaning of life is the most urgent of questions so i have to ask what joshua bach is the meaning of life it is an urgent question according to kamu i don't think that there's a single answer to this nothing makes sense unless the mind makes it so so you basically have to project a purpose and if you zoom out far enough there's the heat test of the universe and everything is meaningless everything is just a blip in between and the question is do you find meaning in this blab in between do you find meaning and observing squirrels do you find meaning in raising children and projecting a multi-generational organism into the future do you find meaning in projecting an aesthetic of the world that you like to the future and trying to serve that aesthetic and if you do then life that has that meaning and if you don't then it doesn't i kind of enjoy the idea that you just create the most vibrant the most weird the most unique kind of blip you can given your environment given your set of skills just be the the most weird set of uh like local pocket of complexity you can be so that like when people study the universe they'll pause and be like uh that's weird it looks like a useful strategy but of course it's still motivated reasoning [Laughter] you're obviously acting on your incentives here it's still a story we tell ourselves within a dream that's that's hardly in touch with reality it's definitely a good strategy if you're a podcaster uh and a human which i'm still trying to figure out if i am yeah there's a mutual relationship somehow somehow josh you're you're one of the most incredible people i know um i really love talking to you i love talking to you again and um it's really an honor that you spend your valuable time with me i hope we get to talk many times throughout our uh through our short and meaningless lives well meaningful or meaningful thank you alex i enjoyed this conversation very much thanks for listening to this conversation with joshua bach and thank you to coinbase codecademy linode netsuite and expressvpn check them out in the description to support this podcast now let me leave you with some words from carl young people will do anything no matter how absurd in order to avoid facing their own souls one does not become enlightened by imagining figures of light but by making the darkness conscious thank you for listening and hope to see you next time you

Channel: Lex Fridman

Views: 636,458

Keywords: agi, ai, ai podcast, artificial intelligence, artificial intelligence podcast, joscha bach, lex ai, lex fridman, lex jre, lex mit, lex podcast, mit ai

Id: rIpUf-Vy2JA

Length: 192min 21sec (11541 seconds)

Published: Sat Aug 21 2021

Reddit Comments

[u/beno73]

Round 1 was probably my favorite episode so far, can't wait to listen to this.

[u/lifuh]

Oh my GOD! I was begging the universe to give me this, and lex and behold, Fridman delivers us Joscha Bach.

Been waiting so long for this one. Rewatching it now.

Joscha is definitely one of the greatest minds alive right now.

[u/mojambowhatisthescen]

I feel like 18 year old me could’ve benefited immensely from listening to Joscha Bach explain things.

Mainly because it might’ve shown me that I really wasn’t as smart as I thought I was, helping me be less of a prick.

I feel like the “there are levels to this” meme applies to this guy and human intelligence.

[u/In5ight]

Man, he’s unrivaled in how he so easily breaks down complicated notions in just a few words.

For anyone feeling discouraged by his statements on reality and consciousness. I think the most important thing to consider is, we as humans can only make educated guesses about the weirdness of this existence. Even as well thought out as Joscha is, there’s no one answer that’s correct, and even if it was, there’s no real way (at least yet) to confirm or deny such theories.

The possibilities are endless, and the mystery of it all is a big part of what makes life enjoyable (for me at least). With that said, I very much respect Joscha’s point of view and this was an extremely entertaining podcast. Possibly the best!

[u/drrrraaaaiiiinnnnage]

The first one was my favorite interview on this podcast, and this one is even better so far.

[u/couchTomatoe]

His takedown of post-modernist philosophies at 1:26:05 is amazing. The best articulation of what is wrong with those styles of thinking I’ve ever heard.

[u/Prodicy]

Finally

Edit: Ok I'd say that was even better than Joscha's first appearance. The level of clarity in this conversation is unparalleled. Must watch for anybody on the fence about it being 3 hours.

[u/valschermjager]

Lex: "What role does Love play in the human condition?"

Joscha: "It is the facilitator of non-transactional interactions."

Omg, and Lex calls himself a robot?? ;-)

[u/Grizius]

Waited for this one! Truly great minds