Joscha Bach: Artificial Consciousness and the Nature of Reality | Lex Fridman Podcast #101

the following is a conversation of Yoshi Bach VP of research at the AI foundation with a history of research positions at MIT and Harvard Yosha is one of the most unique and brilliant people in the artificial intelligence community exploring the workings of human mind intelligence consciousness life on Earth and the possibly simulated fabric of our universe I could see myself talking to Yoshi many times in the future quick summary of the ads to sponsors Express BPM and cash app please consider supporting the podcast by signing up at expressvpn comm slash FlexPod and downloading cash app and using code lex podcast this is the artificial intelligence podcast if you enjoy it subscribe on youtube review it with five stars in a podcast supported on patreon or simply connect with me on Twitter at Lex Friedman since this comes up more often than I ever would have imagined I challenge you to try to figure out how to spell my last name without using the letter E and it'll probably be the correct way as usual I'll do a few minutes of ads now and never any ads in the middle that can break the flow of the conversation this show sponsored by expressvpn get it at expressvpn comm slash flexpod to support this podcast and to get an extra three months free on a one-year package I've been using expressvpn from many years I love it I think expressvpn is the best VPN out there they told me to say it but think it actually happens to be true it doesn't log your data it's crazy fast and it's easy to use literally just one big power on button again for obvious reasons it's really important that they don't log your data it works on Linux and everywhere else too shout out to my favorite flavor of Linux Ubuntu mottai 2004 once again get it at expressvpn comm slash FlexPod to support this podcast and to get an extra three months free on a one-year package this show is presented by cash app the number one finance app in the App Store when you get it use code Lex podcast cash app lets you send money to friends buy Bitcoin and invest in the stock market with as little as one dollar since cash app does fractional share trading let me mention that the order execution algorithm that works behind the scenes to create the abstraction of the fractional orders is an algorithmic marvel so big props the cash app engineers for taking a step up to the next layer of abstraction over the stock market making trading more accessible for new investors and diversification much easier so again if you get cash out from the App Store Google Play and use the collects podcast you get ten dollars in cash wrap will also donate ten dollars to first an organization that is helping advanced robotics and STEM education for young people around the world and now here's my conversation with the OSHA buck as you've said up in a forest in East Germany just as what we're talking about off mic to parents who were artists and now I think at least to me you become one of the most unique thinkers in the AI world so can you try to reverse engineer your mind a little bit what were the key philosophers scientists ideas maybe even movies or just realizations that a impact on you when you're growing up that kind of led to the trajectory or what the key sort of crossroads in the trajectory of your intellectual development my father came from a long tradition of architects distant branch of the family and so basically he was technically a nerd and nerds need to interface in society with non-standard ways sometimes I define a nerd as somebody who thinks that the purpose of communication is to submit your ideas to peer review and normal people understand that the primary purpose of communication is to negotiate alignment and these purposes tend to conflict which means that nerds have to learn how to interact with society at large who is the reviewer in the nerd view of communication everybody who will consider to be a peer so whatever happiest individual is to around well you would try to make him or her the gift of information okay so you're now by the way my research will have Mellon for me so you're architect or artist I study architecture but basically my grandfather made the wrong decision he married an aristocrat and I was drawn into a window into the war and he came back after 15 years so basically my father was not parented by a nerd by but by somebody who tried him tell him what to do and expected him to do what he was told and he was unable to he's unable to do things if he's not intrinsically motivated so in some sense my grandmother broke her son and her son responded by when he became an architect to become an artist so he bought wounded bizarre architecture he built houses without right angles he'd be lots of things that didn't work in more brutalist traditions of eastern Germany and so he bought an old water mill moved out of the countryside and did only what he wanted to do which was art eastern Germany was perfect for p'jem because you had complete material safety put was heavily subsidized Oskar was free you didn't have to worry about rent or pensions or anything so as a socialized communist side yes and the other thing is it was almost impossible not to be in political disagreement with your government which is very productive for artists so everything that you do is intrinsically meaningful because it will always touch on the deeper currents of society of culture and be in conflict visit and tangent visit and you will always have to define yourself and with respect to this so what impact did your father this outside the bar outside the box thinker against the government against the world artists have it was not a thinker he was somebody who only got self-aware to the degree that he needed to make himself functional so in some sense he's it was also late 1960s and he was in some sense a hippie so he became a one-person cult he lived out there in his kingdom he built big sculpture gardens and he started many avenues of art and so on and convinced a woman to live with him she was also an architect and she adored him and decided to share her life with him and I basically grew up in a big cave full of books I'm almost feral and I was bored out there it was very very beautiful very quiet and quite lonely so I started to read and by the time I came to school I've read everything until fourth grade and then some and there was not a real way for me to relate to the outside world and I couldn't quite put my finger on why and today I know it was because I was a nerd obviously and it was the only nerd around so there was no other kids like me and there was nobody interested in physics or computing or mathematics and so on and this village school that I went to was busy in high school kids were nice to me I was not beaten up but I also didn't make many friends or but relationships that only happened and starting from ninth grade when I went to a school for mathematics and physics do you remember any key books from my cigarette everything so I went to the library and I've worked my way through the children's and young adult sections and then I read a lot of science fiction for instance Danny's laflamme basically the great author of cybernetics has influenced me back then I didn't see him as a big influence because everything that he wrote seem to be so natural to me and it's only later that I contrasted it with what other people wrote another thing that was very influential on me were the classical philosophers and also the Tudor of Romanticism so German poetry and art cross two heads off and Heine and up to Heather and so on that's a love Heather so at which point is a classical philosophers end at this point or in the 21st century what's what's the latest classical philosopher does this stretch through even as far as Nietzsche or just I were talking about Plato and there's that one I think that Nietzsche is the classical equivalent of a [ __ ] poster yeah but he's not so much tolling others he's trolling himself because he was at odds with the world largely his romantic relationships didn't work out he got angry and he basically became a nihilist and his nether is not a beautiful way to be isn't until I show it to cast him be trolling yourself to be in that conflict in that no Venice at some point you have to understand the comedy of your own situation if you take yourself seriously and you are not functional it ends in tragedy as I did for Nietzsche by thinking you think he took himself too seriously in the in that tension and as we apply the same thing and in HESA and so on this step involves two enormous classic a dollar sense where you basically feel misunderstood by the world and you don't understand that all the misunderstandings are the result of your own lack of self-awareness because you think that you are a prototypical human and the others around you should behave the same way as you expect them based on your innate instincts and it doesn't work out and you become a transcendentalist to deal with that and so it's very very understandable great sympathies for this to the degree that I can have sympathy for my own intellectual history but out of it was an intellectual a life well-lived a journey well traveled is one where you don't take yourself seriously from now I think that you are neither serious or not serious yourself because you need to become unimportant as a subject that is if you are if a lot of a belief is not a verb you don't do this for the audience you don't do it for yourself you have to submit to the things that are possibly true and you have to follow wherever your inquiry leads but it's not about you and has nothing to do with you so do you think then people like Iran believed sort of an idea of there's a objective truth so GE what's your sense in the philosophical well if you remove yourself a subjective from the picture you think it's possible to actually discover ideas that are true or we just in a measure relative concepts they're an either true nor false it's just a giant mess you cannot define objective truth without understanding the nature of truths in the first place so what does the brain mean by saying that it covers something as truth so for instance a model can be predictive or not predictive then there can be a sense in which a mathematical statement can be tool because it's defined as true under certain conditions so it's basically a particular state that a variable can have an assembled game and then you can have a correspondence between systems and talk about truth which is again a type of model correspondence and that also seems to be a particular kind of ground rules so for instance you're confronted with the enormity of something existing at all right that's standing when you realize something exists rather than nothing and this seems to be true right there is two EPs absolute truth in the fact that something seems to be happening yeah that that to me is a showstopper I could just think about that idea and be amazed by that idea for the rest of my life and not go any farther because I don't even know the answer to that why does anything exist at all well the easiest answer is existence is the default right so this is the lowest number of bits that you would need to encode this whose answer who brought the simplest answer sympathisers that existence is that if what about non-existence I mean that seems non-existence might not be a meaningful notion in the sense so in some sense if everything that can exist exists for something to exist it probably needs to be implementable the only thing that can be implemented as finite automata so maybe the whole of existence is the superposition of all finite automata and we are in some region of the fractal that has the properties that it can contain us what does it mean to be a superposition of fine and vanish superposition of all power like all possible rules imagine that every automaton is basic an operator that acts on some substrate and as a result you get emergent patterns most a substrate is no idea to know so it's based on substrate it's something that can store information something that can store information there is a counter something that can hold state still doesn't make sense to me the why that exists at all I could just sit there with a with a beer or or a vodka and just enjoy effect monitoring the why may not have a why this might be the wrong direction so a skin to this so there could be no relation in in the Y direction without asking for a purpose or for a course it doesn't mean that everything has to have a purpose or cause right so we mentioned some philosophers in that early just taking a brief step back into in today okay so we asked ourselves when did classical philosophy end I think what Germany largely ended was the first revolution that's basically even which was that this was when we ended the monarchy and started a democracy and at this point we basically came up with a new form of government that didn't have a good sense of the this new organism that society wanted to be and in a way it decapitated the universities so the university spent on so modernism like a headless chicken at the same time democracy failed in Germany and we got fascism as a result and it burnt down things in the similar way as Stalinism burnt down intellectual traditions in Russia and Germany boast Germany's have not recovered from this Eastern Germany at this bog or a dialectic materialism and western Germany didn't get much more edgy that Hamas so in some sense both countries lost their intellectual traditions and killing off and driving out Jules didn't help yeah so that was the end that was the end of really rigorous well you would say it's classical classical philosophy is also this thing that in some sense the low-hanging foods in philosophy were mostly wrapped and the last big things that we discovered was the constructivist turn in mathematics so to understand that the parts of mathematics that work are computation it was a very significant discovery in the first half of the 20th century and it hasn't fully permeated philosophy and even physics yet physicists checked out the core libraries from mathematics before constructivism became universal what's constructivist and what are you French girls incompleteness theorems that kind of discuss so it basically girdle himself I think didn't get it yet Hilbert could get it Hilbert saw that for instance a country's set theoretic experiments and mathematics led into contradictions and he noticed that mr. current semantics we cannot build a computer in mathematics that runs mathematics without crashing and a good proof could prove this and so what Google could show is using classical mathematical semantics you run into contradictions and because gödel strongly believed in these semantics and one then in what he could observe and so on he was shocked it basically shook his well to the core because in some sense he felt that the world has to be implemented in classical mathematics and for Turing it wasn't quite so bad I think that you were in could see that the solution is to understand the quest mathematics was computation all along which means you're for instance PI and classical mathematics is a value it's also a function but it's the same thing in a computation a function is only a value of n you can compute it and if you cannot compute the last digit of pi you only have a function you can plug this function into your local Sun let it run until the Sun burns out this is it this is the last digit of pi you will know but it also means that it can be no process in the physical universe or in any physically realized computer that depends on having known the last digit of pi yes which means there are parts of physics that are defined in such a way that cannot strictly be true because assuming that this could be true leads under contrary actions so I think putting computation at the center of the the worldview is actually the right way to think about it yes and Wittgenstein could see it and Wittgenstein basically preempted the largest program of AI that Minsky started later like thirty years later Turing was actually a pupil of Vidkun Stein and really I didn't know there's any connection if it can stand even cancel some classes venturing was not present because he thought it was not worth spending the time if you read the attract address it's a very beautiful book but capacity one salt on 75 pages it's very non typical for philosophy because it doesn't have arguments in it and it doesn't have references in it it's just one thought that is not intending to convince anybody hisses says it's mostly for people that had the same insight as me just spell it out and this insight is there is a way in which mathematics and philosophy ought to meet mathematics tries to understand the domain of all languages by starting with those that are so form Aliza bulette you can prove all the properties of the statements that you make but the price that you pay is that your language is very very simple so it's very hard to say something meaningful in mathematics yes and it looks complicated to people but it's far less complicated than what our brain is casually doing all the time it makes sense of reality and philosophy is coming from the top so it's mostly starting from natural languages which vaguely defined concepts and the hope is that mathematics and philosophy can meet at some point and Wittgenstein was trying to make them meet and he already understood that for instance you could express everything Western and calculus that you could produce the entire logic to NAND gates as we do in all modern computers so in some sense he already understood - and universality before touring spelled it out I think he when he wrote the Tractatus he didn't understand yet that the idea was so important and significant and I suspect then when curing wrote it out nobody cared that much your chewing was not that famous when he lived it was mostly his work in decrypting the German codes that made him famous and or gave him some notoriety but this same status that he has to computer science right now in the eye is something that I think he could acquire later it's kind of interesting and do you think of computation and computer science and you represent that to me is maybe that's the modern-day you in a sense are the new philosopher by sort of the computer scientist who dares to ask the bigger questions that philosophy originally started is the new philosophy is the new philosopher certainly not me I think I mostly the oldest child that grows up in a very beautiful Valley and looks at the world from the outside and tries to understand what's going on and my teachers tell me things and they largely don't make sense right so I have to make my own models I have to discover the foundations of what the others are saying I have to try to fix them to be charitable I try to understand what they must have thought originally or what their teachers or their teachers teachers must have thought until everything are lost in translation and how to make sense of the reality that we are in and whenever I have an original idea I'm usually late to the party by say 400 years and the only thing that's good is that the parties get smaller and smaller the older I get and the more I explore the part the party gets smaller and more exclusive and more exclusive so it seems like one of the key qualities of your upbringing was that you are not tethered whether it's because your parents or in general maybe you're something within your within your mind some genetic material you were not tethered to the ideas of the general populace which is actually a unique property we're kind of throughout you know the education system and whatever from that education system just existing in this world forces certain sets of ideas onto you can you uh disentangle that why were you why are you not so tethered even in your work today you seem to not care about perhaps a best paper in Europe's right being tethered to particular things that current today in this year people seem to value as a thing you put on your CV and resume you're a little bit more outside of that world outside of the world of ideas that people are especially focusing the benchmarks of today the things what can you disentangle that because I think that's inspiring and if there were more people like that we might be able to solve some of the bigger problems that sort of AI dreams to solve and that's a big danger in this because in a way you are expected to marry into an intellectual tradition and visit this tradition into a particular school if everybody comes up with their own paradigms the whole thing is not cumulative as an enterprise right so in some sense you need a healthy balance you need paradigmatic thinkers and you need people that work within given paradigms basically sciences today to find themselves largely by methods and it's almost a disease that we think as a scientist somebody who was convinced by the guidance counselor that they should join a particular discipline and then they find a good mentor to learn the right methods and then they are lucky enough and privileged enough to join the right team and then they will their name will show up on influential papers but we also see that there are diminishing returns with this approach and when our field computer science day I started most of the people that joined this field had interesting opinions and today's thinkers and AI either don't have interesting opinions at all or these opinions are inconsequential for what they actually doing because what they're doing is they apply the state-of-the-art methods with a small epsilon and this is often a good idea if if you think that this is the best way to make progress and for me it's first of all very boring if somebody else can do it why should I do it right if if the current methods of measuring learning lead to strong AI why should I be doing it right well just wait and hold that done and wait until they do this on the beach or read interesting books or write some and have fun but if you don't think that we are currently doing the right thing if we are missing some perspectives then it's required to think outside of the box it's also required to understand the boxes but it's it's necessary to understand what worked and what didn't work and for what reasons so you have to be willing to ask new questions and design new methods whenever you want to answer them and you have to be willing to dismiss the existing methods if you think that they're not going to give the right answers it's very bad career advice to do that so maybe to briefly stay for one more time in the early days one would you say for you was the dream before we dive into the discussions that we just almost started one was the dream to understand or maybe to create human level intelligence born for you I think that you can see AI largely today as advanced information processing if you would change the acronym of AI and to that most people in the field would be happy it would not change anything what they're doing for your automating statistics and when you of the statistical models are more advanced than what statisticians had in the past and it's pretty good work it's very productive and the the other aspect of AI is is philosophical project and this philosophical project is very risky and very few people work on it and it's not clear if it succeeds so first of all let's this is you you keep throwing a sort of a lot of really interesting ideas and I have to pick which ones we cook with but sort of first of all you use the term information processing just information processing as if it's it's the mirror it's the muck of existence as if it's the epitome of a logistic that that the entirety the universe may be information processing it consciousness the intelligence might be information problem so that maybe you can comment on if that's if the advanced information processing is is a limiting kind of realm of ideas and then the other one is would II mean by the philosophical project so I suspect that general intelligence is the result of trying to solve general problems so intelligence I think is the ability to model it's not necessarily goal directed rationality or something many intelligent people are bad at this but it's the ability to be presented with a number of patterns and see a structure in those patterns and be able to predict the next set of patterns right to make sense of things and some problems are very trainable usually Intel serfs control so you make these models for a particular purpose of interacting as an agent with the world and getting certain results but it's the intelligence itself is in the sense instrumental to something but by itself it's just the ability to make models and some of the problems are so general that the system that makes them needs to understand what itself is and how it relates to the environment so as a child for instance you notice you do certain things despite you perceiving yourself as wanting different things so you become aware of your own psychology you become aware of the fact that you have complex structure in yourself and you need to model yourself to reverse-engineer yourself to be able to predict how you will react to certain situations and how you deal with yourself in relationship to your environment and this process if this project if you reverse engineer yourself new relationships or reality in the nature of a universe that can continue if you go all the way this is basically the project of AI or you could say the project of AI is a very important component in it the tutoring test in a way is you ask a system what is intelligence if that system is able to explain what it is how it works then you would should assign it the property of being intelligent in this general sense so the test the Turing was administering in a way I don't think that he couldn't see it but he didn't express it yet and the original 1950 paper is that he was trying to find out other that he was generally intelligent because in order to take this test the wrappers of course you need to be able to understand what that system is saying and we don't yet know if we can build an AI have you don't yet know if you are generally intelligent basically you win the Turing test by building an AI yes so it so in a sense hidden within the Turing test is a kind of recursive test yes it's a test on us yeah the Turing test is basically a test of the conjecture whether people are intelligent enough to understand themselves okay but you also mentioned a little bit of a self-awareness and then the project of AI do you think this kind of emergent self-awareness is one of the fundamental aspects of intelligence so as opposed to goal oriented ease you said kind of puzzle solving is coming to grips with the idea that you're an agent in the world and I find that many highly intelligent people are not very self-aware right so self-awareness and intelligence are not the same thing and you can also be surf aware if you have put priors especially it without being especially intelligent so you don't need to be very good at solving puzzles if the system that you are already implements the solution but I do find intelligence so you kind of mentioned children right it is that the fundamental project of AI is to create the learning system that's able to exist in the world so you kind of drew a difference between self-awareness and intelligence and yet you said that the self-awareness seems to be important for children so I call this ability to make sense of the world and your own place and so to understable make you able to understand what you're doing in this world sentience and I would distinguish sentience from intelligence because sentience is the possessing certain classes of models and intelligence is the way to get to these models if you don't already have them I see so can you maybe pause a bit and try to answer the question that we just said we may not be able to answer and might be a recursive meta question of what is intelligence and I think that intelligence is the ability to make models the models is I think it's useful as examples very popular now neural networks form representations of large-scale data set they they form models of those data sets when you say models and look at today's new all networks what are the difference of how you're thinking about what is intelligent in saying that intelligence is the process of making models two aspects tool to this question one is the representation is the representation adequate for the domain that we want to represent and the other one is is the type of the model that you arrive at adequate so basically are your modeling the correct domain and I think in both of these cases modern AI is lacking stuff and I think that I'm not saying anything new you're not criticizing the field most of the people that design our paradigms are aware of that and so one aspect that you're missing is unified learning when we learned we'd at some point discover that everything that we sends this part of the same object which means we learn it all into one model and we call this model the universe so an experience of the world that we are embedded on it's not a secret direct via to physical reality physical reality is a view at quantum graph that we can never experience or get access to but it has this properties that it can create certain patterns at our systemic interface to the world and we make sense of these patterns and the relationship between the patterns that we discover is what we call the physical universe so at some point in our development is a nervous system we discover that everything that we relate to and in the world it can be mapped to a region in the same three-dimensional space by and large we now know in physics that this is not quite true well it's not actually three-dimensional but the world that we are entangled is at the level of which we are entangled this is largely a flat three-dimensional space and so this is the model that our brain is intuitively making and this is I think what gave rise to this intuition of res extends a-- of this material world this material domain it's one of the mental domains but it's just the class of all models that relate to this environment this v dimension of physics engine in which we are embedded physics engine or embedded i love that phrase it just slowly pause so the the quantum graph i think you called which is the real world which you can never get access to there's a bunch of questions i want to sort of disentangle that maybe one useful one one of your recent talks i looked at can you just describe the basics can you talk about what is dualism what does idealism what is materialism what is functionalism and what connects with you most in terms of because you just mentioned there's a reality we don't have access to okay what does that even mean and why don't we get access to it only part of that one week why can we access it so the particular trajectory that mostly exists in the West is the result of our indoctrination by a card for 2000 years occult which yes the Catholic cause mostly yes and for better or worse right it has created or defined many of the modes of interaction that we have that have best created this society but it has also in some sense scarred our rationality and the intuition that exists if you would translate the mythology of the Catholic Church into the modern world is that the world in which you and me interact is something like a multiplayer role-playing adventure yes and the money and the objects that we have in this world this is all not real or is Eastern philosophers would say it's my eye it's just stuff that is it appears to be meaningful and this embedding in this meaning and people leave in it is samsara this it's basically the identification with the needs of the mundane secular everyday existence and the Catholics also introduced the notion of higher meaning the sacred and this existed before but eventually the natural shape of God is the Platonic form of the civilization that you're part of it's basically the super organism that is formed by the individuals as an intentional agent and basically the Catholics used relatively crude mythology to implement software on the minds of people and get the software synchronized to make them walk in lockstep this basically get they get this got online and you make it efficient and effective and I think our God technically is just itself that spends multiple brains as opposed to your and myself which mostly exists just on one brain right and so in some sense you can construct yourself functionally as a function is implemented by brains that exists across brains and this is a God with a small G that's one of the if you look evil Harare kind of talking about this is one of the nice features of our brains it seems to that we can all download the same piece of software I got in this case and kind of share it yes you give everybody a spec and the mathematical constraints that are in front to information-processing make sure that given the same spec you come up with a compatible structure okay so that's there's the space of ideas that we all share and we think that's kind of the mind and but that's separate from the idea is from from Christianity for from religion is that there's a separate thing between the mind as a real vault and this real world is the world in which God exists God is the quarter of the multiplayer adventure so to speak and we are all players in this game and that's dualism usually but it is because the mental realm is exists in a different implementation than a physical realm and the mental realm is real and a lot of people have this intuition that there is this real room in which you and me talk and speak right now then comes a layer of physics and abstract rules and so on and then comes another real room where our souls are and our tool form isn't the thing that gives us phenomenal experience and this of course a very confused notion that you would get and it's basically it's the result of connecting materialism and idealism in the wrong way so okay I apologize but I think it's really helpful if we just tried to define try to define terms like what is joules and what is idealism what is materialism for people done' so the idea of dualism and our cultural tradition is that there are two substances a mental substance and a physical substance and they interact by different rules and the physical world is basically causally closed and is built on a low level causal structures or the bezier bottom level that is causally closed it's entirely mechanical and mechanical in the widest sense so it's computational there's basically a physical world in which information flows around and physics describes the laws of how information flows around an adult would you compare it to like a computer where you have a hardware and software the computer is a generalization of information flowing around basically but join discovered that there is genuine universal principle you can define this Universal machine that is able to perform all the computations so all these machines have the same power this this means that you can always define a translation between them as long as they have unlimited memory to be able to perform each other's computations so would you then say that materialism is this whole world is just the hardware and idealism is this whole world is just a software why I think that most idealists don't have a notion of software yet because software also comes down to information processing right so what you notice is the only thing that is real to you and me is this experimental world in which things matter in which things have taste in which things of color phenomenal content and so on and you are bringing up consciousness okay and this is distinct from the physical world in which things have values in only in an abstract sense and you only look at cold patterns moving around so how does anything feel like something in this connection between the two things is very puzzling to a lot of people of course to many philosophers so idealism starts out with the notion that mind is primary materialism thinks that matter is primary and so for the idealist the material patterns that we see a play in playing out a part of the dream that the mind is dreaming and we exist in the mind on a higher plane of existence if you want and for the materialist there is only this material thing and that generates some models and via the result of these models and in some sense I don't think that we should understand if you understand it properly materialism and idealism is a dichotomy but there's two different aspects of the same thing so the via thing is we don't exist in the physical world we do exist inside of a store way that the brain tells itself ok that's it let me uh let my my my information processing I take they take that in we don't exist in the physical world we exist in the narrative basic your brain cannot feel anything New York cannot feel anything they're physical things physical systems are unable to experience anything but it would be very useful for the brain or for the organism to know what it would be like to be a person and to feel something yeah so the brain creates a simulacrum of such a person that it uses to model the interactions of the person's the best model of what that brain this organism thinks it is in relationship to its and so it creates that model it's a story a multimedia novel that the brain is continuously writing and updating but you also kind of said that you said that we kind of exist in the head and that's alright yes that story yeah what is real in any of this so like there's a again these terms are you kind of said there's a quantum graph I mean what is what is this whole thing running on then is this story and is it completely fundamentally impossible to get access to it because isn't the story supposed to is in the brain in a in something in existing in some kind of context so what we can identify as computer scientists we can engineer systems and test our theories this way that may have the necessary and sufficient properties to produce the phenomena that you're observing which is there is itself in a virtual world that is generated in somebody's neocortex who that is contained in the skull of this primate here and when I point at this this indexicality is of course wrong but I do create something that is likely to give rise to patterns on your retina that allow you to interpret what I'm saying right but I both know that the world that you and me are seeing is not the real physical world what we are seeing is a virtual reality generated in your brain to explain the patterns on your retina how close is it to the real world that's kind of the the question is it when you have when you have like people like Donald Hoffman let's say that like that you're really far away the thing we're seeing you and I now that interface would have it's very far away from anything like we don't even have anything close like to the sense of what the real world is or is it a very surface piece of architecture imagine you look at the Mandelbrot fractal right this famous thing that when a man would discover deadlines if you're you see an overall shape and they're right but you know if you truly understand it you know it's two lines of quote it's basically in a series that is being tested for complex numbers and in the complex number plane for every point and for those for this year is is diverging you paint this black and where it's converging you don't and you get the intermediate colors by taking how far it diverges yes right this gives you this shape of this fractal but imagine you live inside of this fractal and you don't have access to where you are in the fractal or you have not discovered the generator function even right so what you see is all of all I can see right now is the spiral and the spiral moves a little bit to the right is this an accurate model of reality yes it is right it is an adequate description is you know that there is actually no spiral and the mailboat fractal it only appears to like this to an observer that is interpreting things as a two-dimensional space and then define certain regularities in there at a certain scale that currently observes because if you zoom in the spiral might disappear and turn out to be something different at the different resolution right yes so at this level you have the spiral and then you discover the spiral moves to the right and some point it disappears so you have a singularity at this point your model is no longer valid you cannot predict what happens beyond the singularity but you can observe again and you will see it is another spiral and at this point it disappeared so maybe we now have a second-order law and if you make 30 layers of these laws then you have a description of the world that is similar to the one that we come up with when we describe the reality around us it's reasonably predictive it does not cut to the core of it so you explain how it's being generated how it actually works but it's relatively good to explain the University of your entangled fence but you don't think the tools are computer sizes the tools of physics could get could step outside see the whole drawing and get at the basic mechanism of how the pattern the spiral is generated imagine you would find yourself embedded into a mother but Franklin you try to figure out what works and you you know somehow have a throwing machine there's enough memory to think and as a result you've come to this idea it must be some kind of automaton and maybe you just enumerate all the possible automata until you get to the one that produces your reality so you can identify necessary and sufficient condition for instance we discover that mathematics itself is the domain of all languages and then we see that most of the domains of mathematics that we have discovered are in some sense describing the same this is what category theory is obsessed about that you can map these different domains to each other so they're not that many fractals and some of these have interesting structure and symmetry breaks and so you can just cover what region of this global fractal you might be embedded in from first principles yes but the only way you can get there is from first principles so basically your understanding of the universe has to start with automata and the number theory and then spaces and so on yeah I think like Stephen Wolfram still dreams that he's it that he'll be able to arrive at the fundamental rules of the cellular automata or the generalization of which is behind our universe yeah it's you've said on this topic you said in a recent conversation that quote some people think that a simulation can't be conscious and only a physical system can but they got a completely backward a physical system cannot be conscious only a simulation can be cautious yeah consciousness is a simulated property that's simulate itself yeah just like you said the mind is kind of the call it story narrative there's a simulation or our mind is essentially a simulation and usually I try to use the terminology so that the mind is basically a principles that produce the simulation it's the software that is implemented by your brain and the mind is creating both the universe that we are in and the self the idea of a person that is on the other side of attention and is embedded in this world why is that important that idea of a self why is that an important feature in simulation it's basically a result of the purpose that the mind has it's a tool for modeling right we are not actually monkeys via side effects of the regulation needs of monkeys and what the monkey has to regulate is the relationship of an organism to an outside world that is a large part also consisting of other organisms and as a result it basically has regulation targets that it tries to get to this regulation target start with priors they're basic like unconditional reflexes that we are more less born with and then we can reverse-engineer them to make them more consistent and then we get more detailed models about how the world works and how to interact with it and so these priors that you commit to are largely target values that our needs should approach set points and this deviation to the set point creates some urge some tension and we find ourselves living inside of feedback loops right consciousness emerges over dimensions of disagreements with the universe things that you care things are not the way there should be but you need to regulate and so in some sense the sense self is the result of all the identifications that you're having an identification is a regulation tracker that you're committing to it's a dimension that you care about do you think is important and this is also what locks you in if you let go of these commitments of these identifications you get free there's nothing that you have to do anymore and if you let go of all of them you're completely free and you can enter Nirvana because you're done and actually this is a good time to pause and say thank you to sort of a friend of mine Gustav's or Ostrom who introduced me to your work I wanted to give him a shout out he's a brilliant guy and I think the AI community is actually quite amazing and Gustav is a good representative that you are as well some I'm glad first of all I'm glad the internet exists you - who's this where I can watch your talks and then get to your book and study your writing and think about you know that's that's amazing okay but the you've kind of described instead of this emergent phenomena of consciousness from the simulation so what about the hard problem of consciousness the can you just linger on it like but why this is still feel like I understand you're kind of the self is an important part of the simulation but why does the simulation feel like something so if you look at the book by say george RR martin with the characters have plausible psychology yeah and they stand on a hill because they want to conquer the city below the hill and they've done in it and then look at the color of the sky and they are Princip and feel empowered and all these things why do they have these emotions it's because it's written into the story right and threatened with the story because it's an adequate model of the person that predicts what they're going to do next and the same thing is helpful it's basically a story that our brain is writing it's not written in words it's written in perceptual content basically multimedia content and it's a model of what the person would feel if it existed so it's a virtual person and you and me happen to be this virtual person so if this virtual person gets access to the language center and talks about the sky being blue and this is us but hold on a second do I exist in your simulation you do exist even almost similar way as me so there are internal states that I that are less accessible for me in that you have and so on and you're my model might not be completely adequate there are also things that I might perceive about you that you don't perceive but in some sense both you and me are some puppets - puppets that enact this play in my mind and I identify with one of them because I can't control one of the puppet directly and with the other one I can create things in between so for instance we can go or in an interaction that even leads to a coupling to a feedback loop so we can sync things together in a certain way or feel things together but this coupling is itself not a physical phenomena entirely a software phenomenon it's a result of two different implementations interacting with each other so this is thing so are you suggesting I did like the way you think about it is the entirety of existence simulation and we're kind of each mind is a little sub simulation that like why don't you why doesn't your mind have access to my mind's full state like for the same reason that my mind hasn't have access to its own full state so what I mean there is no trick involved so basically when I say know something about myself it's because I made a model yes of your brain is tasked with modeling what other parts of your brain are doing yes but there seems to be an incredible consistency about this world in the physical sense that is repeatable experiments and so on yeah how does that fit into our silly the center of apes sim you of the world so why is it some repeat why is everything so repeatable and not everything there's a lot of fundamental physics experiments that are repeatable for a long time all over the place and so on laws of physics how does that fit in it seems that the parts of the world that are not deterministic are not long-lived so if you build a system any kind of automaton so if you build simulations of something you'll notice that the phenomena that endure are those that give rise to stable dynamics so basically if you see anything that is complex in the world it's the result of usually of some control of some feedback that keeps it stable around certain attractors and the things that are not stable that don't give rise to certain harmonic patterns and so on they tend to get weeded out over time so if we are in a region of the universe that sustains complexity which is required to implement Minds like ours this is going to be a region of the universe that is very tightly controlled and controllable so it's going to have lots of interesting symmetries and also symmetry breaks that allow the creation of structure but they exist where so there's such an interesting idea that our - simulation is constructing the narrative but my question is just to try to understand how that fits with this with the entirety of the universe you're saying that there's a region of this universe that allows enough complexity to create creatures like us but what's the connection between the the brain the mind and the broader universe which comes first which is more fundamental is the is the mind the starting point the universe is emergent is the universe the starting point the minds are emergent I think quite clearly the letter it's at least a much easier explanation because it allows us to make causal models and I don't see any way to construct an inverse cos allottee so what happens when you die to your mind simulation my implementation ceases so basically the thing that implements myself will no longer be present it means if I am NOT implemented on the minds of other people to think that I identify this is the weird thing is I don't actually have an identity beyond the identity that I construct if I was the Dalai Lama he identifies as a form of government so basically the dad Adama gets reborn not because he is confused but because he is not identifying as a human being he runs on a human being he's basically a governmental software right that is instantiated in every new generation in you so his advisors will pick someone who does this in the next generation so if you identify as this you are no longer human and you don't die and essentially what dies is only the body of the human that you ran on here to kill the Dalai Lama you would have to kill his tradition and if we look at ourselves we realized that we are to a small part like this most of us so for instance if you have children you realize something lives on in them or if you spark an idea in the world something lives on or if you identify it as a society around you because you are part that you are not dressed this human being yes so in a sense you are kind of like a Dalai Lama and since that you Jascha Bach is just a collection of ideas so like you have this operating system on which is a bunch of ideas live and interact and then once you die they kind of part some of them jump off the should it put it the other way identity is a software state it's a construction it's not physically real identity is not a physical concept it's basically a representation of different objects on the same world line but identity let lives and eyes are you attached this is it's what's the fundamental thing is that the ideas that come together to form identity or is each individual identity actually a fundamental thing it's a representation that you can get agency over if you care so basically you can choose what you identify best if you want to nobody just seems if if the mind is not very real it's not that the the birth and death is not a crucial part of it well maybe I'm silly maybe I'm attached to this whole biological organism but it's that the physical being a physical object in this world is is a an important aspect of birth and death like it feels like it has to be physical to die it feels like simulations don't have to die the physics that we experience is not the real physics that explains is no color and sound in the real world color and sound are types of representations that you get if you want to model reality with oscillators right so colors and sound in some sense have octaves yes and it's because they are represented probably with oscillators right so that's why colors form a circle of views and colors have harmonic sounds have harmonics is a result of synchronizing oscillators in in the brain right so the world that we subjectively interact with is fundamentally the result of the representation mechanisms in our brain they are mathematically to some degree Universal they are certain regularities that you can discover in the patterns and not others but the patterns that we get this is not the real world the world that we interact with is always made of too many parts to count right so when you look at this table and so on it's consisting of so many more molecules and atoms that you cannot count them so you only look at the aggregate dynamics at limit dynamics if you had almost infinitely many patterns of particles what would be the dynamics of the table and this is roughly what you get so geometry that we are interacting this is the result of discovering those operators that work in the limit that you get by building an infinite series that converges for those parts where it converges its geometry for those parts or a dozen convergence chaos right and then so all that is filtered through with the cuts of the consciousness that's emergent in our narrative the the consciousness gives it color gives a feeling gives it flavor so I think the feeling flavor and so on is given by the relationship that a feature has to all the other features it's basically a giant relational graph that is our subjective universe the color is given by those aspects of the representation or the this experiential color where you care about but you have identifications but something means something where you are the inside of a feedback loop when the dimensions of of caring are basically dimensions of this motivational system that we emerge over the the meaning of the relations the graph can you elaborate that a little bit like where does the maybe we can even step back and ask the question of what is consciousness to be sort of more systematically what what what do you how do you think about consciousness consciousness is largely a model of the contents of your attention it's a mechanism that has evolved for a certain type of learning at the moment of a machine learning systems we largely work by building chains of weighted sums of real numbers with some non-linearity and you will learn by typing an error signal so these different chained layers and adjusting the weights in this way that Samms and you can approximate most polynomials if you have enough training data but the prices you need to change a lot of these weights basically the error is piped backwards into the system until it accumulates at certain junctures in the network and everything else evens out statistically and only at these junctures this is where you had the actual error in the network you make the change there this is a very slow process and our brains don't have enough time for that because we don't get old enough to play go the way that our machines learn to play go so instead what we do is an attention based learning we pinpoint the probable region in the network where we can make an improvement and then we store the this binding state together with the expected outcome in a protocol and there's ability to make index memories for the purpose of learning to revisit these commitments later this requires an memory of the contents of our attention another aspect is when I construct my reality and make mistakes so I sees things that turn out to be reflections or shadows and so on which means I have to be able to point out which features of my perception gave rise to a present construction of reality so the system needs to pay attention to the earth features that are currently in its focus and it also needs to pay attention to whether it pays attention itself in part because the attentional system gets trained is the same mechanism so it's reflexive but also in part because your attention lapses if you don't pay attention to the attention itself all right so it's this thing that I'm currently seeing just a dream that my brain has spun off into some kind of daydream or am I still paying attention to my percept so you have to periodically go back and see whether you are still paying attention and if you have this loop and you make it tight enough between the system becoming aware of the contents of its attention and the fact that it's paying attention itself and makes attention the object of its attention I think this is the loop over which if you wake up so there's this so there's this attentional mechanism that's somehow self referential that's fundamental to what consciousness is mm-hmm so just uh ask you a question I don't know how much you're familiar with the recent break there is a natural English processing they use attentional mechanisms used something called transformers to learn patterns and sentences by allowing a network to focus its attention to particular parts of a sentence at each individual so like parameterize and make it learn about the dynamics of a sentence by having like a little window into the into the sentence do you think that's like a little step towards that eventually would will take us to the intentional mechanisms from which consciousness could emerge not quite I think it models only one aspect of attention in the early days of automated language translation there was a example that I found particularly funny where somebody tried to translate a text from English into German and it was a bet broke the window and the translation in German was eine Fledermaus it's a practice Fenster MIT einem baseball schlager so to translate it back into English a bet the this flying mammal broke the window with a baseball bat yes and it seemed to be the most similar to this program because it somehow maximized the possibility of translating the concept bat into German in the same sentence and this is some a mistake that the Transformer model is not doing because it's tracking identity and the attentional mechanism in the Transformer model is basically putting its finger on individual concepts and make sure that these concepts pop up later in the text yeah and tracks basically the individuals through the text and it's why the system can learn things that other systems couldn't before it which makes the for instance possible to write a text where it talks about the scientist then the scientist is a name and has a pronoun and it gets a consistent story about that thing what it does not do it doesn't fully integrate this so his meaning falls apart at some point it loses track of this context it does not yet understand that everything that it says has to refer to the same universe and this is where this thing falls apart but the attention in transformer model does not go beyond tracking identity and tracking identity is an important part of attention but it's a different very specific attentional mechanism and it's not the one that gives rise to the type of consciousness that they have okay just to linger I know what what do you mean by identity in the context of language so when you talk about language that you have different words that can refer to the same concept got it and in the sensor concepts so yes and it can also be in a nominal sense or an indexical sense that you say yeah this word does not only refer to this class of objects but it refers to a definite object to some kind of agent that waves their way to through the story and it's only referred by different ways in the language so the language is basically a projection from a conceptual representation from a scene that is evolving into a discrete string of symbols and what the transformer is able to it learns aspects of this projection mechanism that other models couldn't learn so have you ever seen an artificial intelligence or any kind of construction idea that allows for unlike neural networks or perhaps within your networks it's able to form something where the space of concepts continues to be integrated so the way you're describing building an all knowledge base building this consistent larger and larger sets of ideas that would then allow for a deeper understanding of it concerns thought that we can build everything from language from basically a logical grammatical construct and I think to some degree this was also what Minsky believed so that's why I focus so much on common sense reasoning and so on and project that was inspired by him both psyche um there was special going on yes of course ideas don't die only people die and that's true but in doubt psyche is a productive project it's just probably not one that is going to converge to general intelligence the thing that Wittgenstein couldn't solve and he looked at this in his book at the end of his life philosophical investigations was the notion of images so images play an important role in track titles the Tractatus an attempt to basically turn philosophy into logical probing language to design a logical language in which you can do actual philosophy that rich enough for doing this and the difficulty was to deal with perceptual content and eventually I think he decided that he was not able to solve it and I think this preempted the failure of the logit his program in AI in the solution as we see it today is we need more general function approximation there are functions geometric functions that we learn to approximate that cannot be efficiently expressed and computed in a grammatical language can of course build automata that go via number theory and so on and to learn linear algebra and then compute an approximation of this geometry but to equate language and geometry is not an efficient way to think about it so functional is well you kind of just said then you'll now work sir the sort of the approach in you all know this takes is actually more general than the then what can be expressed through language yes so what can be efficiently expressed through language at the data rates at which we process grammatical language okay so you don't think so you don't think languages so you disagree with Wittgenstein that language is not fundamental - I agree with commit constrain it I just agree with the late Wittgenstein and I also agree with the beauty of the early Wittgenstein I think that the Tractatus itself is probably the most beautiful philosophical text that was written in the twentieth century but but language is not fundamental to cognition and intelligence and consciousness so I think that language is a particular way or the natural language that we're using is a particular level of abstraction that we used to communicate with each other but the languages in which people express geometry are not grammatical languages in the same sense so they work slightly different they're more general expressions of functions and I think the general nature of a model is you have a bunch of parameters these are have arranged it as these are the variances of the world and you have relationships between them which are constraints which say if certain parameters have these values then other parameters have to have the following values and this is a very early insight in computer science and I think the some of the earliest formulations is the Boltzmann machine and the problem is the Boltzmann machine is that it has a measure of whether it's good this is basically the energy on the system the amount of tension that you have left and the constraints where the constraints don't quite match it's very difficult to despite having this global measure to train it because if yes as soon as you add more than trivially fuel elements parameters into the system it's very difficult to get it settle in the right architecture and so we the solution that Hinton and Sinofsky found was to use a restricted Boltzmann machine which uses the hidden links the internal links and in the Boltzmann machine and only has based the input and output layer but this limits the Express ativy Civet e of the boltzmann machine so now he builds a network of small of these primitive Boltzmann machines and in some sense you can see a almost continuous development from this to the deep learning models that we are using today even though we don't use Boltzmann machines at at this point but the idea of the Boltzmann machine is you take this model you clamp some of the values to perception and this forces the entire machine to go into a state that is compatible with the states that you currently perceive and this state is your model of the world right so I think it's a very general way of thinking about models but we have to use a different approach to make it work this is we have to find different networks that train the Boltzmann machine so the mechanism that trains the Boltzmann machine and the mechanism that makes the Boltzmann machine settle into its state are distinct from the constrained architecture of the Boltzmann machine itself the the kind of mechanism we want to develop yes so this the direction in which I think our research is going to go is going to for instance what you notice in perception is our perceptual models of the world are not probabilistic but possible istic which means with them you should be able to perceive things that are improbable but possible right the sexual State is valid not if it's probable but if it's possible if it's quite coherent yeah so if you see a tiger coming after you should be able to see this even if it's unlikely and the probability is necessary for convergence of the model so given the state of possibilities that is very very large and a set of perceptual features how should you change the state of states of the model together to convert with your perception but the space of the space of ideas that are coherent with the context that you're sensing is perhaps not as large I mean that that's perhaps pretty small the degree of coherence that you need to achieve depends of course how deep your models goal is for instance politics is very simple when you know very little without game theory and human nature so the younger you are the more obvious is how politics should work right yes and because you get in a Korean aesthetics from relatively few inputs and the more layers you model them add more layers you model reality the harder it gets to satisfy all the constraints so you know the current neural networks are fundamentally supervised learning system with the feed-forward neural network is back propagation to learn what's your intuition about what kind of mechanisms might we move towards to improve the learning procedure I think one big aspect is going to be meta learning and architecture search starts in this direction in some sense the first wave of AI classical a I work by identifying a problem into the possible solution and implementing the solution right program that plays chess and right now we are in the second wave of AI so instead of writing the algorithm that implements the solution revise an algorithm that automatically searches for an algorithm that implements the solution so the learning system in some sense is an algorithm that itself discovers the algorithm that solves the problem or goes too hard to implement it by dissolution by hand but we can implement an algorithm that finds the solution yes so now let's move to the third stage right the third stage would be meta-learning find an algorithm that discovers the learning algorithm for the given domain our brain is probably not a learning system but a meta learning system this is one way of looking at what we are doing there is another way if you look at the way our brain as for instance implemented there is no central control that tells all the new ones how to wire up yes instead every neuron is an individual reinforcement learning agent every neuron is a single-celled organism that is quite complicated and in some sense quite motivated to get fed and it gets fed if it fires on average at the right time yes auntie the right time depends on the context that the neuron exists in which is the electrical and chemical environment that it has so it basically has to learn a function over its environment that tells us when to fire to get fat or if you see it as a reinforcement learning agent every neuron is in some sense making a hypothesis when it sends a signal it tries to pipe a signal through the universe and tries to get positive feedback for it and the entire thing is set up in such a way that it's robustly self-organizing into a brain which means you stride out with different neuron types that have different priors in which hypothesis to test on how to get its reward and you put them into different concentrations in a certain spatial alignment and then you entrain it in a particular order and as a result you get develop a nice brain yeah so okay so the brain is a meta learning system with a bunch of with reinforcement learning agents and what I think you said but just to clarify where do the LA there's no centralized government that tells you here's a loss function here's a loss function here's a loss function like what who is who says what's the also governments which impose loss functions on different parts of the brain so we have differential attention some areas in your brain get especially rewarded when you look at faces if you don't have that you will get post of agnosia which basically mean the inability to tell people apart by their faces so and the reason that happens is because it was had an evolutionary advantage like evolution comes in a play here about it's basically an extraordinary attention that we have for faces I don't think that people were supposed to up no see I have Percy a defective brain the brain just has an average attention for faces so people were supposed of agnosia don't look at faces more than they look at cups so the level at which they resolve the geometry of faces is not higher than the one that then four cups and people that don't have prosopagnosia looked obsessively at faces right for you and me it's impossible to move through a crowd without scanning the faces and as a result we make insanely detailed models of faces that allow us to discern mental states of people so obviously we don't know 99% of the details of this meta learning system that's our mind okay but still we took a leap from something much dumber to that from love through the evolutionary process can you first of all maybe say how hard these how big of a leap is that from our brain from our a branch asters to multi cell organisms and is there something we can think about about as we start to think about how to engineer intelligence is there something we can learn from evolution in some sense life exists because of the market opportunity of controlled chemical reactions we compete with some chemical reactions and we win in some areas against this damp combustion because we can harness those entropy gradients where you need to add a little bit of energy in a specific way to harvest more energy so we are competing combustion yes in many regions we do and we try very hard because when we under ekam petition we lose right yeah so because the combustion is going to close the entropy gradients much faster than we can run yes you gotta quit so I probably am yeah so basically to this because every cell has a Turing machine built into it it's like literally a read/write head of the tape and so everything that's more complicated than a molecule that just is a vortex around attractors that needs the Turing machine in it for its regulation and then you bind cells together and you get next level organization or organism where the cells together implement some kind of software and for me very interesting discovery in the last year was the word spirit because I realized that what spirit actually means it's an operating system for an autonomous robot and when the word was invented people needed this word but they didn't have robots that they built themselves yet the only autonomous robots that were known were people animals plants ecosystems cities and so on and they all had spirits and it makes sense to say that the plant is an operating system right if you pinch the plant in one area then there's going to have repercussions throughout the plant everything in the plant is in some sense connected into some global aesthetics like in other organisms an organism is not a collection of cells is a function that tells cells how to behave and this function is not implemented as some kind of supernatural thing like some more for genetic field it is an emergent result of the interactions of the each service each other cell all right so you're you're saying is the organism is a function that tells what's what what now that the cell sells what to do and the function is an emerging the interaction of the cells yes so it's basically a description of what the plant is doing in terms of macro States and the micro States the physical implementation are too many of them to describe them so the software that we use to describe what the plant is doing the spirit of the plant is the software the operating system of the plant right this is a way in which V the observers make sense of the plant yes okay same is true for people so people have spirits which is their operating system in a very rightness aspects of that operating system that relate to how your body functions and others how you socially interact or you interact with yourself and so on and we make models of that spirit and we think it's a loaded term because it's from a pre-scientific age but we it took the scientific age a long time to rediscover a term that is pretty much the same thing and I suspect that the difference is that we still between the old world and the new world our translation errors over the centuries but can you actually link around that like well why do you say that spirit just to clarify because I'm a little bit confused so the the word spirit is a powerful thing but why did you say in the last year or so they discovered this do you mean the same old traditional idea of a spirit or Jamie I try to find out what people mean by spirit when people say spirituality in the u.s. it usually is the refers to the phantom limb that they developed in the absence of culture and a culture is in some sense you could say the spirit of a society that is long game this thing that it's become self-aware at a level above the individuals where you say if you don't do the following things then the grand crying crying when children of our children will not have nothing to eat yeah so if you take this long scope where you try to maximize the length of the game that you are playing as a species to realize that you're part of a larger thing that you cannot fully control you probably see to submit to the ecosphere instead of trying to completely control it right there needs to be a certain level at which we can exist as a species if you want to endure and our culture is not sustaining this anymore we basically made this bet with the Industrial Revolution that we can control everything and the modernist societies was basically unfettered growth led to a situation in which we depend on the ability to control the entire planet and since we are not able to do that as it seems this culture will die if we realize that it doesn't have a future right we called our children generations that it's not very optimistic things yeah you can have this kind of intuition that our civilization you say culture but you really mean this the spirit of the civilization do in the entirety the civilization may not exist for long yeah so what can you kion tangle that what's your intuition behind that so you you kind of offline mentioned to me that the Industrial Revolution was kind of a the moment we agreed to accept the offer sign on the paper on the dotted line with the Industrial lucien we doomed ourselves can you elaborate and this is suspicion i of course don't know how it plays out but gosh it seems to me that in society in which you leverage yourself very far over an entropic a piss without land on the other side it's relatively clear that your cantilevers at some point going to break down into this entropic abyss and you have to pay the bill okay russia is my first language and i'm also an idiot this is just two apes instead they're playing with the banana trying to have fun by talking okay and throbbing what in what's anthropic and tropic and drop and and so n tropic in the sense of entropy and all entropic that yes so this end and tropical oils the other word you have this what's that it's a big porch abyss abyss yes and tropic abyss so many of the things you say are poetic it's and often rings meb's amazing right it's miss Burrell which makes you do more poetic Wittgenstein would be proud so entropic abyss okay let's let's rewind then the Industrial Revolution so how does that get us into the entropic abyss so in some sense we burned a hundred million years worth of trees to get everybody plumbing yes and the society that we had before that had a very limited number of people so basically since 0 BC we hovered between 300 and 400 million people yes and this only changed with the Enlightenment and the subsequent Industrial Revolution and in some sense the Enlightenment a feat of rationality and also freed our norms from the pre-existing order gradually it was an process that basically have been feedback loop so it was not that just one cost the other it was a dynamic that started and the dynamic worked by basically increasing productivity to such a degree that we could fit all our children and I think the definition of property is that you have as many children as you can feed before they die which is in some sense the state that all animals on earth are in the definition of poverty is having enough so you can have only so many children as you can feed and if you have more they die yes and in our societies you can basically have as many children as you want they don't die right so I the reason why we don't have as many children as we want us because we also have to pay a price in terms of you have to insert ourselves in the lowers also tritonus yeah if you have too many so basically everybody in the under middle and lower upper class has only a limited number of children because having more of them would mean a big economic hit to the individual families yes because children especially in u.s. super expensive to have and you only are taken out of this if you are basically super rich or if you are super poor if you're super poor it doesn't matter how many kids you have because your status is not going to change and these children are largely not going to die of hunger so how does this leads us just self-destruction so there's a lot of unpleasant properties about this process so basically what we try to do is we try to let our children survive even if they have diseases it's like I would have died and before my mid-twenties without modern medicine and most of my friends would have as well and so many of us wouldn't live without the advantages of modern medicine and modern industrialized society we get our protein in largely by subduing the entirety of nature imagine there would be some very clever microbe that would live in our organisms and would completely harvest them and change them into a thing that is necessary to sustain itself and it would discover that for instance brain cells are kind of edible but they're not quite nice so you need to may have more fat in them and you turn them into more fat cells yes and basically this big organism would become a vegetable that is barely alive and it's going to be very brittle and not resilient when the environment changes yeah but some part of that organism the one that's actually doing all the using of the there's still be somebody thriving so as it relates back to this original question I suspect that we are not smartest thing on this planet I suspect that basically every complex system has to have some complex regulation if if it depends on feedback loops and so for instance it's likely to that we should describe a certain degree of intelligence to plants the problem is that plants don't have a nervous system so they don't have a way to Telegraph messages over large distances almost instantly in the plant and instead they will rely on chemicals between adjacent cells which means the signal processing speed depends on their signal processing with a rate of a few millimeters per second yes and as a result the if the plant is intelligent it's not going to be intelligent it's similar timescales yes the ability process the timescales different so you suspect we might not be the most intelligent but when were the most intelligent and this in our timescale so basically if you would room out very far you might discover that they have been intelligent ecosystems on the planet that existed for thousands of years in a almost undisturbed state and it could be that these ecosystems actively related their environment so basically change the course of the evolution within this ecosystem to make it more efficient and as brittle as possible something like plants is actually a set of living organisms an ecosystem of living organisms they're just operating a different time scale and a far superior intelligence than human beings and then human beings will die out and plus will still be there and they'll be there yeah they also there's an evolutionary adaptation playing a role at all of these levels for instance if mice don't get enough food and get stressed the next generation of mice will be more sparse in most quani and the reason for this is because they in a natural environment the mice have probably hidden a drought or something else and if they over grace then all the things that sustain them might go extinct and there will be no mice a few generations from now so and to make sure that there will be mice and five generations from now they see the mice scale back and a similar thing happens with the Predators of mice they should make sure that the mice don't completely go extinct so in some sense if the Predators are smart enough they will be tasked this shepherd their food supply may be the reason why Alliance have much larger brains and antelopes is not so much because it's so hard to catch antelope as opposed to run away from the lion but the Lions need to make complex models of their environment more complex than the antelopes so the first of all just describing that there's a bunch of complex systems and human beings may not even be the most special or intelligent to those complex systems even on earth makes me feel a little better about the extinction of human species that we're talking about yes maybe you addressed Gaia's ploy to put the carbon back into the atmosphere this is just a nice big stain on evolution is not as it was trees hers I evolved trees before they could be to adjust it again right there were no insects that would break all of them apart cellulose is so robust that you cannot get all of it with microorganisms so many of these trees fell into swamps and all this carbon became inert and could no longer be recycled into organisms and via the species that is destined to take care of that so this is kind of dig it out of the ground for the decade the atmosphere in the u.s. is already greening yeah so visitin million years or so when the ecosystems have recovered from the rapid changes yeah that they're not compared to us right now yeah this is going to be awesome again and there won't be even a memory of us of us little apes I think that will be memories of us I suspect we are the first generally intelligent species in the sense we are the first species with an industrial society because we believe more phones than bones in the stratosphere well see I have phones them bones I like it but then let me push back idea you've kind of suggested that with a very narrow definition of of until I mean why aren't trees more general a higher-level general intelligence than trees very intelligent and it would be at different time scales which means within a hundred years the tree is probably not going to make models that are as complex as the one step you make in ten years but maybe the trees are the ones that made the phones right like like you could say the entirety of life did it you know the first cell never died the first cell only split right and every divided and every cell in our body is still an instance of the first cell that split off from that a first sell it was only one sell on this planet as far as we know and so the cell is not just a building block of life it's a hypo organism yeah right and we are part of this type of organism so nevertheless this type of organism no the this little particular branch of it which is us humans because the Industrial Revolution and maybe the exponential growth of technology might somehow destroy ourselves so what what do you think is the most likely way we might destroy ourselves so some people worry about genetic manipulation some people as we've talked about worry about either dumb artificial intelligence or super intelligent artificial intelligence destroying us some people worry all nuclear weapons and weapons of war in general what do you think if you had to if you are a betting man what would you bet on in terms of self-destruction and it would be higher than 50 or to be higher than 50% so it's very likely that nothing that we bet on matters after we win our bet so I don't think that bets are literally the right thing way to go about I mean once you're dead it doesn't you you won't be there to collect so it's also not clear if we as a species go extinct but I think that our present civilization is not sustainable so the thing that will change is there will be probably fewer people on the planet NR today and even if not then still most of people that are alive today will not offering 100 years from now because of the geographic changes and so on in the change in the food supply it's quite likely that many areas of the planet will only be livable is a closed cooling chain in 100 years from now so many of the areas around the equator and in subtropical climates that are now quite pleasant to live in will stop to be inhabitable this is out everyday you honestly Wow cooling chain close knit cooling chain communities so you think you have a strong worry about the the effects of global warming itself it's not the big issue if you will live in Arizona right now you have basically three months in the summer in which you cannot be outside yes and so you have a closed cooling chain you have air conditioning in your car in your home and you're fine and if the air conditioning would stop for a few days then in many areas you would not be able to survive frankly we just pause for a second like you say so many brilliant poet ik things like what is a closed is that do people use that term closed cooling chain I imagine that people use it when they describe how they get meat into a supermarket right it could break the cooling chain and this thing's rights to saw you had trouble and you have to solve it away there's such a beautiful way to put it's like calling a city a closed social chain or something like that I mean yeah that's right I mean the locality of is really yeah but it basically means you wake up in the climatized room you go to work in the climatized car you work in the car all into the shop and acclimatized supermarket and in between you have very short distance which you run from your car to the supermarket but you have to make sure that your your temperature does not approach the temperature of the environment yeah so the usual thing is the bad pub temperature the what the best pub temperature it's what you get when you take wet clothes and you put it around your thermometer and then you move it very quickly through the air so you get the evaporation heat yes and as soon as you can no longer cool your body temperature via app evaporation to a temperature below something like I think 35 degrees you die right and which means if the outside world is dry you can still cool yourself down by sweating but if it has a certain degree of humidity or if it goes up over a certain temperature then sweating will not save you and this means you even if you're a healthy fit individual within a few hours even if you try to be in the shade and so on you'll die unless you have some climate sizing equipment and this itself if you as long as you maintain civilization and you have energy supply and you have food trucks coming to your home that are climatized everything is fine but what if you lose a large scale open every culture at the same time so basically we'll run into food insecurity because climate becomes very irregular or weather becomes very irregular and you have a lot of extreme weather events so you need to roll most of your foot maybe indoor or you need to import your food from certain regions and maybe you are not able to maintain the civilization notes without the planet to get the infrastructure to get the foot to your home right but there could be is so there could be significant impacts in a sense that people begin to suffer they could be wars over resources and so on but ultimately do you have do you not have a lot of faith but what do you make of the capacity of technology technological innovation to help us prevent some of the worst damages that this condition can create so as an example as a almost out there example is the work of SpaceX Elon Musk is doing of trying to also consider our propagation throughout the universe in deep space to colonize other planets that's one technological step but of course what Hamas is trying on Mars is not to save us from global warming because Mars looks much worse than planet Earth will look like after the worst outcomes of global warming imaginable right yes Martha said essentially not habitable it's exceptionally harsh environment yes but what he is doing what a lot of people throughout history since the Industrial Revolution are doing are just doing a lot of different technological innovation was some kind of target and one ends up happening is totally unexpected new things come up so trying to trying to terraform or trying to colonize Mars extremely harsh environment might give us totally new ideas of how to expand the or increase the power of this closed cooling circuit that that empowers the community so like do you it seems like there's a little bit of a race between our open-ended technological innovation of this communal operating system that we have and our general tendency to want to overuse resources and thereby destroy ourselves would you don't think technology can win that race I think the probability is relatively low given that our technology is Prince the u.s. is stagnating since the 1970s roughly in terms of technology most of the things that we do are the result of incremental processes sort of our Intel what about Moore's law it's basically it's very incremental the things that we're doing is so after the invention of the microprocessor was a major thing right the miniaturization of transistors was really major but the things that we did afterwards largely were not that innovative trifle changes of scaling things into a foams GPUs into from CPUs into GPUs and things like that but I don't think that there are basic they're not many things if you take a person that died in the 70s and was at the top of that game they would not need to read that many books to all be current again but it's all about books who cares about books so the there might be things that are beyond what books might be every papers or no papers forget papers there might be things that are so papers and books and knowledge that's a that's a concept of a time when you were sitting there by candlelight and individual consumers of knowledge what about the impact that you we're not in the middle of we're not might not be understanding of Twitter of YouTube the reason you and I are sitting here today is because of Twitter and YouTube yes so the the ripple effect and there's there's two minds sort of two dumb apes coming up with the new perhaps a new clean insights and there's 200 other apes listening right now 200,000 other Apes listening right now and that effect it's very difficult to understand what that effect will have that might be bigger than any of the advancements of the microprocessor Ernie the Industrial Revolution the ability of spread knowledge and that that the the that knowledge the like it allows good ideas to reach millions much faster and the effect of that that might be the new that might be the 21st century is the multiple the multiplying of ideas of good ideas because if you say one good thing today that will multiply across you know huge amounts of people and then they will say something and then they'll have another pocket and I'll say something and then I'll write a paper that that could be a huge you don't think that yeah if you should have billion fun for Normans right now often omens right now in two rings and we don't for some reason I suspect the reason is that we destroy our attention span also the incentives of course different but in Cardassians yeah so the reason why we are sitting here and doing this as a YouTube video is because you and me don't have the attention span to write a book together right now and you guys probably don't have the attention span to read it so let me tell you but we're you know we're an hour and 40 minutes in and I guarantee you that 80% of the people are still listening so there's an attention span it's just the the forum you know who said that the book is the optimal way to transfer information that's said this is still an open question I mean that's what we're something that social media could be doing that other forms could not be doing I think the end game of social media is a global brain and Twitter is in some sense a global brain that is completely hooked on dopamine doesn't have any kind of inhibition and as a result is caught in a permanent seizure yes it's also in some sense a multiplayer role-playing game and people use it to play an avatar that is not like them as the Verna's sane world and they look through the world through the lens of their phones and think it's the real world but it's the Twitter of all that is thwarted by the popularity incentives of Twitter yet the the incentives and just our natural biological the the dopamine rush of alike no matter how like I consider I try to be very kind of zen-like and minimalist and not being influenced by likes and so on but it's probably very difficult to avoid that to some degree the speaking at a small tangent of Twitter what how can be how can Twitter be done better I think it's an incredible mechanism that has a huge impact on society by doing exactly what you're doing oh sorry doing exactly you described which is having this but we're like is this some kind of game and we're kind of our individual RL agents in this game and it's uncontrolled because there's not really a centralized control neither jack dorsey nor the engineers at twitter seem to be able to control this game or can they that's sort of a question is there any advice you would give and control is advice because I am certainly not an expert but I can give my thoughts on this and I our brain is has solved this problem to some degree right our brain has lots of individual agents that manage to play together anyway and you have also many contexts in which other organisms have found ways to solve the problems of cooperation that we don't solve on Twitter and maybe the solution is to go for an evolutionary approach so imagine that you have something like reddit or something like Facebook and something like Twitter and do you think about what they have in common what they have in common they're companies that in some sense own a protocol and this protocol is imposed on a community and the protocol has different components for monetization for a user management for user display for rating for anonymity for importer of other content and so on and now imagine that you take these components of the protocol apart and you do it in some sense like communities visiting this social network and these communities are allowed to mix and match their protocols and design new ones so for instance the UI and the UX can be defined by the community the walls for sharing content across communities can be defined the monetization can be redefined the way you reward individual users for what can be redefined the way users can represent themselves and to each other can redefined and will be the redefine er so it can individual human beings build enough intuition to redefine those things if self can become part of the protocol so for instance it could be in some communities it will be a single person that comes up with these things and others it's a group of friends some might implement a voting scheme that has some interesting weighted voting who knows who knows what will be the best self organizing principle for this but the process can be automated I mean it seems like the brain can be automated so people can write a software for this and eventually the idea is let's not make a assumption about this thing if you don't know what the right solution is in those areas that we have no idea whether the right solution will be people designing this ad hoc or machines doing this whether you want to enforce compliance by social norms like weak Orvis software solutions or this AI that goes through the post of people or is a legal principle and so on this is something maybe you need to find out and so the idea would be if you let the communities evolve and you just control it to say in such a way that you are incentivizing the most sentient communities hmm the ones that produce the most interesting behaviors and that allow you to interact in the most helpful ways to the individuals right so you have a network that gives you information that is relevant to you it helps you to maintain relationships to others in healthy ways it allows you to build teams it allows you to basically bring the best of you into this thing and goes into a coupling into a relationship with others in which you produce things that you would be unable to produce alone yes beautifully put so but the key process of that with incentives and evolution is things that don't adapt themselves to effectively get the incentives have to die and the thing about social media is communities that are unhealthy or whatever you want and it defines the incentives really don't like dying one of the things that people really get aggressive protests aggressively is when they're censored especially in America I don't know I don't know much about the rest of the world but the idea of freedom of speech the idea of censorship is really painful in America and so what yeah well what do you think about that have been growing up in East Germany what do you think censorship is an important tool in our brain in the intelligence and in the social networks so basically if you're not a good member of the entirety of the system they should be blocked away well locked away blocked important thing is who decides that you are a good member who is it distributed or and what is the outcome of the process that decides it both for the individual and for society at large for instance if you have a high trust society you don't need a lot of surveillance and the surveillance is even in undermining trust yes because it's basically punishing people that look suspicious when surveyed but do the right thing anyway and the opposite if you have a low trust society in there and surveillance can be a better trade-off and the u.s. is currently making a transition from a relatively high trust a mixed trust society to a low trust society so surveillance will increase another thing is that beliefs are not just Inuit representations there are implementations that run code on your brain and change for a reality and change the way you interact with each other at some level and some of the beliefs are just public opinions that we use to display our alignment so for instance people might say all characters have are the same and equally good but still they prefer to live in some cultures over others very very strongly so and it turns out that the cultures are defined by certain rules of interaction and these rules of interaction lead to different results when you implement them right so if you adhere to certain rules you get different outcomes in different societies and this all leads to very tricky situations when people do not have a commitment to shared purpose and our societies what we need to rediscover what it means to have a shared purpose and how to make this compatible with a non totalitarian view so in some sense the u.s. is caught in a conundrum between totalitarianism and diversity and doesn't it need to how to resolve this and the solutions that the u.s. has found so far a very crude because it's a very young society that is also under a lot of tension it seems to me that the US will have to reinvent itself what do you think just uh philosophizing what kind of mechanisms of government do you think we as a species should be involved with us or broadly what do you think will work well as a system of course we don't know it all seems to work pretty crapoly some things worse than others some people argue that communism is the best others say yeah look at the Soviet Union some people argue that anarchy is the best and then completely discarding the positive effects of government you know there's a lot of argument u.s. seems to be doing pretty damn well in in the span of history there says respect for human rights which seems to be a nice feature not a bug and economically a lot of girls law technological development people seem to be relatively kind and the grand scheme of things what lessons do you draw from that what kind of government system do you think is good ideally a government would not be perceivable all right it should be frictionless the more you notice the influence of the government the more friction you experience the less effective and efficient the government probably is right so a government game theoretically is an agent that imposes an offset on your payout metrics to make your Nash equilibrium compatible with the common good right so you have these situations and these local incentives everybody does the thing that's locally the best for them but the global outcome is not good and this is even the case when people care about the global outcome because a regulation mechanisms exist that creates a course of relationship between what I want to have for the global good and what I do so for instance if I think that we should fly less and I stay at home there is not a single plane that is going to not start because of me right it's not going to have an influence but I don't get from A to B so the way to implement this would be to have a government that is sharing this idea that you should fly less and is then imposing a regulation that for instance makes flying more expensive and it gives incentives for inventing other forms of transportation that are less putting the strain on the environment for instance so there's so much optimism and so many things you described and yet there's the pessimism of you think our civilization is gonna come to an end so that's not a hundred percent probability nothing in this world is so what's the trajectory out of self-destruction do you think I suspect that in some sense we are both too smart and not smart enough which means we are very good at solving near-term problems and at the same time we are unwilling to submit to the end to the imperatives of that we would have to follow in if you want to stick around right so that makes it difficult if you were unable to solve everything technologically you can probably understand how it the child mortality needs to be to absorb the mutation rate and tell why the mutation mutation rate needs to be to adapt to a slowly changing ecosystemic environment right so you could in principle compute all these things game theoretically and adapt to it but if you all cannot do this because you are like me and you have children you don't want them to die you will use any kind of medical information to keep travel to a mortality low even if it means that our visit a few generations we have enormous genetic drift and most of us have allergies as a result of not being adapted the changes that we made to our food supply that's for now I say technologically speaking which is a very very very young you know 300 years industrial revolution we're very new to this idea so you're attached to your kids being alive and not being murdered for the greater good of society but that might be a very temporary moment of time yes that we might move might evolve in our thinking so like you said when we're both smart and not smart enough you're probably not this first human civilization that has discovered technology that allows to efficiently over grace our resources and this overgrazing is think at some point we think they can compensate this because if we have eaten all the grass we will find a way to grow mushrooms right but it could also be that the ecosystems tip and so what really concerns me is not so much the end of the civilization because we will invent a new one but what concerns me is the fact that for instance the oceans might tip so for instance maybe the plankton dies because of ocean acidification and cyanobacteria take over and as a result we can no longer raise the atmosphere this would be really concerning so basically a major reboot of most complex organisms on earth and I think this is a possibility I don't know if what the percentage for this possibility is but it doesn't seem to be our language to me if you look at the scale of the changes that we've already triggered on this planet and so Danny Hillis suggests that for instance we may be able to put chalk into the stratosphere to solar radiation maybe it works maybe there's a sufficient to counter the effects of what we've done maybe it won't be maybe we won't be able to implement it by the time it's prevalent I have no idea how how the future is going to play out in this regard it's just I think it's quite likely that we cannot continue like this all our cousin species the other home units are gone so the right step would be to what to rewind rewind towards a destro Revolution and slow the the so it's to try to contain the technological process that leads to the overconsumption of resources imagine you had get to choose you have one lifetime yes you get born into a sustainable agricultural civilization 300 maybe 400 million people on the planet tops or before this some kind of nomadic species feels like a million or two million and so you don't meet new people unless you give birth to them you cannot travel to other places in the world there is no internet there is no interesting intellectual tradition that reaches considerably deep so you would not discover your own completeness probably and so on so we wouldn't exist and the alternative is you get born into an insane world one that is doomed to die because it has just burned 100 million years worth of trees in a single century which one do you like I think I like this one it's a very weird thing then when you find yourself on a Titanic and you see this iceberg and it looks like we are not going to miss it and a lot of people are in denial and most of the counter arguments sound like denial to me that don't seem to be rational arguments and the other thing is we were born on this Titanic without this Titanic we wouldn't have been born we wouldn't be here we wouldn't be talking we wouldn't be on the internet we wouldn't do all the things that we enjoy and if you're not responsible for this happening it's basically if he had the choice we would probably try to prevent it but when we were born we were never asked when we want to be born in which society we want to be born but incentive structures we want to be exposed to we have relatively little agency in the entire thing humanity has relatively daily machine the whole thing it's basically a giant machine it's tumbling down a hill and everybody is Fanta Klee trying to push some buttons nobody knows what these buttons are meaning what they connect - and most of them are not stopping this tumbling down the hill is impossible the artificial intelligence will give us an escape latch somehow so the you know there's a lot of worry about existential threats of artificial intelligence but what AI also allows in general forms of automation allows the potential of extreme productivity growth that will also perhaps in a positive way transform society that may allow us to inadvertently to return to the more to the same kind of ideals of closer to nature that's represented in hunter-gatherer societies you know that's not destroying the planet that's not doing overconsumption and so on I mean generally speaking do you have hope that a I can help them uh I think it is not fun to be very close to nature until you completely subdue nature so our idea of being close to nature means being close to agriculture basically forests that don't have anything in them that eats us see I mean I want to disagree with that I I think the niceness of being close to nature is to being fully present and in like Wirthlin survival becomes your primary not just your goal but your whole existence mm-hmm it I mean that is a in I'm not just romanticizing I can just speak for myself I am self-aware enough that that is uh that is a fulfilling existence that's one that's very to be in nature ah and not fight for my survival I think fighting in yourself for your survival well being in the cold and in the rain and being hunted by animals and having open wounds it's very unpleasant well there's a contradiction in there yes I in you just as you said would not choose it but if I was forced into it it would be a fulfilling existence Lemar adapted to it basically if your brain is fed up in such a way that you get rewards optimally in such an environment and there's some evidence for this that for a certain degree of complexity basically people are more happy in such environment because it's what we largely have evolved for in between we had a few thousand years in which I think we have evolved for a slightly more comfortable environment so there is probably something like an intermediate stage in which people would be more happy than there would be if they would have to fend for themselves in small groups in the forest and often die versus something like this very now have basically a big machine a big of Mordor in which we run concrete boxes and press buttons and machines and largely don't feel well cared for as the monkeys that we are so returning briefly to not briefly but returning to AI what let me ask a romanticized question what is the most beautiful - you silly ape the most beautiful or surprising idea in the development of artificial intelligence well there in your own life or in the history of artificial intelligence that you've come across if you built an AI it probably can make models at an arbitrary degree of detail right of the world and then it would try to understand its own nature it's tempting to think that at some point when we have general intelligence we have competitions very evil that the AIS wake up in different kinds of physical universes and we measure how many movements of the rubik's cube it takes until it's figured out what's going on in its universe and what it is and its own nature and its own physics and so on right so what if we exists in the memory of an AI that is trying to understand its own nature and remembers its own genesis and remembers lex and Yasha sitting in hotel sparking some of the ideas of that led to the development of general each other so we're a kind of simulation is running in an AI system is trying to understand itself it's not that I believe that but as I think it's a beautiful I mean it you kind of return to this idea with the Turing test of intelligence being of intelligence being the process of asking and answering what is intelligence I mean what why do you think there's there is an answer what why is there such a search for an answer what so does there have to be and I can I can answer you just had an AI system that's trying to understand the why of what you know understand itself is that a fundamental process of greater and greater complexity greater greater intelligence is the continuous trying of understanding itself no I think you will find that most people don't care about that because they're well adjusted enough to not care and the reason why people like you and me occur about it probably has to do with the need to understand ourselves it's because we are in fundamental disagreement is the universe that we wake up in what looks like me and I see oh my god I'm caught in a monkey what's that sorry that's the feeling right it's just the government and I'm unhappy with the entire universe that I fight myself in so you don't think that's a fundamental aspect of human nature that some people are just suppressing that they're they wake up shocked they're there in the body of a monkey no there is clear adaptive value to not be confused by that and by well no no that's our air so oh so you have to clear adaptive value then there's clear adaptive value to while fundamentally your brain is confused by that by creating an illusion another layer of the narrative that says you know that tries to suppress that and instead say that you know what's going on with the government right now is the most important thing what's going on with my football team is the most important thing but it seems to me the like I would like for me it was a really interesting moment reading Ernest Becker's denial of death the you know there's this kind of idea that we're all you know the fundamental thing from which most of our human mind Springs is this fear of mortality being cognizant of your mortality and the fear of that mortality and then you construct illusions on top of that I guess I'm you being just a push on it you you really don't think it's possible that this worry of the big existential questions is actually fundamental as of as the existentialist thought to our existence I think that the fear of death only plays a role as long as you don't see the big picture the thing is that Minds our software States right software doesn't have identity software in some sense is a physical law but if last like a brief yeah right so but it feels like there's an identity I thought that was the for this particular piece of software and then narrative it tells there's a fundamental property of assigning it maintenance of the identity is not terminal it's instrumental to something else you maintain your identity so you can serve your meaning so you can do the things that you're supposed to do before your bad died and I suspect that for most people the fear of death is the fear of dying before they're done with the things that they feel they have to do even though they cannot quite put their finger on it what it is what that is right but in the software world okay they return to the question then what happens after we die because what you care you will not be longer there the point of trying is that you're gone or maybe I'm not and this is what you know it it seems like there's so much any idea that this is just the mind is just the simulation is constructing a narrative around some particular aspects of the quantum mechanical wave function world that we can't quite get direct access to then like the idea of mortality seems to be a little fuzzy as well it doesn't maybe there's not a clear and the quasi idea is the one of continued existence we don't have continuous existence how do you know that like that it's not computable because you're saying it's good it's no process the only thing that binds you together with the leg Sweetman from yesterday is the illusion that you have memories about him so if you want to upload it's very easy you make a machine that thinks it's you because this the same thing that you are you are a machine that thinks it's you but that's that's more and that's immortality yeah but it's just a belief you can create this body very easily once you realize that the question whether you are immortal or not depends entirely on your beliefs and your own continuity but then it then then you can be immortal by the continuity of the belief you cannot be immortal but you can stop being afraid of your mortality because you realize you were never continued ously existing in the first place well I don't know if I'd be more terrified or less terrified with that it seems like the fact that I existed also you don't know this state in which you don't have itself you can turn off yourself you know I can't turn you can turn it off you can turn it off I can yes and you can basically meditate yourself in a state where you are still conscious there's still things are happening where you know everything that you knew before but you no longer identified was changing anything and this means that yourself and way it dissolves there is no longer this person you know that this person construct exists in other states and it runs on this brain of legs Freedman but it's it's not a real thing it's a construct it's an idea and you can change that idea and if you let go of this idea if you don't think that you are special you realize it's just one of many people and it's not your favorite person even right it's just one of many and it's the one that you are doomed to control for the most part and that is basically informing the actions of this organism yeah as a control model and this is all there is and you are somehow afraid that this control model gets interrupted or loses the identity of continuity yeah so I'm attached I mean yeah there is a very popular it's a somehow compelling notion that being being attached like there's no need to be attached to this idea of an identity but that in itself could be a an illusion that you construct so the process of meditation while popular is thought of as getting under the concept of identity it could be just putting a cloak over it just telling it to be quiet for the moment you know it I think that meditation is eventually just a bunch of techniques that let you control attention and when you can control the attention you can get access to your own source code hopefully not before you understand what you're doing and then you can change the way it works temporarily or permanently so yeah meditations in get a glimpse at the source code get under there so basically how much role or is it that you learn to control attention so yeah everything else is downstream from controlling attention and control the attention that's looking at the attention not only only get attention in the parts of our mind that create heat where you have a mismatch between model and the results that are happening and so most people are not self-aware because their control is too good if everything works out roughly the way you want and the only things that don't work out is whether your football team vents then you will mostly have models about these domains and it's only when for instance your fundamental relationships to the world around you don't work because the ideology of your country is insane and the other kids are not nerds and don't understand why you understand physics and you don't why you want to understand physics and you don't understand why somebody would not want to understand physics so we kind of brought up neurons in the brain as reinforcement learning agents and there's been some successes as you brought up with go with alpha go alpha zero with ideas of self play which I think are incredibly interesting ideas those systems playing each other and in an automated way to improve by playing other systems of in a particular construct of a game that are a little bit better than itself and then thereby improving continuously all the competitors in the game are improving gradually so being just challenging enough and from learning from the process of competition do you've hoped for that reinforcement learning process to achieve greater and greater level of intelligence so we talked about different ideas in AI then we need to be solved is RL a part of that process of trying to create a GI system so it forms of unsupervised learning but the many algorithms that can achieve that and I suspect that ultimately the algorithms that work there will be a class of them or many of them and they might have small differences of like a magnitude in efficiency but eventually what matters is the type of model that you form and the types of models that we form right now are not sparse enough just bars that what does it mean to be sparse so it means that ideally every potential model State should correspond to a potential world state so if I see if you vary States in your model you always end up as valid world States and all mind is not quite there so an indication especially what we see in dreams the older we get the more boring our dreams become because we incorporate more and more constraints that we learned about how the world works so many of the things that we imagined to be possible as children turn out to be constrained by physical and social dynamics and as a result fewer and fewer things remain possible it's not because our imagination scales back but the constraints under which it operates become tighter and tighter and so the constraints under which our neural networks operate are almost limitless which means it's very difficult to get a neural network to imagine things that look real right so as a SPECT part of what we need to do is we probably need to build dreaming systems I suspect that part of the purpose of dreams is to similar to a generative adversarial network to learn certain constraints and then it produces alternative perspectives on the same set of constraints so you can recognize it under different circumstances maybe we have flying dreams as children because we recreate the objects that we know on the maps that we know from different perspectives which also means from the bird's eye perspective so I mean aren't we doing that anyway I mean not without with our eyes and with our eyes closed and when we're sleeping are we just constantly running dreams and simulations in our mind as we try to interpret the environment I mean it's sort of considering all the different possibilities there's the way we interact with the environment it seems like essentially like you said of creating a bunch of simulations that are consistent with our expectations with previous experiences with the things we just saw recently and through that hallucination process we are able to then somehow stitch together what actually we see in the world with the simulations that match it well and thereby interpret it I suspect it you're in my brain are slightly unusual in this regard which is probably what got you into MIT so this obsession of constantly pondering possibilities and solutions to problems I'll stop I think I I'm not talking about intellectual stuff I'm talking about just doing the kind of stuff it takes to walk and not fall I guess this is largely automatic yes but the process is mean it's not complicated it's very easy to pull the neural network that in some sense learns the dynamics the fact that we haven't done it right so far it doesn't mean it's hard because you can see that a biological organism does it there's relatively few neurons yeah as you build a bunch of neural oscillators that in train themselves this the dynamics of your body in such a way that the regulator becomes isomorphic and it's modeled through the dynamics that are regulates and then is automatic and it's only interesting the sense that it captures attention when the system is off see but thinking of the kind of mechanism that's required to do walking as a controller as like a as a neural network I think I think it's a compelling notion but it's discards quietly or at least makes implicit the fact that you need to have something like common sense reasoning to walk it's not as an open question whether you do or not but my intuition to be to act in this world there's a huge knowledge base that's underlying it somehow there's so much information of the kind we have never been able to construct in our in your networks on an artificial intelligence systems period which is like it's humbling at least in my imagination the amount of information required to act in this world humbles me and I think saying that your levels can accomplish it is missing is missing the fact that we don't yeah we don't have yet a mechanism for constructing something like common sense reasoning I mean what's your sense about to linger on how much if you know to linger on the idea of what kind of mechanism would be effective at walking you said just a neural network not maybe the kind we have but something a little bit better we'll be able to walk easily don't you think it also needs to know like a huge amount of knowledge that's represented under the flag of common sense reasoning how much common sense knowledge to be actually have imagine that you are pretty hard working through all your life and you form two new concepts every half hour or so yes you end up with something like a million concepts because you don't get that old so a million concept that's not a lot it's not just a million concepts I think you'll be a lot I personally think it might be much more than a million if you think just about the numbers you don't live that long if you think about how many cycles do your neurons have in your life it's quite limited you don't get that all yeah but the the powerful thing is a number of concepts and they're probably deeply hierarchical in nature the relations as you described between them is the key thing so it's like even if it's the chameleon concepts the the graph of relations that's formed and some kind of perhaps some kind of probabilistic relationships that's the that's what's common-sense reasoning is a relationship between things that yeah so but in some sense I think of the concepts as the space for our behavior programs and the waiver poems allow us to recognize objects and interact with them also mental objects and a large part of that is the physical world that we interact with which is this res extend Lansing which is basically navigation of information in space and basically it's similar to a game engine it's a physics engine that you can use to describe and predict how things that look in a particular way that feel when you touch them particular way they love proprioception I love auditory perception and so on how they work out so basically the geometry of all these things and this is probably 80% of what our brain is doing is dealing with debt with this real-time simulation and by itself a game engine is fascinating but it's not that hard to understand what it's doing right and our game engines are already in some sense approximating the Magna deep fidelity of what we can perceive so if we put on an oculus quest we get something that is still qualitatively crude with respect to what we can perceive but it's also in the same ballpark already right it's just a couple order of magnitudes away to home saturating our perception jumps of the complexity that it can produce so in some sense it's reasonable to say that our the computer that you can buy it the put into your home is able to give a perceptual reality that has a teacher that is already in the same ballpark as what your brain can process and everything else our ideas about the world and I suspect that they are relatively sparse and also the intuitive models that we form about social interaction social interaction is it's not so hard it's just hard for us nerds because we all have our wires crossed so we need to use them but the priors are present in most social animals so it's interesting thing to notice that many domestic social animals like cats and dogs have better social cognition than children right I hope so I hope it's not that many concepts fundamentally and - due to existence world social sorry it's more like I'm afraid so because this thing that we only appear to be so complex to each other because we are so stupid it's a little bit interesting now one that yeah to me that's inspiring if we're indeed as as as stupid as it seems so thinks our brains don't scale and the information processing that we built tend to scale very well yeah but I mean one of the things that worries me is that the you know that the fact that the brain doesn't scale means that that's actually a fundamental feature of the brain you know the all the flaws of the brain everything we see that we see has limitations perhaps there's a fundamental the constraints on the system could be the requirement of its power which is like different than our current understanding of intelligent systems were scale especially with deep learning especially with reinforcement learning the hope behind open a eye deep mind all the major results really have to do with huge compute and it also be that our brains are so small not just because they take up so much glucose in our body like 20% of the glucose so they don't arbitrarily scale there's some animals like elephants which have larger brains than us and atoms need to be smarter all right elephants seem to be autistic they have very very good motor control and they're really good with details but they really struggle to see the big picture so you can make them recreate drawings stroke by stroke they can do that but they cannot reproduce a still life so they cannot make a drawing of a scene that I see there will always be only able to reproduce the line drawn at least as far away from what I could see in the experiments yeah by is that maybe smarter elephants would meditate themselves out of existence because their brains are too large so they basically the elephants that were not autistic they didn't reproduce yet so we have to remember that the brain is fundamentally interlinked with the body and our human and biological system do you think that a GI systems that we try to create or greater intelligence systems would need to have a body so I think that should be able to make use of a body if you give it to them but I don't think that a fundamentally new body so I suspect if you can interact with the world by moving your eyes and your head you can make controlled experiments and this allows you to have many magnitudes fewer observations in order to reduce the uncertainty in your models alright so you can pinpoint the areas in your models but you're not quite sure and you just move your head and see what's doing what's going on over there and you get additional information if you just have to use YouTube as an input and you cannot do anything beyond this you probably need just much more data but if we have much more data so if you can build a system that has enough time and attention to browse all of YouTube and extract all the information that there is to be found I don't think that's an obvious limit to what it can do yeah but it seems that the interactivity is a fundamental thing that the physical body allows you to do but let me ask on that topic sort of that does what a body is is allowing the brain to like touch things and move things and interact with the weather the physical world exists or not whatever but interact with someone interface to the physical world what about a virtual world do you think do you think we can do the same kind of reasoning consciousness intelligence if we put on a VR headset and move over to that world do you think there's any fundamental difference between the interface the physical world that is here in this hotel and if we were sitting in the same hotel in a virtual world the question is just as physical this non-physical world with this other environment near entice you to solve problems that require general intelligence if it doesn't then you probably will not develop general intelligence and arguably most people are not genuinely intelligent because they don't have to solve problems that make them generally intelligent and even for us it's not yet clear if we are smart enough to put AI and understand our own nature to this degree right so it could be a matter of capacity and for most people it's in the first place a matter of interest they don't see the point because the benefit of attempting this project are marginal because you're probably not going to succeed in it and the cost of trying to do a requires complete dedication of your entire life all right but it seems like the possibilities of what you can do in a virtual world so imagine a cut is much greater than you can in the real world so imagine a situation maybe interesting option for me if somebody came to me and offered what I'll do is yeah so from now on you can only exist in the virtual world and so you put on this headset and when you eat we'll make sure to connect your body up in a way that when you eat in the virtual world your body will be nourished in the same way in the virtual world so the aligning incentives between the our common sort of real world in the virtual world but then the possibilities become much bigger like I could be other kinds of creatures I could do I can break the laws of physics as we know them I can do a lot I mean the possibilities are endless right it that's as far as we think it's an interesting thought whether like what existence would be like what kind of intelligence would emerge there what kind of consciousness what kind of maybe greater intelligence even me and me Lex even I'm at this stage in my life if I spend the next 20 years in our world to see how that intelligence emerges and if I was if that happened at the very beginning before I was even cognizant of my existence in this physical world it's interesting to think how that child would develop and the way virtuality and digitization of everything is moving it's not completely out of the realm of possibility that we're all that some part of our lives will be if not entirety of it we'll live in a virtual world to a greater degree than we currently have living on Twitter and social media and so on do you have I mean it does something draw you intellectually or naturally in terms of thinking about AI to this virtual world we're more possible easier I think that currently it's a waste of time to deal with the physical world before we have mechanisms that can automatically learn how to deal with it the body gives you a second order agency but you conserve what constitutes the body is the things that you can indirectly control I third or our tools right and the second order is the things that are basically always present but you operate on them with first order things which are mental operators yes and the zero order is in some sense the direct sense of what you are deciding right so you in you observe yourself initiating an action there features but that you interpret as the initiation of an action then you are the operations that you perform to make that happen and then you see the movement of your limbs and you learn to associate those and thereby model your own agency over this feedback right but the first feedback that you get is from this first order thing already basically you decide to think a thought and the thought is being thought you decide to change the thought and you observe how the thought is being changed yes and in some sense this is you could say an embodiment already right and I suspect it's sufficient as an embodiment really origins and so it's not that important at least at this time to consider variations in the second order yes but the thing that you also put a mentioned just now as physics that you could change in any way you want so you need an environment that puts up resistance against you if you if there's nothing to control you cannot make models right there needs to be a particular way that resists you and by the way your motivation is usually outside of your mind it resists your motivation is what gets you up in the morning even though it would be much less work to stay in bed and so it's basically forcing you to resist the environment and it forces your mind to serve it to serve this resistance to the environment so in some sense it is also putting up resistance against the natural tendency of the mind to not do anything yeah but some of that resistance just like you described as motivation is like in the first order space in the mind some resistance is in the second order like the actual physical objects pushing against you so ah yeah it seems that the second order stuff and virtuality could be recreated of course but it might be sufficient that you just do mathematics and mathematics is already putting up enough resistance against you so basically just visiting a static motive this could may be sufficient to form a type of intelligence it would probably not be a very human intelligence but it might be one that is already general so to to mess with this 0th order may be first order what do you think about ideas of brain computer interfaces so again returning to our friend Elon Musk and your link a company that's trying to of course there's a lot of trying to cure diseases and so on with a near term but the long term vision is to add an extra layer to so basically expand the capacity of the brain and connected to the computational world aha do you think one that's possible - how does that change the fundamentals of the zeroth order in the first order it's technically possible but I don't see that the FDA would ever allow me to drill holes on my skull to interface my neocortex the veyron mask envisions so at the moment I can do horrible things to mice but I'm not able to do useful things to people except maybe at some point down the line in medical applications so this thing that we are envisioning which means recreational and creational brain computer interfaces are probably not going to happen in the present legal system I love it how I'm asking you out there philosophical and sort of engineering questions and for the first time ever he jumped to the legal FDA there would be enough people that would be crazy enough to have holes drilled in their skull to try a new type of brain computer interface but also if it works it FDA will approve it I mean it's the yes you're it's like exert on most vehicles yes you can say that's gonna be very difficult regulatory process of approving with honesty but it doesn't mean autonomous vehicles are never gonna happen so no devil totally happen as soon as we create jobs for at least we lawyers and one regulator per car yes lawyers that's actually like lawyers is the fundamental substrate of reality it's a very good system it's not Universal in the world the law is a very interesting software once you realize it right these circuits are in some sense streams of software and this is largely works by exception handling so you make decisions on the ground and they get synchronized with the next level structure as soon as an exception as being wrong is a yeah so so isolates the exception handing the process is very expensive especially since it's incentivizes the lawyers for producing lot of work for lawyers yes so the exceptions are actually incentivize for for firing often but but to return outside of lawyers is there anything fundamentally like is there anything interesting insightful about the possibility of this extra layer of intelligence a little rain I do think so but I don't think that you need technically invasive procedures to do so we can already interface with other people by observing them very very closely and getting in some kind of empathetic resonance and I'm a nerd so I'm not very good at this but I noticed that people are able to do this to some degree and it basically means that we model an interface lay off the other person in real time and it works despite our neurons being slow because most of the things that we do are built on periodic process these two just need to entrain yourself with the oscillation that happens and if the Association itself changes slowly enough you can basically follow along right but the bandwidth of the interaction the you know it seems like you can do a lot more computation one yes of course the but the other thing is that the event was that our brain our own mind is running on is actually quite slow so the number of thoughts that I can productively think in any given day is quite limited but it's much if they had the discipline to write it down and the speed to write it down maybe it would be a book every day or so but if you think about the computers that we can build the magnitudes at which they operate right this would be nothing it's something that it can put out in a second well I don't know so as possible sort of the number of thoughts you have in your brain is it could be several orders of magnitude higher than what you're possibly able to express through your fingers or through your voice like so most of them are going to be repetitive because they how do you know that I mean they have to control the same problems every day when I walk they are going to be processed this in my brain that model my walking pattern and regulate them and so on but it's going to be pretty much the same every day but that movies every step but I'm talking about intellectual reasoning like thinking so the question what is the best system of government so you sit down and start thinking about that one of the constraints is that you don't have access to a lot of like you you don't have access to a lot of facts a lot of studies you have to do you always have to interface with something else to learn more to to aid in your reasoning process if you can direct access all over Kapadia in trying to understand what is the best form of government then every thought won't be stuck in a loop like every thought that requires some extra piece of information will be able to grab it really quickly that that's the possibility of if the bottleneck is literally the information that you know the bottleneck of breakthrough ideas is just being able to quickly access huge amounts of information then the possibility of connecting your brain to the computer could lead to totally new like you know totally new breakthroughs you can think of mathematicians being able to you know just up the orders of magnitude of power in their reasoning about that matter what humanity has already discovered the optimal form of government to a revolutionary process is an evolution and so what we discover is that maybe the problem of government doesn't have stable solutions for us right as a species because we are not designed in such a way that we can make everybody conform to them so but there could be solutions that work under given circumstances or that are the best for certain environment and depends on for instance the primary forms of ownership and the means of production so if the main means of production is lent then the forms of government will be regulated by the landowners and you get a monarchy if you also want to have a form of government in which a subset you depend on some form of slavery for instance where the peasants have to work very long hours for very little gain so very few people had enough plumbing then maybe you need to promise them that we had paid in in the afterlife there over time right so you need a theocracy and so for much of human history in the West we had a combination of monarchy and theocracy that was our form of governance right at the same time the Catholic Church implemented game theoretic principles I recently reread Thomas or kindness it's very interesting to see this because he was not duelist he was translating Aristotle in a particular way for the designing an operating system for the Catholic society and he says that basically people our animals and very much the same way as Aristotle envisions which basically organisms with cybernetic control and then he says that there are additional rational principles that humans can discover and everybody can discover them so they are universal if you are saying you should understand you should submit to them because you can rationally deduce them and these principles are roughly you should be willing to self-regulate correctly you should be willing to do correct social regulation it's intro organismic you should be willing to act on your models so we have skin in the game and you should have called rationality you should be choosing the right to calls to work on and so basically these three rational principles call rationality he calls prudence or wisdom the social regulation is justice the correct social one and the internal regulation is temperance and this thing to be willingness to act on your models is courage and then he says that they are additionally to these four cardinal virtues three divine virtues and these three divine virtues cannot be resonated used but they reveal themselves by the harmony which means if you assume them and you extrapolate what's going to happen you will see that that makes sense and it's often been misunderstood as God has to tell you that these are the things so they're a see there's something nefarious going on with the christian conspiracy forces you to believe some guy with a long beard that they discovered this but so these principles are relatively simple again you need it's for high level organization for the resulting civilization that you form commitment to unity so basically you serve this higher larger thing this structural principle on the next level and he calls that phase then there needs to be a commitment to shared purpose this is basically this global reward that you try to figure out what that should be and now you can facilitate this and this is love the commitment to shared purpose is the core of love right you see the sacred thing that is more important than your own organism ayk interests in the other and you serve this together and this is how you see the sacred and the other and the last one is hope which means you need to be willing to act on that prayer principle without getting rewards in the here and now because it doesn't exist yet then you start out building the civilization right so you need to be able to do this in the absence of its actual existence yet so it can come into being so yes so the way it comes into being is by you accepting those notions and then you see there these these three divine concepts then you see them and realized now the most divine is the loaded concept and olive oil and ice because we are outside of this cart and we are still scarred from breaking free of it but the idea is basically we need to have a civilization that acts as an intentional agent like an insect State and we are not actually a tribal species we are state building species and was what enabled State Building is basic the formation of religious states and other forms of rule-based administration in which the individual doesn't matter as much as the rule or the higher goal right we got there by the question what's the optimal form of governance so I don't think that chaos or Catholicism is the optimal form of governance because it's obviously on the way out right so it is for the present type of society that we are in religious institutions don't seem to be optimal to organize that so what we discovered right now that we live in in the West is democracy and democracy is the rule of oligarchs there are the people that currently own the means of production that is administered not by the oligarchs themselves because they there's too much distraction right here so much innovation that we have in every generation new means of production let me invent and corporations dive usually after 30 years or so and something either takes the leading role in our societies so it's administered by institutions and these institutions themselves are not elected but they provide continuity and they are led by electable politicians and this makes it possible that you can adapt to change without having to kill people right so you can tell for instance of a change in government's if people think that the current government is too corrupt or is not up-to-date you can just elect new people or if a journalist finds out something inconvenient about the institution and the institution is has no plan B like in Russia the journalist has to die this is what but when you run society by the deep state so ideally you have a administration layer that you can change if something bad happens right so you will have a continuity in the whole thing and this is the system that we came up in in the West and the way it's set up in the US is largely result of low-level models was mostly just second third order consequences that people are modeling in the design of these institutions it's relatively young society that doesn't really care take care of the downstream effects of many of the decisions that are being made and I suspect that AI can help us this in a way if you can fix the incentives the Society of the u.s. is a society of teeters it's basically cheating so indistinguishable from innovation and we want to encourage innovation can you elaborate on what you mean by cheating especially people do things that they know are wrong it's acceptable to do things that you know are wrong in this society who a certain degree you can for instance suggest some non sustainable business models and implement them right but you're always pushing the boundaries I mean yeah you're yes you're and yes this is seen as a good thing largely yes and this is different from other societies so for instance social mobility is an aspect of this social mobility is the result of individual innovation that would not be sustainable at scale for everybody else right normally you should not go up you should go deep right we need Baker's and if you are very good bakers but in a society that innovates maybe you can replace all the Baker's with a really good machine right and that's not a bad thing and it's a thing that made us so successful right but it also means that the u.s. is not optimizing for sustainability but for innovation and so it's not obvious as the evolutionary processes unrolling is not obvious that that long term would be better it's it has side effect so you basically if you cheat you will have a certain layer of toxic sludge that covers everything there is a result of cheating and we have to unroll this evolutionary process to figure out if these side effects are so damaging that the system is horrible or if the benefits actually outweigh the the the negative effects how do we get to the which system of government is best that was from I'm trying to trace back like five minutes I suspect that we can find a way back to AI by thinking about the way in which our brain has to organize it right in some sense our brain is a society of neurons and our mind is a society of behaviors and they need to be organizing themselves into a structure that implements regulation and government is social regulation we often see government is the manifestation of power or local interests but it's actually a platform for negotiating the conditions of human survival and this platform emerges over the current needs and possibilities in the trajectory that we have so given the present state there are only so many options on how we can move into the next stage without completely disrupting everything and we mostly agree that it's a bad idea to disrupt everything because it will endanger our food supply for a while and the entire infrastructure and fabric of society so we do try to find natural transitions and they're not that many natural transitions available at any given point Murray you're a natural transition so we try to not to have revolutions if he can have it right so speaking of revolutions and the connection between in government systems in the mind you've also said that you said that in some sense becoming an adult means you take charge of your emotions maybe never said that maybe I just made that up but in context of the mind what's the role of emotion and what is it first of all what is emotion was its role it's several things so psychologists often distinguish between emotion and feeling and in common day parlance we don't don't I think that in motion is a configuration of the cognitive system and that's especially true for the lowest level for the affective state so when you have an effect it's the configuration of certain modulation parameters like arousal valence your your attentional focus whether it's right or narrow interception or extra reception and so on and all these parameters together put you in a certain way to you relate to the environment and to yourself and this is in some sense an emotional configuration and the more narrow sense an emotion is an affective state it has an object and the relevance of that object is given by motivation and motivation is a bunch of needs that are associated with rewards things that give you pleasure and pain and you don't actually act on your needs you act on models of your needs because when the pleasure and pain manifest it's too late you've done everything but so you act on expectations what will give you a pleasure and pain and these are your purposes the needs don't form a hierarchy they just coexist and compete and your organism is why brain has to find it on dynamic homeostasis between them but the purposes need to be consistent so you basically can create a story for your life and make plans and so we organize them all into hierarchies and there is not a unique solution for this and people eat to make art and other people regard to eat and they might up be end up doing the same things but they cooperate in very different ways because they automate codes are different and vie cooperate based on shared purpose everything else it is not cooperation on shared purpose is transactional I don't think I understood the last piece of the achieving the homeostasis are you distinguishing between the experience of emotion and the expression of emotion of course so the experience of emotion is a feeling and in the sense what you feel is an appraisal that your perceptual system is made of the situation at hand and it makes this based on your motivation yes and on the you are estimates not your but of the subconscious geometric parts of your mind that assess the situation in the world with something like a neural network and this neural network is making itself known to the symbolic parts of your mind to your conscious attention by our mapping the them as features into a space so what you will feel about your emotion is a projection usually enjoy your body map you might feel anxiety in your solar plexus and you might feel it as a contraction which is all geometry right your body map is the space that is always instantiate and always available so it's a very obvious cheat if your non-symbolic parts of your brain try to talk to your symbolic parts of your brain to map the feelings into the body map and then you perceive them as pleasant and unpleasant depending on whether the appraisal has a negative or positive valence and then you have different features of them that give you more knowledge about the nature of what you're feeling so for instance when you feel connected to other people you typically feel this new chest region around your heart and you feel this is an expansive feeling in which you're reaching out right and it's very intuitive to encode it like this that's why it's encoded like it's incredible it's in code it's a code in which the non symbolic parts of your mind talk to the symbolic ones and then the expression of emotion is then the final step that could be sort of gestural or visual yeah so on that's part of this MOOC is probably evolved as part of an adversarial communication so as soon as you started to observe the facial expression and poster of others to understand what emotional state they're in others started to use this as signalling and also to subvert your model of the emotional state so we now look at the inflections at the difference between the standard face that they're going to make in this situation right when you were at the funeral everybody expects you to make a solemn face but the solemn face doesn't express whether you're said or not it just expresses that you understand what face you have to make it a funeral nobody should know that you are Trump triumphant so when you try to read the emotion of another person you try to look at the Delta between said truly said expression and the things that are animated mating this face behind the curtain so the interesting thing is so having done these having them as podcast and the video component one of the things I've learned is that now I'm Russian and I don't know how to express emotion on my face when I see that as weakness but whatever the people look to me after you say something they look to my face - just to help them see how they should feel about we said which is fascinating because then they'll often comment on why did you look bored or why did you particularly enjoy that part or why did you whatever it's a kind of interesting it makes me cognizant of on part like you're basically saying a bunch of brilliant things but I am part of the play that you're the key actor and by making my facial expressions and then do and therefore telling the narrative of what the big like the big point is which is fascinating makes me makes me cognizant I'm supposed to be making facial expressions even this conversation is hard because my preference will be to wear a mask with sunglasses to wear I could just listen yes which is understand this because it's intrusive to interact with others this way and basically Eastern European society have a taboo against that and especially Russia the further you go to the east and in the u.s. it's the opposite you are expected to be hyper animated in your face and you're also expected to show positive affect yes and if you show positive effect without a good reason in Russia they people will think you are a stupid and sophisticated person exactly and here positive effect without reason goes either appreciate or goes unnoticed no it's the default it's being expected everything is amazing have you seen these lego movie no there was a diagram where somebody gave the appraisals that exist in the US and Russia so you have your black curve and the lower 10% in u.s. yeah are it's a good start everything about the lowest 10% is it's amazing it's amazing and for Russians yeah everything below the top 10% is it's terrible and then everything except the top percent is I don't like it and the 10% is even so yeah it's funny but it's kind of true no yeah there's a deeper aspect to this it's also how we construct meaning in the u.s. usually you focus on the positive aspects and you just suppress the negative aspects and and our Eastern European traditions we emphasize the fact that if you hold something above the waterline you also need to put something below the waterline because existence by itself is as best neutral right that's the basic intuition if at best neutral yes or can is just suffering the default there are moments of beauty but these moments of beauty are in is inextricably linked to the reality of suffering and to not acknowledge the reality of suffering means that you are really stupid unaware of the fact that basically every conscious being spends most of the times of yeah you just summarized the ethos of the Eastern Europe yeah most of life is suffering with an occasional mobile to beauty and if your facial expressions don't acknowledge the abundance of suffering in the world and in existence itself then you must be an idiot it's an interesting thing when you raise children in the yes and you in some sense preserve the identity of the intellectual and cultural traditions that are embedded in your own families and your daughter asks you about Arielle the mermaid yeah and ask you why is Aria not allowed to play with the humans and you tell her the truth she was a siren siren see people you don't play with your does not end well and then you tell her the original story which is not the one by Anderson which is the romantic one and there's a much darker one Eugene a story what happened so when Dean is a mermaid or a water woman she lives on ground off a river and she meets this prince and they fall enough and the prince really really wants to be with her and she says okay but the deal is you cannot have any other woman if you marry somebody else even though you cannot be with me because obviously you cannot breathe and the water and I have other things to do then managing your Kingdom busy up here you will die and eventually after a few years he falls in love with some princess and marries her and she shows up and quietly goes into his chamber and nobody is able to stop her or willing to do so because she is fierce and she comes quietly and said out of his chamber and they asked her what has happened what did you do when she said I kissed him to death all done and do you know the end is in story right in the Anderson story the mermaid is playing with this prince that she saves and she falls in love with him and she cannot live out there so she is giving up her voice and her tail for a human-like appearance so she can walk among the humans but this guy does not recognize that she is the one that you would marry instead he marries somebody who has a kingdom and economical and political relationships to his own kingdom and so on as he shoots quite tragic she dies so yeah instead Disney The Little Mermaid story has a little bit of a happy ending that's the Western that's the American Way my own problem is business of course that I read Oscar Wilde before I read the other things so I mean doctor II needed inoculated with this romanticism and I think that the mermaid is right you sacrifice your life for romantic love that's what you do because if you are confronted with either serving the Machine and doing the the obviously right thing under the economic and social and all other human incentives that's wrong you should follow your heart so do you think suffering is fundamental to happiness along these lines suffering is the result of caring about things that you cannot change and if you are able to change what you care about to those things that you can't change you will not suffer well then would you then be able to experience happiness yes but happiness itself is not important happiness is like a cookie when you are a child you think cookies are very important and you want to have all the cookies in the world you look forward to being an adult because then you have as many cookies as you want right yes but as an adult you realize the cookie is a tool it's a tool to make you eat vegetables and once you eat your vegetables any way you stop eating cookies for the most part because otherwise you will get diabetes and will not be around for your kids yes but then the cookie the scarcity of a cookie if scarcity is enforced nevertheless so like the pleasure comes from the scarcity yes but the happiness is a cookie that your brain bakes for itself it's not made by the environment the moment cannot make you happy it's your appraisal of the environment that makes you happy and if you can change the appraisal of the environment which you can learn to then you can create arbitrary states of happiness and some meditators fall into this trap so they discover the room this basement room in their brain where the cookies are made and they indulge in stuff themselves and after a few months it gets really old and the big crisis of meaning comes because they saw before that their unhappiness was the result of not being happy enough so they fixed this right they can release the neurotransmitters at will if they train and then the crisis of meaning pops up at a deeper layer and the question is why do I live how can I make a sustainable that is meaningful to me how kinda insert myself would do this and this was the problem that you couldn't solve in the first place well at the end of all this let me then ask that same question what is that the answer to that what could but possibly answer be of the meaning of life what what could an answer be what is it to you I think that if you look at the limiting of life you look at what the cell is the life is the cell is cell yes or this principle the cell it's this self-organizing thing that can participate in evolution in order to make it work it's a molecular machine it needs a self replicator and entropy extractor and the Turing machine if any of these parts is missing you don't have a cell and it is not living right and life is facing the emergent complexity over that principle once you have this intelligent super molecule the cell there is very little as you cannot make it to it's probably the optimal compute for human especially in terms of resilience it's very hard to sterilize the plant at once it's infected with life so it's active function of these three components or the super cell of cell is as present in the cell is present in us and it's just the are just an expression of the cells a certain layer of complexity and the organization of cells that so in a way it's tempting to think of the cell as a von neumann probe if you want to build intelligence on other planets the best way to do this is to infect them b-cells and wait for long enough and visit reasonable chance the stuff is going to evolve into an information processing principle that is general enough to become sentient whether that idea is very akin to sort of the the same dream and beautiful ideas that are expressed the cellular automata in their most simple mathematical form you just inject the system with some basic mechanisms of replication so our basic rules amazing things would emerge that the cell is able to do something that James Hardy calls existential design he points out that in technical design we go from the outside in we work in a highly controlled environment in which everything is deterministic like about computers of our labs or our engineering workshops and then we use this determinism to implement a particular kind of function that dream up and that seamlessly interfaces with all the other deterministic functions that we already have in our world so it's basically from the outside in and biological systems designed from the inside out as seed will become a seedling by taking some of the relatively unorganized matter around it and turn it into its own structure and thereby subdue the environment and cells can cooperate if they can rely on other cells having a similar organization that is already compatible but unless that's that's there the cell needs to divide to create that structure by itself right so it's a self organizing principle that works on a somewhat chaotic environment and the purpose of life in the sense is to produce complexity and the complexity allows you to harvest negentropy gradients that you couldn't harvest without the complexity and in the sense intelligence and life are very strongly connected because the purpose of intelligence is to allow control and the conditions of complexity so basic you shift the boundary between the ordered systems into the realm of the Kay of chaos you build bridgeheads into a chaos with complexity and this is what we are doing this is not necessarily a deeper meaning I think the meaning that we have priors for that we evolved for outside of the priors there is no meaning meaning only exists if a mind protects it right yeah there is only civilization I think that what feels most meaningful to me is to try to build and maintain the sustainable civilization and taking a sliced Abad outside of that we talked about a man with a beard and God but something some mechanism perhaps must have planted the seed the initial seed of the cell do you think there is a God what is a God and what would that look like so if there was no spontaneous abiogenesis in the sense that the first cell formed by some happy random accidents where the molecules just happened to be in the right consolation to each other but there could also be the mechanism of that allows for the random I mean there's like turtles all the way down there seems to be there has to be a head turtle at the bottom consider something really wild imagine is it possible that a gas giant could become intelligent but would that involve so imagine you jet you have vortices that spontaneously emerge on the gas giants like big storm systems that endure for thousands of years and some of these form systems produce electromagnetic fields because some of the clouds are ferromagnetic or something and as a result they can change how certain clouds react rather than other clouds and thereby produce some self-stabilizing patterns that eventually to regulation feedback loops nested feedback loops and control so imagine you have such this thing that basically has emergent self-sustaining self-organizing complexity and at some point this wakes up and realizes and basically LEM Solaris I am a thinking planet yes but I will not replicate because I cannot recreate the conditions of my own existence somewhere else I'm just basically an intelligence that has spontaneously formed because it could and now it was a von Lohmann probe and the best von Neumann purpose at resting might be the cell so maybe it will because it's very very clever and very enduring create cells and sends them out and one of them has infected our planet and I'm not suggesting that this is the case but it would be compatible with the prints Permian hypothesis and with my intuition that abiogenesis is very unlikely it's possible but it's you probably need to all the cosmic dice very often maybe more often than they are planetary surfaces I don't know so god is just a large enough a system that's large enough that allows randomness now I don't think that God has anything to do with creation I think it's a mistranslation of the time wood into the Catholic mythology I think that Genesis is actually the childhood memories of a God so the when sorry that he Anna says is the world the childhood memories of a God it's basically a mind that is memory remembering how it came into being and we typically interpret Genesis is the creation of a physical universe by a supernatural being yes and I think when you'll read it there's light and darkness that is being create it and then you discover sky and ground you create them you will construct the plants and the animals and you give everything their names and so on that's basically cognitive development it's a sequence of steps that every mind is to go through then it makes sense of the world and then you have children you can see how initially they distinguish light and darkness and then they make out directions in it and they discover sky and ground and they discover the plants and the animals and they give everything their name and it's an creative process that happens in every mind because it's not given right your mind has to invent these structures to make sense of the patterns on your retina also if there was some big nerd who set up a server and runs this world on it this would not create a special relationship between us and the nerd this nerd would not have the magical power to give meaning to our existence right so this equation of a Creator God is the God of meaning is a slate off hand you shouldn't do it the other one that is done in Catholicism is the equation of the first mover the prime mover of Aristotle which is basically automaton that runs the universe earth total says if things are moving and things seem to be moving here something must move them right if something moves them something must move the thing that is moving it so there must be a prime mover this idea to say that this prime mover is a supernatural being is complete nonsense right it's an automaton in the simplest case so we have to explain the enormity that this automaton exists at all but again we don't have any possibility to infer anything about its properties except that it's able to produce change in information right so there needs to be some kind of computational principle this is all there is but to say this automaton is identical again with the creator of first cause over the thing that gives meaning to our life is confusion now I think that what we perceive is the higher being that we are part of and the higher being that we are part of is the civilization it's the thing in which you have a similar relationship as the cell has 12 a body and we have this prior because we have evolved to organize in these structures so basically the Christian God in its natural form without the mythology if you to undress it it's basically the Platonic form of the civilization is the is the ideal it's this ideal that you try to approximate when you interact with others not based on your incentives but on what you think is right Wow we covered a lot of ground and we left with one of my favorite lines and there's many which is happiness is a cookie that the brain bakes itself it's been a huge honor and a pleasure to talk to you I'm sure our paths will cross many times again Joshua thank you so much for talking today or they protect your necks yeah it's so much fun I enjoyed it awesome thanks for listening to this conversation with Yoshi Bach and thank you to our sponsors expressvpn and cash app please consider supporting this podcast by getting expressvpn at expressvpn comm slash FlexPod and downloading cash app and using collects podcast if you enjoy this thing subscribe on youtube review it with five stars an apple podcast supported on patreon are simply connect with me on Twitter at lex friedman and yes try to figure out how to spell it without the e and now let me leave you with some words of wisdom from your Shabak if you take this as a computer game metaphor this is the best level for humanity to play and this best level happens to be the last level as it happens against the backdrop of a dying world but it's still the best level thank you for listening and hope to see you next time you

Channel: Lex Fridman

Views: 708,885

Rating: 4.8962059 out of 5

Keywords: joscha bach, agi, artificial intelligence, ai, ai podcast, artificial intelligence podcast, lex fridman, lex podcast, lex mit, lex ai, lex jre, mit ai

Id: P-2P3MSZrBM

Length: 180min 18sec (10818 seconds)

Published: Sat Jun 13 2020

Reddit Comments

[u/TheAceOfHearts]

This has been one of my all-time favorite Lex podcasts, and I really hope Joscha Bach gets invited back regularly! Normally I'll listen to podcasts at 2x speed, but this one required slowing down to 1x speed along with occasional pauses just so I could process everything. I'm definitely going to be re-listening to this episode again; a very rare occurrence.

For anyone looking for more content from Joscha Bach, he has a ton of posts which span a large number of topics on his website. The videos section also has a handful of interesting presentations that he's given over the years.

[u/zoro_kongari]

Can we find the transcript for this podcast somewhere? It's too much for my brain to handle while listening.

[u/dunnolol123]

Give that man a universe to toy around with. That would be a Bible worth reading.

[u/mojambowhatisthescen]

This was among the most intellectually stimulating 3 hours I’ve experienced since grad school.

Joscha has the kind of mind that makes you rethink all the other people you’ve ever considered smart.

[u/BlizUdarzG]

Love this episode! Joscha is one smart cookie 😆

[u/Slaatje_Bla]

I can't understand why such a brilliant guy works for a company that associates, strongly, with Deepak Chopra. Very disturbing.

[u/GreyLichen]

I have been listening to the podcast sporadically for a year or so, picking out the ones that interest me. This is the first one that I had to comment on. I deliberately searched for a discussion community in order to do so. I haven't finished listening, but I will undoubtedly have to re-listen, as others have mentioned.

It struck me that the topics were highly emotionally charged: death, the simulation and illusion of self, the complexity (or not) of everyday human living, the importance (or not) of common sense reasoning.

There are some things said that I felt an impulse to challenge. And other things I wished I could have interrupted and commented on in real time.

Especially the question of whether the human brain/mind is impressive or not: whether what we do to get through the day and our lives, growing from infants, is complex.

I wanted to ask Dr. Bach how his suggestion, that the brain's models are not that impressive ("a million concepts") jives with the difficulty we have had building them. I suspect he would point out that a brain like ours, evolved for certain tasks, is not that well-suited to comprehensive self-reflection, and cannot intuit its own functioning, or describe it in a complete and formal way.

It seemed like some of the conversation topics led our host to lose almost lose his cool a few times. Not a criticism! Lex seemed so engaged that his thinking exceeded his ability to translate it into linear representation in words. I know the feeling. Conversation is a terribly inefficient way to exchange ideas, with their many dimensions and perspectives and inter-connections.

Anyway, amazing podcast, will definitely read more from the guest, and feel inspired to learn more deeply about some of the subjects raised.

[u/WCBH86]

So here's a question that Bach raises and I struggle to come to a conclusion on. We know that our way of life is unsustainable. To what degree should we immediately be working to restrict our lifestyles in line with this? Should I feel okay about working to own my dream low-carbon (but high end) house? Or should I really be looking to live in a fully upcycled home with minimal electronics etc etc, truly minimising my individual impact?

[u/Daemmon09]

Can someone help me out and provide me the book / author that he references around the 8 minute mark? I tried to understand what he said / google / view the subtitle. Unfortunately I am failing miserably.