EMERGENCE.

in this episode of street talk unplugged and so one thing that i find fascinating is that there is absolutely no reason why this should work like at all there is nothing that we can observe that says that these kinds of rules should exist at all this this model in principle is it's like it's too simple for it to actually work welcome back to street talk this week we are coming live from lisbon in portugal i haven't had access to my studio or any of my uh you know normal recording equipment so it's going to be a bit of an interesting one but i've been working out of coffee shops building an introduction doing all my stuff so yeah it's going to be a show about strong and weak emergence about cellular automata and we're going to be interviewing dr danielle gratarola and speaking all about his work on graph cellular automata so i hope you enjoy it folks see you soon one thing that really fascinates me is um a lot of the interesting phenomena happens at a different level a different rung of the emergence ladder if that makes sense and i'm starting to see this everywhere like even at work i'm building a code review platform and at the low level the metrics are obvious i know ken talks about the uh the tyranny of metrics by the way but you know it's how many code reviews that has an engineer done how many customer engineers do i have it's easy and then i start going up the levels of abstraction i'm talking to the senior leaders and now i'm starting to use much more abstract language like vertical information flows and trust and engineering culture and all of a sudden it's impossible for me to quantify and if i do i'm making it up and it's the same thing you're talking about these population scale phenomena that happen now so i've got all of these intelligent agents they're doing things and and i can try and because now i've got a meta optimization problem right so i want to encourage interesting phenomena in the emergent scale so i might say well this type of thing is interesting i want more of that but i'm i'm kind of i'm reaching because i don't know how to describe it i mean i think one other one other characteristic you can point to that that kind of separates um like population from just individual is sort of specialization versus generalization like i think population driven algorithms sort of implicitly are more about specialization a lot of the time because like each member of the population you want to be doing some different thing so they're kind of becoming specialists but i think there's a huge amount of generalization snobbery kind of within machine learning like we're looking for the ultimate general list all the time it's like get it to do all the tasks you can possibly do and then throw more in and the data set just gets bigger and we're all very impressed with that the population implies i feel like something's in spirit different because it's more just like actually i want to see a lot of different things and like hyper specializations to all kinds of exotic things that like probably the generals won't do because it's basically all cares about is being general this comes down to like the focus on like the particular level of abstraction or the level of agency that we have professor melanie mitchell wrote a beautiful book on complexity about 10 years ago i hope one day we can get it back on the podcast and discuss it in detail in the book she led with a quote from john locke i call complex such as our beauty gratitude a man an army the universe the animal kingdom has several examples of what i would call externalized or collective intelligence melanie quote biologist nigel franks in her book the solitary army ant is behaviorally one of the least sophisticated animals imaginable if 100 army ants are placed on a flat surface they will walk around and around in never decreasing circles until they die of exhaustion yet if you put half a million of them together the group as a whole becomes what some have called a super organism with collective intelligence the whole is in some sense more than the sum of its parts although we need to be quite careful with the language that we use here an emergent behavior or emergent property can appear when a number of simple entities or agents operate in an environment forming more complex behaviors as a collective if emergence happens over disparate size scales then the reason is usually a causal relation between different scales western science has a strong tendency towards reductionism because it assumes that things have essences so science keeps chopping up things into smaller and smaller pieces to find their essence it's an intellectual and philosophical position which interprets a complex system as the sum of its parts this is in stark contrast with relationism which you could say is related to the philosophical ideas we were discussing with andrew lampanen from deepmind last week biologist peter corning asserted that this whole discussion rather misses the point he said that holes produce unique combined effects but many of these effects may be co-determined by the context and the interactions between the whole and its environments now weak emergence describes new properties arising in systems as a result of low-level interactions these might be interactions between components of the system or components and their environment emerging properties are scale dependent though and can only be observed at large enough system scale one reason emergent behavior is hard to predict is that the number of interactions between a system's components increases exponentially with the number of components thus allowing for many new and subtle types of behavior to emerge emergence is often a product of particular patterns of interaction negative feedback introduces constraints that serve to fix structures or behaviors in contrast positive feedback promotes change allowing local variations to grow into global patterns on the other hand merely having a large number of interactions is not enough by itself to guarantee emergent behavior many of the interactions may be negligible or irrelevant or may cancel each other out in some cases a large number of interactions can in fact hinder the emergence of interesting behavior by creating a lot of noise to drown out any emerging signal the system has to reach a combined threshold of diversity organization and connectivity before emergent behavior appears mark bedau said in his 1999 paper titled weak emergence that an innocent form of emergence what he called weak emergence is now commonplace in the thriving interdisciplinary nexus of scientific activity sometimes called the sciences of complexity interestingly which he elected to put in air quotes for some reason he said that this included connectionist modelling and non-linear dynamics which is now commonly known as chaos theory and indeed artificial life he gave two interesting hallmarks of emergent phenomena in his opinion one emergent phenomena are somehow constituted by and generated from an underlying process and two emergent phenomena are somehow autonomous from the underlying process so he said that emergence is a perennial philosophical puzzle and at best the idea raises the spectre of illegitimately getting something from nothing he said that any defense of emergence should aim to explain that is to say explain away the apparent illegitimate metaphysics and indeed demonstrate emergence to be entirely compatible with materialism he argued that emergence must be more than intellectual masturbation putting words in his mouth here and actually demonstrate tangible value to the empirical sciences and be a constructive player in our understanding of the natural world he argued that weak emergence meets these goals but argued that stronger forms of emergence are entirely irrelevant he said that the failings of strong emergence can be traced back to this idea of strong downward causation which is this notion that things in the lower resolution emergent domain can cause things in the high resolution domain mark said that strong emergence is uncomfortably like magic how does a super veniant but irreducibly downward causal power arise since by definition it cannot be the result of the high resolution domain he said this would discomfort reasonable forms of materialism and pay homage to the idea that it's possible to get something from nothing mark concluded by saying that strong emergence is just a mystery which we don't need it's interesting to note that his definition of weak emergence is as follows macro state p of s with micro dynamic d is weakly emergent if and only if p can be derived from d and s is external conditions but only by simulation so interestingly his definition incorporates the necessity for computational irreducibility but not the notion of whether it is effectively computable one of the main hallmarks of weak emergence is the underrivability except for finite simulation the exponential divergence of trajectories or indeed the so-called butterfly effect describing the sensitivity of a physical simulation on its starting parameters is a well-known feature of chaotic systems but mark says that weak emergence is present in almost all complex systems regardless of whether they produce chaotic dynamics which lead to weak emergence being part of the definition of what it means to be a complex system the popular physics youtuber dr sabine hossenfelder wrote a paper called the case for strong emergence she felt that weak emergence was too deterministic and a front on free will if you like she used to think that we're all made of tiny particles which follow strict laws and human behavior is really just a consequence of these particles laws needless to say she's since changed her mind and she thinks that you should as well she led by saying reductionism works large things are made of smaller things and if you know what the smaller things do you know what the larger things do physicists call this idea reductionism now you might not like it but it works pretty well arguably reductionism allowed us to understand molecular bonds and chemical elements atomic fission and fusion the behavior of an atom's constituents and the constituents of those constituents and whoever knows what the physicist will come up with next she said she admits that the best explanation for the world around us right now is almost certainly incomplete sabine decided to discuss the concept of emergence in respect to physical theories and how fundamental they are she said that a physical theory is a set of mathematically consistent axioms combined with an identification of some of the theory's mathematical structures with observables if two physical theories give the same predictions for all possible observables then they are physically equivalent she displayed a figure depicting a directed graph of physical theories an edge between two theories meant that one was more fundamental than the other she said that a physical theory a is more fundamental than b if b can be derived from a but not the other way around in this case the theory b is weakly emergent from a a physical theory is fundamental if it is to the best current knowledge not emergent from any other theory so this is quite interesting weekly emergent is the opposite of more fundamental the idea that the theory at low resolution is always weakly emergent it can be derived at least in principle from the theory at high resolution sabine also discussed the causal exclusion argument which roughly speaking says that if a lower resolution effect can be derived from a theory at high resolution then the effect cannot have another cause the causal exclusion argument combined with effective field theory is the main reason why physicists believe that reductionism is correct and in a sense why strong emergence is not a thing she also spoke about top-down causation which is this idea that the laws of a system at low resolution can dictate the laws at high resolution a good example of this is the mental states in our brain causing our bodies to perform physical actions so it's important not to think of the emergent layers as being independent or assuming that they could or should be modeled in isolation interestingly though in sabine's article she denied that top-down causation even exists at all in her conclusion sabine did a 180 degrees and she decided that in fact there are many examples where there isn't a clear effective computational or functional path between physical theories she gave a hypothetical example of a function which cannot be computed for negative values of x or a taylor series expansion around zero and she said that if there are any points where the coupling can't be continued between resolutions you'll need new initial values which would need to be determined by measurement and therefore strong emergence is viable she said it's only fair on philosophers who believe that strong emergence exists that physicists first show the coupling constraints of a quantum field theory can always be continued to low energies for physically realistic systems so what is emergence emergence is just the interpretation of a phenomenon from the perspective of a different scale at least according to professor david chalmers he wrote a paper called strong and weak emergence where he lamented the abuse of the term strong emergence by complex systems scientists and cognitive scientists echoing mark badal before him chalmers says that it is strong emergence which is most common in the philosophical parlance of emergence and in particular used by the british emergentists of the 1920s he thought that we could say a high-level phenomenon is strongly emerging with respect to a low-level domain when the high-level domain phenomenon arises from the low level domain but truths concerning that phenomenon are not deducible even in principle from truths in the low level domain now i think deducible is a bit of a weasel word but we'll talk more about that in a minute um he says that weak emergence does not yield the same sort of radical metaphysical expansion in our conception of the world as strong emergence but it's no less interesting he says that you can think of weak emergence in terms of the ease of understanding of one level in terms of another level emergent properties are usually properties which are more easily understood in their own right than in terms of properties at a lower level indicating that weak emergence appears to be an observer relative property now how interesting is this high level phenomenon to an observer and how difficult is it to deduce this phenomenon from the lower level that is emergence so chalmers takes emergence in the general sense to mean surprising or interesting and indeed an unexpected phenomena and he uses the strong versus weak designation to delineate a radical paradigmatic surprise he says that the emergence of high-level patterns and cellular automata a paradigm of emergence in recent complex systems theory provides a clear example if one is given only basic rules governing a cellular automaton then the formation of complex high-level patterns such as gliders may well be unexpected therefore the patterns are weakly emerging but the formation of those patterns is straightforwardly deducible from the rules and the initial conditions he concedes that this might take a fair amount of computation which he indicates as a reason why the emergent behavior wasn't obvious to start with and i assume by the word obvious he's kind of means as an autonym to unexpected cellular automata are provably computationally irreducible this means that there are no analytical shortcuts to perform the effective calculation without resorting to running the sequential simulation in its entirety since the computational domain is exponentially large in the case of discrete cellular automata and infinitely large in the case of continuous cellular automata if you were trying to find the initial conditions and rules for a given behavior or even if you had to recompute the simulation we would argue that this constitutes at least a semi-strong designation of emergence because of the effective computability right the effective computability must come into it professor chalmers says that strong emergence has much more radical consequences than weak emergence if there are phenomena that are strongly emergent with respect to the domain of physics then our conception of the natural world would need to be revolutionized to accommodate them with new fundamental theories now i find this a little bit strange i mean given that a class iv cellular automata is touring incomplete which is to say that they can represent any computer program it seems like a contentious point that there's no possible output in a cellular automata which would be paradigmatically surprising maybe i'm wrong to be clear chalmers is a materialist right he's not subscribing to any kooky views by saying this he's a computationalist in the sense that he agrees that if he replicated him atom by atom in the natural world according to our universe then it would have a consciousness right but he argues that consciousness isn't a logical necessity he could imagine a universe which has all the same physical laws where he would be a philosophical zombie because it's not logically necessary yeah so this term emergence is you know it's so woolly and uh ambiguous i mean it gets used for so many different things uh in the sciences in philosophy any kind of phenomenon of a complex system that we don't fully understand we say oh yeah well it's emergent and then okay well great well what's the cash value of uh of that and i've found it useful to distinguish as you were weak and strong emergence where weak emergence is kind of a matter of mostly of complexity um where for example you've got some simple rules at the bottom level that gives rise to some high level macroscopic phenomenon which is complex and surprising hard to predict and derive as a practical matter but it's really more of a practical limitation you can still see in principle why those bottom level principles say laws of physics or rules in a cellular automaton would in principle give rise to these high-level phenomena derivable and principle if not in practice whereas strong emergence would require something that's not even derivable in principle and i guess i think that most of the things you get in ai or complex systems theory and so on involve weak emergence certainly i was very influenced by um doug hofstadter here who's who's you know good alicia bark is in some ways all about the powers of weak emergence how really simple processes at one level could give you complex processes at a higher level and actually get these tangled hierarchies he'd talk or strange loops you'd go up a few levels and then you'd you'd come down so i guess i'm probably more sympathetic with hofstadter's picture of weak emergence than say george ellis's where causation is always within a level i think there are very complex relations between the levels and some of them may be best understood as causal you could think of it you know the butterfly um snapping its wings having some causal relation to some sociological event days later so you do get these tangled hierarchies but all that is still weak emergence so he's agreeing with mark badal by saying that any endorsement of strong emergence is a rejection of physicalism and reductionism which is to say an appeal to magic and esoterica whereas weak emergence can be used to support the physicalist picture of the world by showing how all sorts of phenomena which might seem novel and irreducible at first sight can nevertheless be grounded in underlying simple physical laws chalmers thinks that there is exactly one clear example of strong emergence in our universe which is guess what our consciousness [Music] we can say that a system is conscious when there is something it is like to be that system which is to say it has a phenomenological experience chalmers argues that it is a fact of nature that the universe contains conscious systems we are existence proofs of that and there's reason to believe that the facts about consciousness are not deducible from any number of physical facts it makes the argument that there could be a world physically identical to this one but lacking consciousness entirely which is very similar to that philosophical zombies argument that i just spoke about or even containing conscious experiences which are potentially different to our own roger penrose said that the human ability to understand is undecidable and requires consciousness if this is true it might be a mathematical proof that consciousness is strongly emergent exactly as charm as claims and so the way i the way i view strong emergence at least for for right now is that if i have these different formalizations at different levels and it's just not possible in any practical scheme whatsoever for me to directly go from a lower level to a higher level like for example i just can't computationally do it or there's no mathematics that can ever hope to symbolically you know prove that that uh the properties i observe at a higher level derive from a lower level maybe i you know people say well in principle you could but in reality you just may never be able to do that you know does that qualify as strong emergence or is that a bad definition of it and do you think there is such a thing as strongly emergent behavior or can we ultimately just reduce everything down to uh hypergraph or or loop chrom quantum gravity or whatever every level a sense is independent you cannot expect it to be fully reduced to a lower one or okay a higher one or each level has its own value it has its concepts it has these conclusions it has problems that's suitable to be to be solved at the level not higher or lower that is to me that's the first principle but the second one is you don't want to push it too far you don't want to say all the layers have nothing to do with each other so as after all we're talking about the same object okay we're talking about the same that the google map of the same eye area even though you zoom in block out a different level if he is a two map a different array that's a different story okay so as far as all those theories is in a sense about the same object but there is the different levels of description and they are correlated but they are kind of like very overall high level uh not high level is the wrong way to say it is confusing the relation it's kind of like you have a overall large scale correlation but you don't have one-to-one mapping among the concepts that's also my opinion about the relation for example between neurons and concepts uh of course they're related but there is no one to unmapping or if not even money problem might be it's more like a money to mine in my opinion and also it's very messy in my opinion uh except if you want to limit your discussion to a very special phenomena at a certain level for example we know that some basic concepts in chemistry can be explained very well in physics right because villains talk about the details uh the same story of something in biology can be explained very well with physics and chemistry we should talk about the details so that's true but on the other hand if you say that biology overall can be even truly reduced it to chemistry and uh feelings i say not only that's practically wrong is even theoretically wrong because when you're seeing that you're ignoring the cognitive capability of the researcher and the user of your theory you cannot really reduce everything to uh to the to the lower level without greatly increase the number of concepts right computational cost melanie mitchell pointed out that it's incredibly mysterious how the intricate machinery of the immune system fights disease or how a group of cells organizes itself to be an eye or a brain or how independent members of an economy each working chiefly for their own gain produce complex but structured global markets or most mysteriously how the phenomena we call intelligence or consciousness emerge from non-intelligent non-conscious material substrates the cognitive scientist douglas hofstadter in his book go to lescherbach made an extended analogy between ant colonies and brains both being complex systems in which relatively simple components with only limited communication among themselves collectively give rise to complicated and sophisticated system-wide global behavior the ants in our human brain are of course our neurons they communicate with each other in a similarly simplistic manner yet our intelligence and arguably our consciousness emerge from this low-level primitive communication markets are also complex emergent and self-organizing entities if you like melanie said in her book that they are self-organized on the microscopic and the macroscopic level she said that on the microscopic level individuals and companies and markets try to increase their profitability by learning about the behavior of other individuals and companies the microscopic self-interest has historically thought to push markets as a whole on the macroscopic level towards a so-called nash equilibrium now the process by which markets obtain this equilibrium is called the market efficiency the 18th century economist adam smith called this self-organizing behavior of markets the invisible hand it arises from the myriad microscopic actions of individual buyers and sellers the individual actions on a trading floor give rise to the hard to predict large-scale behavior of financial markets now melanie gives three core properties of complex systems in her book one complex collective behavior large networks of individual components which each one following relatively simple rules with no central control or leader it's the collective action of vast numbers of components that give rise to the complex hard to predict and changing patterns of behavior which fascinate us so much [Music] two signaling and information processing complex systems use information and signals from both their internal and external environments and three adaptation all of these systems adapt that is they change their behavior to improve their chances of survival or success through learning or some evolutionary process so melanie then goes on to give her definition of a complex system as follows a system in which a large network of components with no central control and simple rules of operation give rise to complex collective behavior sophisticated information processing and adaptation by learning or evolution now i spoke with our friend dr duggar on strong emergence and he said that in his opinion it describes behaviors which cannot be analytically derived nor effectively computed from a lower level or higher resolution theory this would place glider wars in a cellular automaton firmly in the domain of strong emergence a cellular automaton is computationally irreducible there's no effective computational path from the lower level rules to the higher level behavior the only thing you can do is run the simulation again from scratch he thought that chalmers and hossenfelder evade the issue and or beg the question by phrases like deducible in principle or a fair amount of computation or follows from that at least in principle etcetera etcetera so you know what he's saying is that they make claims which we can in principle do something but they can't actually demonstrate or perform with a reasonable amount of computation in physical strong emergence you can't even run the computation at least in a continuous cellular automaton you're pretty much in the same boat as the end body problem now a lot of this discussion comes down to whether you believe infinity exists or not actual infinity this is a teaser clip from our conversation with dr joshua bach can the universe can our actual universe that we're in right now be actually infinite in spatial extent the problem is that you can have unboundedness in the sense that you have a computation that doesn't stop giving you results but you cannot take the last result of such a computation and go to the next step you cannot have a computation that relies on knowing the last digit of pi before it goes to the next step in this sense you don't have an infinity but the infinities are about the conclusion of such a function it means that you actually run this function to the end and then do something with the result unboundedness is different in the sense that you will always get something new that you didn't expect that you cannot predict but it's it's just going on and on without this end and it i think it's completely conceivable that our universe is in this class of systems in the sense that it doesn't end but it doesn't mean that there is anything that gives you the result of an infinite computation because if that was the case then it could not be expressed in any language it also means if something cannot be expressed in any language that you cannot actually properly think about it because when you think you need to think in some kind of language not in english but in some kind of language of sort or in a mathematical language that doesn't have contradictions and what girdle has shown is that the language that he hoped to reason in about infinities breaks that it has contradictions in it that at some point it blows blows itself apart so the languages that we can build are only those in which we have to assume that infinities cannot be built so infinity in this sense is meaningless because we cannot make it in any kind of language so the thing is though i'm not limiting what the universe is capable of based on human you know mental and linguistic limitations or even mathematical limitations like i'm i'm asking you if it's possible for this universe that we're in to anticly be right now actually infinite in spatial extent the thing is that you try to make a reference to something that you cannot observe and cannot conceive of other than making a model in some kind of language and to have that model make sense the language needs to work right otherwise you are just maybe in some kind of delusional thing you can't get to infinity from non-infinity and you can't get to discrete from analog so keith believes that there are actual infinities in stark contrast to people like stephen wolfram but our brains are computationally bound we are conducting what is a discrete computation in our mind but we might have access to oracles which is to say we're connected to a turing machine but we can only sample at a certain rate keith believes in infinity therefore there may be many strongly emergent phenomena because they're not computable he therefore doesn't think the universe can even run on a computer or indeed that we exist inside a simulation the simplest way to prove the constructivist hypothesis that natural systems need to perform computation in order to succeed and adapt in respect of its environment is to create an idealized version of the problem that is to say let's simplify it as much as possible while still retaining the features that make the problem interesting and that is exactly what a cellular automaton does cellular automata are a class of computational models that exhibit rich dynamics weakly emerging from the local interactions of cells arranged on a regular lattice for example two-dimensional grid cellular automata were invented by john von neumann back in the 1940s they exhibit extremely complex behavior that's difficult or impossible to predict from the cell update rule now melanie mitchell commented in her book that this is one of the great ironies of computer science since cellular automata often referred to as non von neumann style architectures in contrast with the von neumann style architectures that he also invented von neumann was also able to show that his cellular automaton was equivalent to a universal turing machine and therefore capable of universal computation which is to say computing anything which a turing machine can in 1970 john conway invented his own cellular automata called the game of life and it had significantly simpler update rules than von neumann's version the most simple version is on a 2d grid with discrete binary values where the alive or dead state of every single cell depends on its eight neighboring cells the rules are as follows one any live cell with two or three live neighbors survives two any dead cell with three live neighbors becomes a live cell three all other live cells die in the next generation similarly all of the dead cells stay dead even though the game of life doesn't pretend to be the most sophisticated way to understand complex systems they are a wonderfully simple way to get acquainted in the ideas of complexity science and in particular weak emergence now many of the patterns are incredibly lifelike and that's because these are class four automata they are touring complete which is to say they're capable of representing any computation now being weakly emergent doesn't preclude useful analysis i mean for example it's still possible to model how frequently phenomena like gliders appear in the emergent domain given many random initializations laws governing the weakly emergent states almost certainly exist but can only be discovered through empirical analysis and observation and simulation we can identify motifs systems behaviors mechanisms high level abstractions even in the emergent layer but nothing from first principles in what sense do natural systems compute at a very general level one might say that computation is what a complex system does with information in order to succeed or adapt in its environment morphogenesis means the generation of form it's colloquially described in a biological process that causes a cell or a tissue or an organism to develop its shape but in an artificial intelligence context we can think of it as meaning the blueprint of emergence of any physical form professor sebastian riese recently wrote an article called the future of artificial intelligence is self-organizing and self-assembling and before you ask yes we'll be inviting him to mlst he spoke of a current movement which combines ideas from deep learning with ideas from self-organization and collective systems it's a wonderful tree ties for emergentist open-ended and biologically inspired ai enthusiasts searching for parameters of self-organizing systems which produce particular patterns is a difficult optimization problem trying to make self-organization programmable is a research field of its own called morphogenetic engineering he said that the merger of these ideas could ultimately allow our ai systems to escape their current limitations such as being brittle and rigid and not being able to deal with novel situations however the combination of these methods also poses new challenges and requires novel ways of training to work as efficiently as possible risi said that one of the most fascinating aspects of nature is that groups with millions or even trillions of elements can self-assemble into complex forms based only on local interactions and display what is called a collective type of intelligence sebastian gave the example of ants which can join forces to create bridges and rafts or navigate difficult terrain termites can build nests several meters high without an externally imposed plan and thousands of bees work together as an integrated whole to make accurate decisions on when to search for food or a new nest he said that achieving these incredible abilities is a result of following relatively simple behavioral rules through a process of self-organization kamazin defined self-organization in 2001 as the following as a process in which a pattern at the global level of a system emerges solely from the numerous interactions among lower level components of the system moreover the rules specifying interactions among the systems components are executed using only local information without reference to the global pattern in short the pattern is an emergent property of the system rather than being imposed on the system by an external ordering influence with the emergence of powerful machine learning algorithms sebastian said that the key question is instead of hand designing the algorithms for self-assembly can we learn these algorithms instead allowing more complex forms to be created sebastian said that self-organizing systems are made out of many components which are highly interconnected the absence of any centralized control allows them to quickly adjust to new stimuli and changing environmental conditions additionally because these collective intelligence systems are made of many simpler individuals they have built-in redundancy with a high degree of resilience and robustness individuals in this collective system can fail without the overall system breaking down sebastian points out that evolution was able to exploit self-organizational processes to create artifacts of remarkable complexity however human-made designs are normally put together piece by piece this is similar to the idea of whether an ai architecture and knowledge should be human engineered or revolved blank slate style such as professor rich sutton pointed out in his bitter lesson essay the amount of information it takes to specify the wiring of a sophisticated brain directly is far greater than the information stored in the genome instead of storing a specific configuration of synapses the genome encodes a much smaller number of rules that govern how to wire up a brain through self-organizing processes and how synapses should change based on the activation of neurons this amazing compression has also been called the genomic bottleneck now when humans engineer bridges or teach curricula there's always a plan a pedagogy a curriculum in biological construction there's no blueprint well not one which defines the outcome evolution is a kind of meta optimizer and our dna is incredibly compressed it can't possibly describe the complex configuration of our brains it's a form of optimization which transgresses rungs of the ladder of emergence sebastian says that our genes contain the information to make the structure by controlling a sequence of events during morphogenesis our final physical form is merely a kind of sampled materialization of this lower level process this is very similar to this concept of inverse diffusion which happens in the open ai daily 2 model by the way now as sebastian says in his article deeper neural networks are totally human engineered whether it's the architecture itself or indeed the optimization algorithm which is stochastic gradient descent given enough data deep learning algorithms can learn to decompose any space into a highly sophisticated geometrically tessellated nested compressed representation the problem is that this representation is extremely brittle and breaks with even minor changes in the environment deep learning models efficiently compress what they have seen before with laser-like effectiveness but the problem is that many domains are open-ended and combinatorially large and are not amenable to memorization in this way sebastian argues that using emergence and self-organization might help robustify neural networks in a similar way to how biological systems are robust although he conceded that self-organization is not the only principle that allows biological organisms to display high-level robustness anyway i highly recommend you check out sebastian's article it's brilliant to follow up a bit on this idea of centralized versus decentralized if we look at decentralized systems not always but very often they sort of self-organize into a centralized system for example the brain has the sort of the prefrontal cortex directing everything if we look at humans the first thing humans do is they band together and they elect a leader right um if if we build decentralized computing systems there's always like one leader and so how how much how much do you think this the emergence of properties such as intelligence or whatnot is a property of really decentralized computing or how how important is this sort of leader election among decentralized systems and can we do without it oh uh yeah i think that's a great question again you always ask a great question so you're right like in many decentralized systems like our brain or in civilization eventually something like a centralized system is formed and and usually maybe via our genotype our genes the same centralized system is usually formed across all humans and in even societal structures like this typically you have a leader or a few leaders and they govern the society in a few types of ways right but but i i think the emergence of that structure is very important compared to uh designing it uh top down at the beginning uh let me tell you why uh because like take the example of of say our bodies or the brain right there there are cases like the the the way the uh the structure is is emerged may be the same for most people but for people with certain disabilities of unfortunate disabilities from from birth or accidents uh we're able to see brains or structures evolve differently but they they still function as a whole like like certain certain infants are known to to have half their brain and like uh not functioning at birth and you know like the the it has to be removed like uh from birth but they still grow into a functional brain that has a different structure than what what we traditionally know for most humans and and even for for people with disabilities like like blindness or death they eventually their brain structures would would like change functionalities a blind person would use their visual cortex to process audio for instance so so where the emergence property is very useful for for tackling like changes in the environments as i mentioned in the beginning so so ultimately the goal is to have something that will work even when the environment changes but it'll work maybe optimally when the environment is expected but it's not going to completely not work when the environment changes alexander maude vinsef another guy that we definitely need to get on the show wrote a fascinating article called growing neural cellular automata i've been looking into this and essentially it's a convolutional or neural network type architecture which produces what appears to be an rgb value for every single pixel but actually the output is a 16 channel space including a bunch of other information they've turned a self-healing image generation process into an emergent phenomena and so they're kind of continuously applying this convolutional neural network over the um input space much like you would do with the traditional cellular automata except this one of course is a continuous cellular automata which is learned with a neural network but then it has this incredible thing where if you interactively delete components of the image or perturb components of the image it'll dynamically repair itself which is fascinating just imagine some of the applications for this where you could have self-healing systems and you could have these agents that learn to heal a system as an emergent behavior so there's this really interesting intellectual journey here which starts with discrete cellular automata which are binary and run on a regular lattice which is say a two-dimensional grid and then all sorts of interesting things happen when we increase the resolution or run on different manifolds or have continuous values and then even use something like a learnable neural network for performing the update rules dr danielle gratarola is a scientist and postdoctoral researcher at epfl he recently published a fascinating paper called learning graph cellular automata which was published in in eurips and in that work he focused on a generalized version of a typical cellular automata called a graph cellular automata in which the lattice structure is replaced by an arbitrary graph uh in particular they extended the previous work which i just showed you from alex malvinsef you know which was when they they learned a 2d convolutional neural network for applying um the cellular automaton update rule to now using a graph neural network and learning the update rule on that with message passing it's absolutely fascinating so now i give you danielle gratarola cool right let me get my notes out by the way i've i've just been on a crash course in cellular automata oh nice uh it it is absolutely fascinating yeah um i'm completely hooked on it actually yeah it is it is fascinating tim i'm gonna i i have to share with you i have to share with you my little uh cellular tama that that tries to do um you know uh light casting or shadow casting roguelike roguelike games oh nice yeah many of those like tiny games like even uh i think it's called gnome fortress something like that yeah it was an old-school linux game uh it's used like celero tamato to generate the terrain and stuff like that it's super fast fascinating yeah i played around with with simple simple cellular automata quite a bit and little hobby yeah you know hobby games or simulations i mean even the um you know tim that that uh galton board simulation in our our video that was a solar automata yeah i hadn't thought about that of course yeah it's fascinating i mean and by the way i mean because you linked alexander's article um you know he did the kind of the the 2d gridded cnn version of uh morphogenesis and i mean maybe you should just introduce i'll tell you what we're we're doing this all wrong uh daniel why don't you introduce all right right so uh yeah so my name is daniel i am currently i just graduated actually from idcia in lugano so i'm currently working at epfl in lausanne so i moved to the french-speaking part of switzerland and so right now i'm working in the domain of proteins and my formal training during my phd was in graph neural networks and at some point i reached out to my current let's say supervisors or pi that were starting out this project on applying graph neural networks to the protein domain in particular protein design which is like essentially the inverse problem to alpha fold if you've heard about alpha fold recently so alpha false goes from the sequence of amino acids to the folded structure and one still open and very interesting problem is so how to do the opposite so if i want a particular structure what is the sequence that would fall into that structure right and you would think that having solved one direction would essentially means you've solved the other but it's still com computationally expensive to go over all possible sequence and try and see if they fall in the correct state and so like um there's still like this open question of uh whether the structure of a folded protein somehow informs uh the sequence and as you can if you can predict one from the other uh and so i'm working in that whole domain uh right now but as i said like my background is in graph neural networks uh during my phd i've worked on a thousand different things related to graph neural networks and by the way i started like 2017 i started my bhd so it was still at the time where graph neural networks were starting to emerge a little bit so there was like this feeding frenzy of you know finding applications and trying to see if stuff worked which was a really exciting time to be in graph neural networks i should say um and then like uh at some point during my phd towards the end i i decided to link back to one of my oldest passions which was this idea of the cellular automata and and trying to see if some of the tools that i have been working on had been working on would actually be useful to to say something about that whole world and it turned out it did so yeah yeah well i i mean i'm so inspired by geometric deep learning after i spoke with michael bronstein and and his friends um but yeah i mean so much of the work that we've been brought up on is is uh euclidean or gridded data and then when you start to think about some of the applications that you can do with graphs and and you know curved surfaces and so on it blows my mind but before we get there why don't we go on a kind of intellectual journey here and start talking about cellular automata now anyone who's had the misfortune of doing leeco challenges in the tech industry probably would have had to implement conway's game of life at some point and um usually the way you know these challenges are formulated is and they are binary which means the cells are one or zero and it's on a regular lattice usually a 2d grid and you have a whole bunch of update rules which are a function of the neighboring cells and then you just kind of execute all of these rules and you just get this emergent phenomena happen when you zoom out it's fascinating so can you just tell us a little bit about about cellular yeah sure so basically the the short story is what you just said so you have this uh essentially it's a computer program or a computational model that has a state and typically the status what you said it's just a bunch of cells arranged in these regular structures which can be you know 1d or 2d or even 3d or whatever and and then every cell has a particular state of its own and then you have this transition rule or update functions however you want to call it that is applied synchronously to every cell and essentially decide what the next state of the cell will be as a function of the cell itself and the neighbors um and really the cool thing that you find is that even though like the complexity at the level of the rule is fairly low so you have pretty simple rules that you can define um you know the behavior that emerges can be like very life like like the the tiny creatures that you see emerging on this on these grids uh really you know they kind of click with with our pattern matching system as humans because they they look like uh tiny creatures moving around the grid they're able to spawn new creatures they're they're they're able to you know move coherently and they add periodicity over time and and so it's either living things or or engineered things there they have the same that kind of regularity that we that we recognize as interesting um and that's like the just the basic version but then with time people have started to complicate the definition of cellular automata right so for example instead of you know binary you can ask the question of okay what happens if i allow the states to be n and possible states over the grid so and i can color the states differently or i can ask you i can ask what happens if the state is continuous right and as you start doing that what you see is that uh the the behavior becomes more and more complex in a sense even though uh let's say the common god of complexity at the level of the rule remains fairly low as you start introducing just that tiny bit more of complication you see this insanely complicated patterns that emerge as a result and so for example at some point people started uh let's say playing with the definition of the neighborhood right so you make it a bit a bit larger so which is equivalent to increasing the resolution right so your your grid approximation which if you think about it like a grid is just a discretization of 3d space or 2d space right but at the same time we are like as humans we are very far from that kind of level of discretization of space if it even exists right so we're we're used to thinking about high resolution in a sense and so what you what you observe if you start to increase the resolution of these models is that their behavior starts to become you know eeriely like living things and so you see like tiny cells forming and moving around and then they start to organize into membranes and stuff like that and this all happens by that same convolution like process that's happening on this grid and and that's why i think they're so fascinating you know leading back also to your comment before like they have this they let you observe behaviors that typically you only see in nature but at the same time you are aware of this inherent simplicity that the behavior stems from and uh and so i'm already like kind of diverging because this topic like automatically makes me go on rants of how these things are super simple and yet super complicated and super fascinating maybe maybe um just playing devil's advocate here because i i too love solar automatize we were talking about before we started the show but just to perhaps pull things back to some grounding here which is that uh in a way folks working on cellular automata have converged in some ways to a very old set of numerical techniques called finite difference you know modeling right so and the way what engineers do with finite difference is they say okay look i have this set of partial differential equations right and as we know tons of things in the world physical phenomena can be described by pdes partial differential equations and they say okay can't solve these symbolically so but i can do it numerically if i have a grid then i can start to write down how the pdes you know result in changing continuous values based on kind of neighboring grid cells and they do this exact thing they create a mesh they uh write down what uh what transition rules the pdes would would imply for each individual cell and then you run simulations and so you wind up with things like you know uh the equivalent of cellular automata with continuous values for the wave equation or for diffusion equations or for all this kind of thing and so in one sense it shouldn't surprise us that cellular automata can reproduce all the behaviors that we see in life because indeed the majority of that or all of it you know stems from partial differential equations at some level and we know that we can approximate those solutions with cellular automata via the you know finite difference route um but at the same time it feels it definitely feels mysterious and i think there is something something deep there and this even goes back to say mandelbrot right with his discovery of of fractals i mean what the heck i just had this simple little quadratic equation in complex space and if i just iterate that map over and over again i wind up with this absurdly complex you know in a sense like infinitely complex you know boundary and and interesting connections so there's these kind of two opposing viewpoints i wonder how you how you reconcile those like on the one hand it seems very interesting and mysterious and yet on the other hand it seems like of course if we go to a high enough resolution we can simulate arbitrary differential equations i think it's like the i don't know if it's an answer but one possible explanation of this apparent dichotomy is uh it's a matter of scale right so if you go low enough with the scale and by low i mean you know modeling uh the low level physics in a sense uh you probably get to the point where the skill is so low that everything makes sense to be modeled as a continuous differential equation right and so at some point you know cellular automata or a set of pdes what's the difference it's not even that well defined like it's like a sort of spectrum that you can decide where you want to place yourself in um what is interesting though is that i think that this idea of locality and and having you know update rules or transition rules or whatever you know execution kind of engine you want to have for your inherent program at that point it it becomes as you go up in the abstraction hierarchy it just makes sense to consider your elements yourselves as discrete objects and then this is maybe not true if you are if you're modeling for example the flow of some some water or whatever for example i had this professor uh when i was at the during the bachelor that he was using cellular automata to model how water moves in like inside coffee for example right and and then you can you can then you can ask the question of okay is it a cellular automata should i be using some something else that is actually continuous but what i think is really fascinating about the idea at least of cas is that as you go up in the hierarchy then it makes more and more sense to think about you know discrete agents interacting according to some rules and at that point the continuum like kind of gets lost anyway or at least it gets lost in the way you want to model it right so if you're modeling you know again low-level physics it makes sense to be working with a grid but if you're you know working out modeling human interaction for example which you can probably still model fairly well as a sort of local kind of dynamical system then it doesn't make sense to model humans as living on a continuous trade well no you kind of want to model them as individual objects in a sense right yeah what's what's interesting to me is as as you're bringing up so you were going in in this direction of um if we're at a very low level it might make sense to think of of a continuum but then at some level of abstraction uh things really behave as these discrete you know objects or say in the case of of carl fristen you know a markov boundary a thing that has this kind of stochastic dynamic you know markup boundary yet it behaves as a unified you know whole um and and what's odd is that neither at the highest scale nor the lower scale does either continuity or discrete nature ever disappear it just seems to be continuously i shouldn't have said continuously it just seems to be forever intertwined you know that okay if i keep going down in this direction i get to a continuum but maybe if i go even deeper i get back to some discrete you know hypergraph or if i'm going in the opposite direction i get up to say the discrete level of molecules but if i keep going it starts to behave as a continuous fluid if i go further it starts to behave as a discrete cell if i go further than that it's a you know you keep interleaving back and forth between discrete being the correct level of analysis versus continuous being the correct level analysis that's kind of the mystery is that they're they don't exist at either extreme they just are constantly interleaved that's the real cool thing that it's like it appears very arbitrary that we as humans at some point decide no this is like i i recognize this as a layer of of this abstraction it's like you know i i can pretty clearly distinguish between being at the layer of atoms and being at layers of cells although if you look close enough then cells are composed of proteins and proteins and composites of atoms so you can kind of always go back uh but at some point there's there's also this kind of uh limit uh that defines um or that decides how far can a layer kind of communicate with the other layers right so we as humans we have no agency to interact with atoms although we are made of atoms but we have kind of essentially zero ability to interact with that layer and so it's like there is a sort of intrinsic boundary that lets you operate on some levels of these obstructions and these levels appear to be fairly close to one another so i might be able to influence the level above me and below me which might be in i don't know if we want to look at some discretization maybe i can act on my organs and i can act on society right one level that one level up but already i cannot act on my proteins right or or my individual cells it's gonna be much much harder uh and as you go down in the hierarchy and of course up in the hierarchy the contribution of that particular layer becomes less and less relevant right so it's like it you you're definitely able uh to recognize those boundaries or like i don't know if you can give a precise definition but there definitely appears to be some level of discreteness uh or at least some ranges where it makes sense to talk about a level of abstraction that stands on its own i would say yeah i'm really interested actually that there's a kind of observer relative problem that you know depending on the ladder of the um let's call it an emergence ladder we're on a rung of the emergence ladder and maybe that determines how we can formalize and understand phenomena on different rungs of the emergent ladder but just to pull the discussion back a tiny bit so um this is all quite new to me and i'm fascinated by it i've actually just started reading a new kind of science by wolfram uh inspired by all the links that that you sent us and um you know one of the one of the first things to learn about with cellular automata is is the basic discrete one-dimensional version and um wolfram's actually given all of them names right because if you think about it in the one-dimensional discrete cellular automata you have a neighborhood of three and then if it's binary you've got two to the power of three which means you've got eight patterns and then for every single pattern it could be one or zero so you've got two to the power of eight which is 256 things and and all of those things you know so for example there's um rule 110 which he says exhibits class 4 behavior now this is interesting as well because i'm interested in you know did he just pluck this out of thin air that you can kind of classify the behavior subjectively of of phenomena in in the emerging uh domain and uh but anyway he said class iv behavior which is neither completely random nor completely repetitive localized structures appear and interact in various complicated looking ways and and then there's this guy called matthew cook who used to work for wolfram and and he said these structures are rich enough to support universality all right this result is interesting because rule 110 apparently is is an extremely simple one-dimensional system and difficult to engineer the form specific behavior anyway so i'm reading wolfram's book and i'm interested like what's he talking about you know universal communication and um he he has he has this wonderful image where he says oh you know look um here are the occurrences of progressively longer blocks than the pattern generated by rule 30 starting from a single black cell and as far as he could tell all of the possible eventual blocks will appear potentially letting the pattern serve as a kind of directory of all possible computations and that was kind of like his argument about the the uh touring universality so what's your take on that uh yeah so they're definitely fascinating objects in that regard meaning that so i think that for rule 110 in particular the argument there is that you can build logic gates essentially with them because you kind of have this and i might be messing up with the specific uh you know one of the 256 ones but like at some point you're you're able to uh shoot this kind of race in one direction and another in another direction and at some point if they interact uh the rays disappear and if they don't interact uh the array goes on and so it's like a zero and a one depending on if the rays survive or dies um and so like all the arguments for universality of these models by the way are not arguments for their efficiency in doing computation they're just like you know showing equivalence is that you can essentially build the basic machinery that you would need to perform computation which is like basic manipulation like the nand gate and stuff like that and i think that the formal proof of computation actually uh goes in a completely different direction with a different computational model but like the general idea is that one right you can simulate a turing machine just by encoding the various rules in essentially the state and that's a very fascinating thing that i've been thinking about a lot recently because uh what's what we see and and much of the things we can do with cellular automata are not necessarily uh given by the rule itself so the rule is always simple and in fact uh it's not necessarily new in a new kind of science but i think in the next work by wolfram which is the wolfram physics project uh like he kind of makes this argument that uh the rules that govern the universal cellular automata should be fairly simple right um and so like there is this bias towards simple rules but the complexity that we typically observe comes from the configuration of the states right and so all of the computation that you might want to to do with the cellular automata comes from configuring the state in the correct way in fact if you google it up it should should be fairly easy to find at some point people uh had this challenge on stack overflow of implementing a functional clock in the game of life as it turns out you can't implement the functional clock in the game of life you just need to configure the states in a particularly smart way and the state in the states will evolve on their own and they will create digits and then change the digits according to the time once uh every second and so on uh and so this is like this is what i find fascinating about the kind of computation that cas do right because you can prove that they are universal because you can do basic operations with them uh but then much of the complexity that you can actually observe comes from the actual states that you have to configure and that of course is by no means trivial which is also why people haven't been using you know cas to implement computers you use different architectures and computational models yeah and and to that point and by the way so the cook proved rule 110 you know turning complete by mapping it to another turing complete system like the cyclic tag systems or something i think it was and then and i think there's only one other rule that might be turing complete but it hasn't been proven you know one way or the other like rule 30 may or may not be touring complete i don't think this is quite fascinating right just this little simple set of rules and maybe the simplest turing complete system out there but as you were just saying the difficulty lies in going from a simple set of rules and some complex initial conditions to then being able to really predict what the large-scale behavior is going to be and i think wolfram you know points us out himself which is that um you know he he has this concept of computational irreducibility which which essentially says look uh predicting what this computation is going to do is just as hard as actually running the computation so there's no shortcuts in a sense you can't you know and it's related to to you know turning completeness you know as well or rather rather the uh you know the halting problem which is essentially a similar kind of problem it says that trying to compute whether or not a turing machine an arbitrary turing machine will halt is just as difficult as running it itself and therefore you can't yeah you can't do it without halting or without failing to halt that sort of thing um so my question to you is and this is kind of this emergence question is is this just an impossible barrier so in other words is it the case that you know something in physical reality runs runs things you know whether you believe it's a computation or not whatever there's some substrate that's executing this uh you know this this automata of the universe okay and it just it displays all this multi-scale behavior and so you get these emergent you know things happening human beings planets you know uh super clusters in the galaxy whatever is it even possible in any in any sense to predict from lower levels higher level emergent behavior or is there just this barrier that in a sense this irreducible complexity that we there's no shortcuts we can't actually predict a higher layer of emergence or abstraction from a lower layer one without just running the universe itself a very interesting question and i don't think i'm really equipped to answer that without making anybody angry i would say so that there's a lot of speculation that you can do uh don't worry i've made people angry by just asking a question so yeah so what i can say is that so first of all like we were saying before there seems to be some some degree of communication or or ability to predict what the layer above or below you will do right so if you are let's say configuring a cellular automata if we accept that that is uh how the universe runs so if you're you're trying to configure or or to study uh cellular attack at a particular level you might have some intuition a priori but what they will do right so it is possible to engineer an emergent system if you are you know even without you know running it necessarily but you probably can come up with some smart rules that have a desired behavior that you might want to have now it's a much more it's a much harder question to say can we build a particular cellular automata that acts like we see in the universe and and that's like because that requires uh us to answer the question of what are we actually trying to model and at what scale are we trying to model the physical reality are we trying to model you know a brain or a network of humans right um and and that i guess uh you can also make that irreducibility argument for intelligence in general right so do i necessarily need to model the brain in order to have intelligence or can i just approximate intelligent behaviors and and maybe just train a neural network to do that right so i think it's a much more profound question than just what is the cellular summary doing and how do i reproduce that in a computer um it's it's a question of about like the nature of reality itself and and whether it's the only possible way to obtain a particular computation is to simulate it starting from the lowest possible level now we have some evidence that this is not necessarily the case like we're having some uh some you know models of computation that are able to emulate some of the phenomenas we see right um and so i guess this is you know it gives us some kind of hope that we might be able to design one such system and people for example have been started to using machine learning to try and design cellular or someone that do a particular thing right so there might be some degree of approximation that we can achieve without necessarily computing everything starting from the lower level and this is in a sense of course wolfram's uh physics program i think he calls it is all about can we start at the lowest possible and i mean like finest scale you know i don't even know 10 to the minus 100 meters or something uh you know substrate in the form of this this hyper graph and then and then with these simple rules does the universe as we know it you know emerge from that at least on that that seems to be the uh you know the case and i mean it or rather that's that's the that's his program and could be could be possible um i think there's so many open questions there and of course it's a very active area of research and almost a new branch of of mathematics too and i or physics at least and i think um wolfram himself said if it can do if it can do these predictions like for example if we can derive general relativity or you know quantum mechanics it's it's at least a century yeah a century out so you know we have a while to wait but i'm curious what you think about the possibility of that so if it if we have this interleaved so here's a question we talked earlier about the interleaved discrete and continuous you know that as you move along kind of scales of emergence or reduction you keep coming across the need to either view things as a continuum or is a is a discrete you know spectrum of something and so it's like this alternating series you know in math that we learned that never converges right it's plus one minus one it never converges well if you go to zero so if you go to that end of infinity all the way down to the smallest possible scale do you think we arrive at a discrete system like a hypergraph like like wolfram's envisioning or is zero actually a continuum i'm curious what you think that answer is and and what you think about uh wolfram's paradigm of right so i think that i was reading about this some time ago and the question of whether at the lowest possible level you actually can see something like a cell and that kind of discreteness to it i think it's still an open question in physics yeah exactly the atom of space like that's probably from what i understand it's still an entirely open question in physics like they haven't been able to answer that uh and and the question there would be like can we actually discretize the notion of space and the notion of time because that would be an answer answering yes to that question would be a fairly strong argument for wolf from theory of you know the universe being this huge l system that's constantly rewriting itself right um but of course it's it's difficult to say like and at some point one should even ask a question of whether for us it's even important to be going at that particular level which is what you were asking before like do we necessarily need to answer the question of whether the universe is continuous to be doing interesting things and then then it becomes like a matter of goals so personally i find it fascinating to be talking about all this localized emerging computation because i see it as a potential path forward towards agi for example right um and and and the question there at that point becomes can we achieve something like that without necessarily needing to go and simulate the whole universe just to get you know you simulate this huge environment just that maybe it will develop intelligence that seems insanely costly to do for achieving something that we can describe fairly well um and so i i don't know really if at some point it will become obvious or people will be able to answer the question of whether the universe is continuous which would probably discredit the cellular automata theory uh but i think it's interesting that this idea of cellular automata especially how you know wolfram has instantiated it in one particular way but the general paradigm is much more flexible in a sense um it's interesting that we can describe things with this model and you know whatever the level we decide to start at it we also know that if we set the rules right then we will be able to observe the same kinds of emergence and so it becomes kind of arbitrary to for us to decide where do we start simulating the the universe or the system we're trying to simulate right yeah i mean there are so many things to unpack here i find that's absolutely fascinating you know so you know how does our universe work and you know wolfram was kind of making the point that underneath all of this richness and complexity that we see in physics they could just be really simple rules so is the universe an emergent phenomenon and it kind of see i mean that's it's very subjective right it's very vague it seems like it is because when you recreate so many of these emergent systems they produce phenomena that look a lot like the universe right but then there's this notion of well could we find the exact simple rules and the exact representation to create something like the universe and then there's this notion of irreducibility and then it becomes very very vague but what's fascinating though is just by applying the the same rules over and over again you can produce something that looks really really complicated and that's just not what our intuition tells us at all yeah that's true and i think like i just had this uh you know kind of follow up to to the discussion which is this uh idea that at some point you're interested in understanding the universe at some particular scale right so you you might be interested in understanding intelligent systems or you know just living system if you want to be at one level lower and so at some point the questions you need to be asking it goes it goes back to what we were saying before right you if you if you're interested in a particular level you don't necessarily need to go all the way down and start there you might just go one level lower and start modeling things there and know that you maybe maybe you will introduce some approximation errors but if you do things well enough then you will kind of see that emergence and you can build on top of that layer of player after layer right and and and i agree that it is fascinating that this kind of emergence appears to be like a sort of constant throughout the different layers right so you it doesn't it doesn't really matter the specific rules that are acting at a particular layer right it what matters is that the same kind of computational engine appears to be working at every layer and so you might have the similar phenomena happening at the physics level or at the cell level or at the society level and the rules will change and in fact michael actually makes this very interesting argument about different layers being recognizable by the kind of goals they're trying to solve right and so for example you might recognize the a layer the layer of society because it's trying to solve the goal i don't know of of surviving as a species for example um and the rules that you will find at that layer will be in a sense emerging to solve that particular goal exactly like i don't know in a very similar simple level atoms will have very simple rules that just try to satisfy some electrical or physical constraints right and so as you move up and down you will find different rules but the same computational principle so i will have a rule that tries to solve an objective through localized computation and and that is what i really find fascinating about cas and and just the general idea of of this kind of models because they appear to be reasonable to explain different layers in this architecture and so this is really interesting to me so so i mean this is getting to um something we were just previously talking about with a with our other guests about um formalizing what happens in that emergent space and actually when we speak with ai alignment people they bring up asimov's laws and you can get into you know utilitarianism versus deontological but um you know the thing is even because we're gonna get on to your work when we're talking about morphogenesis and what's fascinating there is that actually you almost want to um evaluate the rules that you're creating based on the emergent phenomena and then there's this thing about how do i how do i formalize the nature of that phenomenon i mean if i was steve stephen wolfram for example how would i formalize universe like behavior right oh this thing's emerging and it looks like the uni looks like the universe how do i formalize that so yeah i think what you're describing is a property that we might find in different aspects of life right once you have the same rules applied in an interconnected system of sort and so there's like this interconnected graph of of information flow and how you act on that on that graph so as you have that system and and it goes on over time probably at some point you will see some emerging phenomena and i guess at that point the question is how do i control that emergence like how can i introduce some sort of metric of my own to try and understand what's going on and how to control it which to which i would answer that it's impossible to tell a priori like it's so task dependent and it's so dependent on what you're doing that like if you are trying to optimize for something then you should have you should go at the level that we were saying before like you should go at that level of abstraction and trying to understand what are your goals at that point right so it could be that you're optimizing for the overall success of the company or or whatever it is that you're developing code for right and so you might introduce that as a higher level goal and hope that what's happening at the lower level gets optimized to it to essentially achieve that objective right which nature has done to selective evolution but if you're trying to introduce that signal into your own control process uh you you might need to to do some particular actions at the lower level so that maybe you can achieve that higher level objective i know but this is the tyranny of objectives though right there's the shortcut rule there's all of these because as soon as soon as you formalize something you you block stepping stones and and you exclude the actual behavior that might lead you to where you actually want to go yeah it's fascinating but um i do want to move on a little bit i want to kind of slowly move the conversation towards your work but i want to go via this fascinating article that you that you shared with me about morphogenesis and actually it was by this guy i think he's at google he's called alexander maude vincent vince and the article was called growing neural cellular automata and it was performing uh morphogenesis and more for jet i mean that sounds like a ridiculously complicated word but it's about um you know it's essentially how if i perturb something or if i have some initial starting state how could i how could i create something that i want to uh create right so um this was using a cellular automata on images on the 2d plane and he was able to design a cellular automata and an update rule which would quickly converge to a desired image so there was a a picture of a lizard and you could you could damage it and perturb it and then it would it would just come back to the lizard and it's just it's it's incredible to me right because um it's like from these local low-level rules you could actually create something that had global coherence yeah and i've just never seen anything like that before so um tell me a bit about that article yeah that's amazing uh so that article in particular uh was inspired by uh this idea of the flat warm which is like a tiny creature that if you take it you cut it in half like both halves are able to regrow into the whole thing like independently and so the the question there was how can this animal be doing this like because you know the decision to grow is a local decision and the decision to stop must be a global decision so there must be something inside the growing process that's somehow coordinating the global shape right and and in fact this this idea of morphogenesis in cellular automata was already uh present in literature of course um and people like had been doing it uh even ten years priors uh you know trying to generate uh or to generate like uh flags so you have like three states uh cellular automata with uh different colors and you try to arrange the cells of a particular color in a particular region of the images that you can generate like the flag of italy or the uk or whatever right so you can kind of uh see that it grows into that shape um and what they did in that paper was bringing it to the absolute next level right so it was looking at this platform and say okay it's able to regrow into the full thing okay can we do the same thing with a neural network and so can can we learn to do the same thing because this goes back to probably what we were saying before which is designing these kind of rules can be insanely hard like you don't know what the rule is that just by letting it evolve locally on inputs of pixels it eventually gives you the lizard or you know they have this smiley emojis they have different kinds of emotions and images and so how do we design that well the answer in that case was well we'll take a convolutional neural network which if you look at it has essentially the exact same kind of shape as the cellular automata so it has like a local three by three kernel uh that updates the the state of every pixel uh synchronously so it's like there's there's a lot of overlap there and they said okay let's just uh train this neural network in a recurrent way so we're just gonna propagate forward and then use by propagation through time to adjust the weights so that after some steps of computation we land in a particular object or objective state um and and so this is this is what they did and they were able to to to make this very very robust actually so they were able to have the image grow from a single pixel into the full thing and then you can start trying to make that robust perturbation so you put what happens if i cut the image in half i would like to grow it i would like it to grow back into the full thing and you can actually can introduce that type of input output examples in the training process and so that's what they did and then they have this whole analysis of what the cellular automata does or the neural cellular automata does for example they let it evolve way past the training horizon that it they trained it for uh and so they let it compute over and over and over again and at some point what happens is that it breaks the stability and it kind of starts producing the same pattern everywhere on the image so it kind of the single lizard becomes a pattern of textured lasers everywhere and in fact they actually have done some really cool follow-up work where they actually do the same to actually generate textures so you can use that instability to generate textures on an image um which is very cool and uh and nature like if you look at the images well it is possible to because i know they did a whole bunch of stuff to robustify it if you like but it is possible to make it lose its global coherence so like on the lizard i would press in the middle of the lizard and if i kind of oscillate the mouse cursor a little bit i could make the lizard grow another pair of legs and feet yeah but what fascinates me is is that this is i don't think it was synchronous either i think they um to make it resemble real life a little bit more they they formed the the update rules randomly and and stochastically but um it's incredible though isn't it that just by having essentially what is a filter back i think it's a bit more complicated than that they have a notion of if it's growing and dead and alpha they've got about 16 different values of are they not just rgb for every single pixel but but but basically it's it's a gridded cnn and you're just updating this thing and you get an insane amount of global coherence from these bottom-up rules and there are loads of people we're speaking to that that think that ai must be top down you know it's not possible for it to be bottom-up but but this is this is fascinating right it is it is and so one thing that i find fascinating is that there is absolutely no reason why this should work like at all there is nothing that we can observe that says that these kinds of rules should exist at all this this model in principle is it's like it's too simple for it to actually work but in fact it turns out that these models that these cnns in particular uh and then in like in my own work we proved it for for genetic graphs but like these are universal models so if there is some kind of computation that can be expressed in the cellular somata and like you know by extension as a gnn as a cnn then the cnn can implement that computation and i think that it is really fascinating that this computation exists so what what this paper answers to me is it's not the question of whether can we do it with a cnn but the really fascinating thing is that yes this can be expressed as a process that you know iteratively and locally kind of grows the image into what we want and that's a real fascinating thing like that they were able like to actually do this at all and it is not at all obvious that they could and but yeah they could do it yeah i think the important thing for the listeners to know is that yes it has this the small grid which is its input however the cellular automata is absolutely not the simple rules you know that we're used to right like it's actually a a a relatively deep you know neural network behind that taking a look at that input deciding you know what to do is that correct oh yeah i mean it depends on what stream by complexity like i i would say that you know being able to compress the information of the image into you know a relatively small kernel of a cnn it's still a fairly simple way of doing it right so maybe it's not as simple as you would have the game of life so you still have several thousands or probably even millions of parameters in that neural network but it's still like it's encoding a lot of information and especially what's fascinating to me it's not that it's just like outputting the image one shot which of course you know there would be better better ways to do it but the fact that it's doing it like iteratively like this is a process that by you know by applying the same rule at every pixel and doing it doing so iteratively it's able to output the image and so this is what i think is really fascinating yeah so i agree with that i'm just i'm just trying to set a baseline here so one thing for the listeners to understand is that there is this quite complicated you know neural network that's looking at the small window and then deciding you know how to update so it isn't old-school cellular automata that have like update rules that can be written down in three lines or something okay and then the other thing that that very much interests me about this project in particular is you know i get on this soapbox pretty often okay that uh a neural network as it's typically conceived which is a neural network takes you know some some inputs and it does what is ultimately equivalent to you can always unroll it as a forward pass through a fixed depth you know uh thing and then you get an output by itself is not turning complete that what you need is the ability to do this iterative you know computation if you will on kind of a working space and that's exactly what we have you know in this work is there's this plane and it and it's it's learned this uh computation that if it gets iterated over and over again can do very fascinating things and so i think it's just important for everyone to understand that without that iterative capability without that kind of working space without that temporal dynamics you know you don't get this kind of behavior no exactly exactly and that's that's exactly what happens right and like for people working in the gnn community this would be much more trivial to to see but like what's happening is that by recurrently feeding the output back into as input into the network what's happening is that every cell essentially is able to see farther and farther away from itself right because like the receptive field in a sense aggregates information from the neighborhood but then so is doing every other receptive field of every other cell so after two iteration you will have reached like a neighborhood of size i guess four by four instead of three by three uh and and so you you go on growing like and this is exactly what we're doing rough neural networks like every layer lets you go one step further and so what's happening with this iterative computation is that as the iteration progresses progresses uh every cell gets access to essentially a more and more large view of the of the world system and at some point this information kind of bounces around like waves in a pond if you if you think about it and um and and it comes back at some point but that but that's the key but that's the key right there is at some point and the problem is that with with irreducible computations you don't know at what point that is and so this t this t parameter is open-ended the only thing you can do is sit there computing computing computing that's why it can never be compressed into any fixed number of layers it's like you have to have this open-ended you know t you have to have an open-ended number of layers yeah i wanted to unpack this sort of as well because that there's something magic about this and it's exactly as keith said right you know people even say consciousness itself what what's emergent and magical about it is it's reflexive property the fact that it's constantly going random random circles and when we look at this cellular automata even this gridded cnn version it appears very lifelike and that's and that's why because we used to think that neural networks were kind of like performing iterative computations and now based on our conversation with randall balestrurio we know that actually they're just decomposing the euclidean space up into these kind of polytopes and and the amount of computation is finite so it's a completely different type of computation but um anyway i i wanted to move the the discussion on to your work danielle so we've we've we've discussed these um gridded um uh cellular automata with with cnns and then graph neural networks are absolutely fascinating because they they extend the notion of a cnn into this world where you can have um any structure at all so you know the the concept of message passing for example it extends the cnn right so you still have this notion of a neighborhood but you're not on this gridded plane or manifold anymore and you've done exactly the same thing right you've done this morphogenesis but um with a point cloud so can you tell us about that it's absolutely amazing yeah it was fun because so that whole paper was about exploring this this very idea of can we come like people have been complicating everything about cas and at some point the question becomes can you complicate the underlying geometry and you get the graph cellular what's on myself um and and then like we were trying to show that actually graph neural networks are universal engines to perform this kind of computation on on gcas uh and so the task at that point became like okay can we do more for genesis on a graph what does it look like on a graph right then and let's say the typical visualizable example on the graph is to take is to take something that somehow represents space something that we are used to interact with as humans uh and and so this was the point cloud so points in in space um and we took several of them we took like this bunny like thing we took a graph that more or less represented a writing so some some letters so something that has like a spatial kind of notion to it oh and we were trying to ask the question of like we were saying before does a rule exist that starting from a random configuration of points actually morphs and and feel like the shape is actually a real shape if you think about it so morphs these points into this coherent shape and and again the cool thing is that this must happen just through sheer local message passing so every node at some point will read the neighbors will read where it is at that point and we will decide where to go next and just by this continuous exchange of information which at some point again bounces around to the graph and by the way we have evidence that no many not not many steps are needed for this kind of information to bounce around what you what you really need is that you at least need to have as many exchanges as the diameter of the graph meaning the the most distance that you have between any two pair of nodes um and so like if you can do that at which what you see is that in fact there exists a rule that takes you from random points to bonnie and it gets there you know fairly stably as well you can train it to be fairly stable and so for example what we saw is that uh because if you think about it there are like these two regimes that the neural network must learn right so it must learn to go from random to bunny and then from bonnie to bunny so it has to remain stable once it gets there right which is what you try to do with the lizard like you would like the lizard to remain a lizard even if you perturb it and the same applies here you would like the bunny to remain a bunny even if you perturb it and so once it gets to the bunny it has to stay to the bunny and what you see is that in fact the network is able to uh you know very quickly put everything where it's supposed to be almost immediately like in two or three steps you know the random point cloud becomes essentially a bunny and then it learns to gradually adjust the remaining points and it is fascinating that everything is happening as a as the same rule gets applied everywhere uh and so we kind of explored that and uh it worked fairly well although sometimes like we had this weird effect that you probably get as you train a recurring neural network in the dynamical system where so instead of like converging immediately uh it starts oscillating around the bunny or around whatever target you have and it does this weird oscillations where it goes from bunny to random to binding to random and so on forever and it's really fun when you animate it because like it looks like the bunny is stomping on the ground because there is this weird oscillation on the foot uh so nice it was really it was really nice to to work in that uh in that space amazing i mean in in a way this is an entirely new as i know you got the work published in europe's which i think should give an indication of how impactful it is but you know you can think of it as an entirely new model of computation in a way i mean you said in the paper that you could apply it to things like swarm optimization and control and modeling uh epidemiological transmission and even improve our understanding of complex biological systems in the brain right yeah forget all that i'm just looking for a first-person shooter where i'm playing in a in a world that's just a cellular automata that self-repairs anytime somebody does damage to it yeah you can do that type of stuff like that that's the cool thing about this is that once you break free from from the grid in a sense you're like every time you have this kind of local interaction and the cool thing is that the interaction can signify anything at that point like it doesn't need to be a discrete projection of 3d space anymore which is what did what the image is like you project the 3d world on the 2d plane and then you discretize that so it can be anything it can be you know relations between humans it can it can be relation between neurons and whatever is like doing this kind of computation through local exchanges in in whatever geometry of the cells you want to have and by the way one thing it also tested was the setting of the dynamical graph so a setting in which the graph changes at every iteration if you want and so all of that like kind of unlocks a lot of possibilities because now what you have is that again if you can't specify the correct objectives which not necessarily use it's not something you can do always or you don't know always how to do it but if you can and and if everything turns out to be differentiable which is another probably big limitation um then you can train this object kind of end to end to give you the desired behavior right if it is yeah is that the idea with the so with the protein uh protein modeling or synthesis you know trying to find the dna sequence that ultimately will give you the desired protein structure what's the connection there uh so the connection is that there is no real connection so far like i'm not trying to apply this type of computation onto the protein design problem because it introduces like an extra layer of complexity that i don't think we're quite there yet to be able to use this gca stuff reliably to solve any kind of problem and i'm not even entirely sure that any function should be expressed as this sort of recurrent computation on graphs so what we're doing with the protein design is is just trying to solve different kinds of sub problems which is for example uh what function should the protein have and what does that look like in terms of structure and once i have the structure what does the sequence look like so there is no idea of like recurrent computation in that space but in a sense like it's it's similar in that you could say you're trying to find let's say a description for something or for some for something that has a higher level behavior uh but in that regard like this the ca stuff kind of lives on its own for now i'm hoping that we'll eventually apply it to you know modeling protein dynamics or something like that well what i'm curious about is suppose uh because the problem of protein folding itself is is obviously difficult but suppose you could design a cellular automata that could take as input a dna sequence and then it the some type of relatively simple you know input range on the dna and then it could run run run iterate and give you the folded protein as a result then if that cellular automata was actually invertible you could run the reverse computation to get back to a possible sequence that would have given you that is there research into invertible cellular automata of this nature okay so let's let's unpack that because so there are two things uh the first is that alpha fold actually kind of works like that meaning that uh it has this refinement procedure meaning that it predicts the structure and then it kind of feeds it back and then tries to iterate on this on this predicted structure um and there is also work uh on reversible cellular samasa although uh i have yet to hear uh about reversible neural cellular because that would uh imply reversible you know your neural network and that required the whole thing to be integrated essentially right um but so yeah what would you say is probably a possible way to do it uh what i'm wondering if it's you know the best way to solve the problem because at some point what i came to realize working so i had i had the luck to work with many uh you know real scientists in a sense meaning people that actually work with you know the brain or biology and then people that actually have a deep biological biological knowledge um and what i came to realize is that at some point it becomes a matter of solving the problem so it's like solving the problem is more important than the way you solve it in a sense right and so if you're trying to solve protein design it feels like an exercise in style to try and do it with a cellular automata for example because you don't have like any strong evidence that the the function that goes from structure to sequence is actually one one such recurrent kind of computation that you need to do and so at this point like what we're trying to do and this is actually like something that i've tried to force myself to do because it can become difficult at times is to not try and use the big guns immediately so you kind of want to step back and go back to basics and you know start from a multiplayer perception and see what what it does and see if it can actually work um and try to you know solve the problem actually solve the problem so you start simple and you you know stress your neural network until you until you hit a wall and once you've hit that wall then you try and make them a little more complicated and you try and see if you know different things could work and i'm i'm seeing that that's probably a good way to approach this this problem so right now multiplayer perceptions are the way to go well wait have you have you taken it one step further towards simplicity and said let's start with a linear model and uh oh right then yeah no but you see that that that's when sometimes you already know that some models are not good right because the thing that i'm trying to do is like highly non-linear and so like a linear model probably wouldn't work you can try even i mean you can try and you should try probably but you know it won't work what i'm talking about is whether you know you already need to go and look at that structural source of information do you need the graph immediately or can you solve the problem let's say from the sequence or from you know just the coordinates without without the graph representation and so that's what we've been doing a lot in this in this proton design space which as i said is kind of orthogonal to the work on uh on cellular automata because like with cellular automata the question is whether this computation exists if i can formulate the objectives can i find the solution with a neural network like it's a different set of questions and they're more like i i don't want to say essential but like they're at a more basic level than just you know trying to actually solve a problem it's more asking questions about the universe or this particular computational model and see if the there are answers right yeah that this whole thing blows my mind i've only i mean i'm i've only just discovered this and um from an engineering point of view i'm fascinated by this notion of having systems that can be self-healing in some sense or even having multiple agents that are going around my system and kind of repairing things that get broken by people so i guess i mean we'll slowly rap but i wanted to um if there are folks that are interested in some of these topics we've spoken about so complex systems theory um you know things like uh craft neural networks and so on and the work you're doing where should they look and also i'm interested just to know personally what other areas are you interested in right uh so as far as resources go uh you you can approach it at different levels so uh if you just want to learn about cellular automata there's tons of resources out there and what i would suggest people do is that they go and look on twitter which is weird as it may sound is where like the actual hacker community is is doing this type of insanely complex cellular automata that really have like once you see them you it's really hard to to think about them being automata because the behavior is just so complex and lifelike and so sometimes you will find you know some some of these hackers in this community that that showed you know their center automata and then you know side by side comparison with the real world living system and they're exactly the same moving in the same way and it's like okay so is the ca predicting what's happening in biology like what what does that tell you about the nature of the world and so yeah so i think for the record i think you may be the first guest that's recommended twitter yeah but but that's the reason for that i love it that's awesome that's what it is like as as weird as un unscientific as it may sound that's that's where interesting things are happening like you know twitter is the new archive huh yeah twitter is the new archive and github is a new archive like you will find you know bleeding edge cellular automata on github you don't necessarily see them published at all right now like uh we are starting to see this ncaa stuff popping up on you know in europe's icml aclr and it's like starting to make a breakthrough but you know it's more there's tim there's hope for us so you and i may yeah become researchers yeah and if you're let's say if you're interested in more uh academic work uh especially on the biology side there is the work of the entire lab of michael 11 um and and it's like he does a lot of work um regarding studying emergence in real biological systems um and so it's like uh they study uh the cells of frogs and they they take cells out of frogs and see what they do in the different environment and they were able to create this like small biological robots that essentially grow out of skin cells to create what they call these xenobots and and that whole group and by the way uh he is also um part of the that uh morphogenesis paper with cnn's um and so like that that whole group is uh is doing excellent work in that regard i would say they are the pioneers of this entire idea of emergence i discovered a youtube channel called emergent garden and it's by a guy called max robinson and he's got some really cool videos um i definitely recommend you guys check that out right okay but does he have a twitter feed because that's the real point you can also keep an eye out as i said like papers are starting to pop up in terms of and and this is all in terms of neural cellular automata if you look at the literature on cellular automata on their own like there's a whole bunch of literature that goes back to the 60s so like any kind of variation on the theme has been explored and proposed and even for graphs um and so like right now we're starting to see this convergence between neural and uh and cellular in a sense but there's tons of literature in in that whole space uh if you just approach it from a perspective of dynamical systems and and you can see actually really interesting things so tim before was asking me about um this idea clustering the behavior of rules uh according to you know plus one plus two plus three plus four and as it turns out there are pretty clear entropy measures that naturally cluster the behavior of rules uh according to their you know abstract uh behavior and so like it's uh it's really fascinating stuff and you can really see the class four rules actually have their own space in this in the center in this entropy description um so yeah it's um there's a lot of things you can look into uh but yeah twitter all the way man if you if you want to see nice nice images that's that's where you go fascinating yeah i mean on the entropy thing i'm sure carl fristen and even people like um kenneth stanley you know the people that study artificial life they think of um life like uh i don't know if agents is the right way to do it but you know the information accumulation is something that's super important for for uh the characteristics of of intelligence and one one fascinating concept is um the edge of chaos uh and the work by langton for example and and that's also something that pops up continuously in uh in the right time whatsoever and elsewhere that's all oh yeah sure sure sure well dr danielle uh greta rolla it's been an absolute honour this has been a really fascinating conversation actually i think we need to do loads more content on on this area this feels like an area that we've just not really done done enough on so yeah this has been amazing thank you so much thank you very much for having me it was a real honor oh thank you pleasure remember to like comment and subscribe we love reading your comments i really hope you've enjoyed this episode if you don't mind please rate us 5 stars on the apple podcasts app and we'll see you back next week

Channel: Machine Learning Street Talk

Views: 14,303

Keywords:

Id: MDt2e8XtUcA

Length: 115min 29sec (6929 seconds)

Published: Fri Apr 29 2022