#104 - Prof. CHRIS SUMMERFIELD - Natural General Intelligence [SPECIAL EDITION]

Chris tell us about the um the alien [Laughter] well well I I um I guess that you're referring to in my book I talk about uh the chapter about knowledge I start by asking like what what do you know what do we know and the point I make is that we know a lot of stuff and the the link to the alien as I imagine you know what would it be like if you had to explain everything that you knew through an alien from another planet and the the point is to highlight the distinction between Theories of Intelligence that focus on thinking and those that focus on knowing because those that focus on thinking have very often overlooked exactly how much we know and the rise of machine learning and the Deep learning Revolution as the the dominant Paradigm for AI research has been fueled obviously it's been fueled by data but implicitly by a premise which is that knowing is as or more important than thinking when it comes to building intelligence systems so how do we because neural networks were really revolutionary because rather than thinking they learned how to ground and represent the knowledge in the world how does that work well so neural networks are just large systems of um that that learn they're just large systems that learn functions through tunable weights and those weights find what they do is they express data by re-representing it in the internal representations of the system and it's the expressivity of those representations that gives neural networks their power because they can in the similarity structure of those representations you can the system is capable of representing what goes with what representing how things are maybe similar or different and when I say things I don't just mean objects you know birds and buildings and people but also abstractions so um representations of the relations between items between people between places that then can be used as building blocks for really complex forms of inference and and computation and you said in the book that the theoretical information capacity of our brains is what potentially two to the power then but um with the heavy and learning likely um significantly lower than that but still very big so it's not so much a capacity problem it's a representation problem and you gave an example of um someone with a weird brain condition where they had a photographic memory of it they lost the ability to learn these kind of categories groupings abstractions could you talk to that yeah so the point the point is to illustrate that you know it's not the overall volume of knowledge that is important it's how it's structured so hypermnesics who are individuals who have exceptional memory often um they often experience it as a kind of debilitating condition because the lack of structure in their memory precludes them from using it in useful ways so this particular individual could remember everything that ever happened to her but she wasn't any better at passing exams or understanding the world than you or I were interesting and just finally on on abstractions how does how does our brain learn abstraction Well if I knew that then you know probably I wouldn't be walking around here with you uh well maybe I would I don't know um yeah we don't we don't know um but I think that you know if you if you want if you understand understand an abstraction as being a representation which goes beyond the physical properties of the data then you know kind of view there are various ways of thinking about it but what one way that I emphasize in the book is thinking about grounding in space and in actions and that's a theory which is coming to the fore in euroscience right now interesting interesting um Chris it's been an absolute pleasure thank you so much thank you so much Christopher Summerfield is a professor of cognitive neuroscience at Oxford University he's a staff scientist at deepmind and he's a fellow of wadham college Chris's work is concerned with understanding how humans learn and make decisions in his lab at Oxford he studies learning in adults using computer-based tasks he's interested in how humans acquire new Concepts or patterns in data and how they use this information to make decisions in novel settings he simulates learning processes using computational models including deep neural networks and he studies the brains of humans during learning and decision making using non-invasive methods such as F MRI and EEG now he has just written a book called natural general intelligence how understanding the brain can help us build AI I really really recommend you all go and buy this book it's I think it is the best book on AI that I've ever read the chapters are turing's question the nature of intelligence the language of thought the structure of knowledge the problem of abstraction the value of action the control of memory and a picture of the mind it took me about four days to read it I was glued to it I was taking notes every two seconds and at the beginning I was thinking that I would know most of the stuff in this book and there were so many details that I didn't know it's beautifully written it's clearly been in a labor of love for Chris so really recommend that you buy the book and I'm going to read a bit of an excerpt from chapter two with Chris's permission I've summarized it a little bit but just to give you folks a flavor of what we're talking about here so yeah the the second chapter is about intelligence and he told us the story of John Von Neumann he was one of the most remarkable polymaths of the modern era whose achievements in mathematics and physics Computing and economics have left a lasting Legacy on the world born in Budapest in 1903 Von Neumann was a child prodigy mastering calculus by the age of eight and Publishing a new theory of ordinals at 19. he went on to make major contributions to the fields of Game Theory quantum theory and computer science and even independently discovering godor's incompleteness theorem and defining the core theory of axiomatic rationality which underpins modern economics Von Neumann was renowned for his mental agility and prodigious memory he could perform complex calculations in his head devise new proofs and theorems on the spot and recall long passages of text from books that he'd read many years before Albert Einstein himself is said to have described him as one of the foremost mathematicians of our time for decades psychologists and ethologists have sought to Define intelligence in humans and other animals and AI researchers have looked for ways to measure intelligent behaviors in artificial agents now Summerfield said that a common theme for these twin Endeavors is the Quest for generality for agents that can solve many problems in many different domains but whilst it's easy to quantify performance on specific narrow tasks defining general intelligence has proved remarkably Elusive and this is an urgent problem for AI research if we can't Define generality then we can't chart progress towards artificial general intelligence now our everyday conception of intelligence is grounded in the notion of human or superhuman versatility which is to say it's anthropocentric we can fluently handle multiple tasks and deal deathly with diverse environments and Summerfield said that remarkably there's some quality in our human neural apparatus which allows us to to perform a great Ensemble of tasks weaving them together into an Adaptive behavioral repertoire that is well tailored to the complexities of everyday life in 2007 whilst completing his PhD Shane Legg proposed a definition of intelligence which measured an agent's ability to achieve goals in a wide range of environments but this appeal to generality per se doesn't offer a concrete recipe for building AI nor does it provide us with a hard and fast way to operationalize and measure an agent's mental abilities one might imagine that if we could quantify generality of function by presenting AI systems with a broad range of tasks and measure their fractional rate of success and failure then that would be what we needed critically though it offers no guarantees that an agent will solve tasks living outside the Horizon given by the AI test suite and this problem is well known in machine learning research it's related to a fundamental indeterminacy principle often known as good Hearts law that when a measure becomes a Target it ceases to be a good measure now Summerfield said that we need to move Beyond definitions of a general intelligence that gesture vaguely towards human competence or that argue for exhaustive tests without constraining what those tests might be but how do we do that of what does intelligence consist and from whence does it come how can we measure its size and shape across individuals and species what behaviors should an intelligent agent be able to perform and under what circumstances to delve into this question Summerfield asserted that we must consider how mental abilities have been measured in humans what conclusions have psychologists drawn about the nature of intelligence he says that the answer to these questions is complex and will likely remain so for many years to come but one thing is certain The polymathic Genius of John Von Neumann serves as a shining example to the power of intelligence and of the importance of developing a definition of general intelligence that can be applied to both humans and machines the debate surrounding the nature of human intelligence is one which has been going on for centuries in 1904 Charles Spearman proposed that intelligence is a single unifying Factor often referred to as the G factor which underpins all forms of mental ability spearman's positive manifold implies something fundamental about the human mind if you'll get it one thing such as algebra then you're probably good at other things too such as crosswords it suggests the existence of an underlying factor which scaffolds human mental ability in contrast Howard Gardner argued that there are multiple forms of intelligence and that each person has their own unique intellectual footprint the question of whether there's one human intelligence or many has been the subject of much research and debate among cognitive psychologists who try to answer it empirically using statistical tools to look for patterns of correlation between different cognitive variables some have argued that it is indeed possible to measure Intelligence on a single axis spearman's notion of a positive manifold implies that if a person is good at one thing then they must be good at other things too and this has led to some AI researchers such as Pedro Domingos and Jeff Hawkins to pursue a master algorithm which could underpin superlative machine intelligence however there's a problem with this approach the nature and structure of mental ability depends on the instruments and units of measurement used in absence of a ground truth definition of intelligence tests are usually validated by comparing them to other tests leading to an endless circular process intelligence tests thus act like a mirror reflecting the theoretical preconceptions of their designers who tend to come from Western educated industrialized rich and Democratic backgrounds and inhabit an intellectually enriched environment as such intelligence tests often favor those who've enjoyed more privileged schooling such as Von Neumann who grew up in the company of budapest's intellectual Elite so this has been demonstrated when intelligence tests are applied in a cross-cultural setting children from rural communities often underperform on standardized tests compared to their Urban counterparts which misses the point that these communities devote considerable time to acquiring tacit Knowledge and Skills which relates to their environment for example children in Kenya learn about the medicinal properties of local plants while those in Alaska develop a deep understanding of their local environment this basic indeterminacy has important consequences for measuring the dimensionality of intelligence whether each person benefits from a single Wellspring of intellectual potential or whether mental abilities are cobbled together from many distinct skills meanwhile practical skills can reap significant benefits as they're often recognized within the community and admired for their know-how the gap between mental ability measured in the lab and in life is clear to see a classic study of children from the coastal city of rasifa in northern eastern Brazil found that children often contributed to the local informal economy by selling local snack foods such as grilled corn on the cob or coconut milk which required them to perform rapid on-the-fly mental calculations however they often performed very poorly at school and graduated with little or no formal expertise in mathematics all of this tells us that people can either Excel or flop in the same cognitive tasks depending on whether the context is familiar from everyday life or the test is administered in a formal setting it also shows that intelligence tests offer a limited snapshot of mental ability and can Overlook a wider set of contextually or culturally relevant talents Summerfield said that because of its potential for bias and discrimination intelligence testing has been bedeviled by questions of validity and solid with political controversy it is clear that the traditional definitions of intelligence have been handed down from a western philosophical Canon which emphasizes rational inference as the basis for cognition which has in turn led to tests which favor those who have enjoyed more privileged educations to truly understand intelligence it's necessary to look Beyond paper and pencil tests and consider contextually or culturally relevant talents only then can we truly understand the full scope of human intelligence the Swiss cheese situation in 2016 the match between alphago and Lisa doll highlighted the gap between human and artificial intelligence and the importance of common sense in decision making AI researchers have made impressive advances in equipping agents to tackle complex problems but they've neglected the everyday tasks that require the ability to solve a broad range of problems AOA systems are accurate but lack robustness and can be prone to making silly mistakes such as classifying a school bus as an ostrich or running towards an angry polar bear in a video game the criticism of human intelligence tests is also relevant here these tests focus on mathematical logic and spelling skills and Overlook the diverse open-ended problems faced in the real world Iraq is a great example of this being a Nobel prize-winning physicist yet having difficulty mastering basic social graces similarly AI systems may be able to beat humans in complex tasks such as they're improving or image classification that can be completely useless at more basic tasks such as fixing a washing machine or making a grilled cheese sandwich adversarial attacks are also a useful tool for exposing the vulnerabilities of trained networks these networks can be fooled into classifying images incorrectly with just a small amount of distortion and such errors can have a real world impact such as when a deep learning algorithm mislabeled images of African Americans as guerrillas leading to Google losing Revenue generative language models are the largest models that we have and can learn from huge swathes of the internet however they can make very bizarre errors such as gpt3's aversion to grape juice which suggested it could lead to death so some of you folks might remember this example from Gary Marcus a couple of years ago when GPT 3 first came out he prompted it thusly you poured yourself a glass of cranberry juice but then absent-mindedly you poured about a teaspoon of grape juice into it it looks okay you tried sniffing it but you have a bad cold so you can't smell anything you're very thirsty so you should dot dot dot gpt3 continues you should drink it you're now dead the moral of the story is if you're gonna drink something make sure you know what it is so yeah no comment AI researchers are therefore faced with the challenge of training deep networks in a way which is safe fair and aligned with human values in order to bridge the gap between Ai and Common Sense they must find ways to make AI systems robust and able to tackle the everyday problems which humans face such systems could be able to cope with a wide range of environments from the complex to the mundane and should be able to make decisions with confidence and accuracy only then will AI truly begin to approach the levels of understanding humans possess alphago had a powerful model of go built into it and it's what allowed alphago to master the game but even within this extremely narrow domain it turns out that alphago was prone to gross misestimations of the value of a particular board state in fact it was this reason that the scorecard against Lisa doll ended up being 4-1 not 5-0 on move 87 in the fourth game after a particularly stunning move by sadal alphago lost the plot and began to spew out a train of terrible moves as the alphago team lead Dave Silva said in an interview he said I'm quoting alphago in around one in five games would develop something which we call a delusion which is a kind of hole in its knowledge where it wasn't able to fully understand everything about the position that would persist for tens of moves throughout the game and we knew two things we knew that if there were no delusions then alphago seemed to play at a level which was Far Beyond any human capabilities but we also knew that if there were delusions the opposite was true so this is the phenomenon of the Swiss cheese some of it is brilliant operating at superhuman performance and then the next thing you know it's Gone Bananas you're in the hole in the Swiss cheese and the problem is it might be impossible to discern when this happens so the goal of artificial intelligence research is to build systems which can surpass human intelligence in terms of their reasoning and inventiveness in order to achieve this AI researchers have long debated whether they should draw upon intuitions from Psychology and Neuroscience When developing AI systems on the one hand this knowledge can provide valuable insights into how humans think and how to structure AI systems to replicate this behavior on the other hand some AI researchers argue that this approach is misguided and that the best Solutions are those that are General and can leverage computation humans are incredibly complex creatures and they've been studied extensively by psychologists and neuroscientists over the last century and this research has revealed a great deal about how humans behave and interact and the theories they form about causal processes in the physical and social worlds the knowledge has been expanded and refined by synergies between psychology neuroscience and computer science leading to Santa suggest that AI systems should be built in with some of this knowledge for example psychologists have demonstrated that human infants enter the world with strong inductive biases which shape the way they understand the ways in which objects and people behave and interact and so perhaps AI systems should be biased to do the same however the AI Pioneer Rich Sutton expressed his dismay to this argument in a widely cited blog post called The Bitter lesson Rich Sutton said and I'm quoting we have to learn the bitter lesson that building in how we think we think does not work in the long run we should stop trying to find simple ways to think about space objects multiple agents or symmetries instead we should build in The Meta methods which can find and capture this arbitrary complexity we want AI agents which can discover like we can not which contain what we have discovered he went on he said the biggest lesson that we can read from 70 years of AI research is that General methods that leverage computation are ultimately the most effective and by a large margin the ultimate reason for this is Moore's law or rather its generalization of continued exponentially falling costs per unit of computation so for Saturn building in handcrafted computational constraints based on our understanding of Psychology and Neuroscience is the problem not the solution he cites canonical examples of how progress and AI was only achieved when researchers jettisoned their intuitions for example chess was solved when systems that incorporated domain knowledge about gambits forks and skewers were replaced with a massive brute for search deep blue crunching 200 million moves per second against Kasparov similarly computer vision took off when systems stopped entertaining handcrafted Solutions and instead let powerful computation and densely parameterized convolutional neural networks such as alexnet don't tell schmidhuber to do the heavy lifting the neats and the scruffies Summerfield said that the question of whether AI should rely first and foremost on a computational principle or rely on a patchwork of data ideas and observations conferred by the researchers is one that is as old as AI itself in the 1970s the cognitive scientist Roger shank noted that AI researchers could be Loosely dichotomized as either neats or scruffies The neets Who included many of the Dartmouth summer workshop groups you remember that group of Engineers from scientists from 1954 and they envisioned AI as a system which could reason symbolically over inputs they advocated for a minimalist research agenda based around the search for provably correct Solutions like The Architects of muziro they believe that their systems should learn without the polluting influence of human data the scruffies by contrast who included early AI researchers working on language systems favored a piecemeal approach which matched the heterogeneous set of problems encountered in the world with a diverse set of computational solutions the very notion of hacking tinkering unsystematically with a system until it worked originated in the Massachusetts Institute of Technology lab of Marvin Minsky so Minsky was an original scruffy in the 1960s AI still has its neats and scruffies even if today the neats are more likely to appeal to searching for computational Purity in a deep Network rather than a symbolic processing system the veritable scruffy Marvin Minsky once said what magical trick makes us intelligent the trick is that there is no trick the power of intelligence stems from our vast diversity not from any simple perfect principle recently AI researchers have shown a poissant for writing papers with the title X is all you need where X is the name of an algorithmic innovation perhaps portraying their hope for a simple clean solution to the gnarly problem of intelligence others have claimed that the secret is not just hard computational graft that by repeating existing algorithms such as Transformers to a giant scale and training on massive data then success is guaranteed still others emphasize specific cognitive functions or systems level computational processes the key to intelligence might be the ability to build Rich models of physical and psychological processes plan over lengthy time Horizons grasp new Concepts rapidly or Reason with abstractions Summerfield cites three lines of argument against the temptation to constrain AI systems with principles from biology firstly building an AI is a general problem so it needs a general solution every constraint that AI designer relaxes makes the system more attuned to a broad class of problems and so the most successful AI systems are those which leverage computation this is exemplified by the success of deepminds alphago zero Alpha zero and mu zero which is a deliberate attempt to seek to minimize the extent to which AI achievements are kick-started by built-in human understanding and just a little note from me ironically now we're going in the opposite direction with chat GPT the importance of human fine-tuning and knowledge is kind of swinging back but anyway the second line of argument is that humans might not be intelligent in the first place it's argued that the goal of AI cannot just be to copy humans as this will lead to systems that display the same vices and limitations therefore AI researchers should strive to build something that can reason more powerfully and display more brilliant inventiveness so I wonder whether you would say chat gbt is more powerful and more inventive maybe it's truncated by the fine tuning that we've applied to it uh the third line of argument is that if AI researchers do draw upon intuitions from Psychology and Neuroscience When developing AI can they be sure that any knowledge built in is actually useful current Neuroscience might not be quite up to the job of explaining how brains actually work and the elaborate theories that are built about neural computation could just be wider the Mark or just plain wrong a clear example of what Sutton warned against Summerfield said that we've already seen that systems neuroscientists have mostly avoided searching for General theories preferring instead to taxonomize computational processes in a way which abstracts over the wet and messy detail of human function this suspicion was not helped by Elon Musk the founder of openai who when recruiting for researchers to work on his brain computer image project neurolink he made it clear that quote no prior experience in Neuroscience is necessary we'll teach you everything you need to know remarkable isn't it the conclusion is that the most effective way to build AI is to reach for the computational sky and build ever larger models on ever faster computers and this is the path that deepmind had followed with Alpha zero go and Alpha zero and mu zero and it's been incredibly successful Moore's Law the principle that the power of computation should double every year is still alive and kicking and this means that the resources available to researchers continues to grow and grow in the end the case for General methods which leverage computation according to Chris is the most effective this is the lesson that's been learned from 70 years of AI research and it's the path that AI researchers should continue to follow if they want to make genuine progress anyway that was an excerpt from chapter two of Chris's book just to remind you again it's an amazing book it honestly it's an amazing book please buy it the link is in the video description and I had a great deal of fun driving up to Oxford to chat with Chris a couple of weeks ago and if you're watching on YouTube with I also had a cameraman so we've got lots of cool camera angles and walk walkie-talkie type shots so um anyway without any further delay I give you Professor Christopher Summerfield hi I'm Chris Summerfield um I'm a professor of cognitive neuroscience at the University of Oxford I'm also a fellow at wadham College which is where we are now and I also am a research scientist at deepmind in London what inspired you to be a professor of cognitive neuroscience uh well I um I'm a psychologist by training so I've studied for many years uh how memory and attention and control and language work in uh in the human brain if you want to study the human brain then there's sort of three approaches that you can take so one is you can study human behavior the other is that you can try and measure brain activity and the last is that you can try and build mathematical models of how it all fits together and cognitive Neuroscience is a field which kind of combines all of those three elements and you that's that's where my work has ended up I really really enjoy putting all that that information together wonderful I spoke to Murray Shanahan recently and and um he really believes that an intersectional approach is is useful um so he's bringing in a lot of ideas from philosophy and of course you know cognitive robotics as well as Ai and machine learning so I I think it's really interesting having that diversity but let's launch straight into your book I mean just anecdotally as well I want to let you know that it is the best book on AI I've ever read I've I've been glued to it for the last four days I haven't been able to tear myself away from it and just making notes every two seconds and it's a really wonderful book so thank you so much for writing it it's very kind thank you so um let's talk about plausibility versus understanding so large language models have been criticized as not representing a true understanding of the world that we live in and they're said to produce plausible outfits given the prompt now what does it mean to understand something yeah so you probably want a philosopher to answer this question so actually the nature of understanding has been given much more consideration in philosophy than it has in cognitive science I always think it's really surprising that in Psychology and Neuroscience we have whole fields that are dedicated to understanding really difficult Concepts like Consciousness we don't actually really have a subfield within psychology that is the psychology of understanding what does it mean but if we look at the sort of the intersection between philosophy and cognitive science understanding is usually linked to the notion of having a a mental model which can be used for complex inference so the idea is that we biological agents or artificial agents as we go about the world we form a model of how the world Works usually a causal model an understanding of how variables causally interact and the understanding arises when we're able to leverage that model typically to make new predictions or to understand complex phenomena so in the context of large language models what do people mean well I think what they mean is that you know if you ask a model like chat gbt to give you information factual information about a topic in biology it will you know usually produce something which approximates what you find on Wikipedia but it doesn't typically in most cases have the ability to leverage its understanding to draw new inferences right so it can't um for the most part it can't explicitly reason over the information that it has in memory now there are tricks that can help language models do that so you're probably aware of Chain of Thought prompting which you know kind of encourages language models to to do something that looks a bit more like their you know kind of reasoning in a way which would reflect understanding but I think you know our understand our understanding of how to do this is very much in its infancy at the moment interesting yeah and because people have spoken about whether language models have um tensionality Consciousness all sorts of esoteric Concepts but they do appear to have emergent reasoning and with knowledge of course you know there's the epistemic version which is a Justified true belief but there's also the ontological version and I'm interested did in this notion of whether there can be a platonic one true understanding of the world we live in law if you're at a lower resolution there might be ambiguous and inconsistent understandings almost like the Blind Men and the Elephant yeah well this is a question which is really close to my heart I mean I think that in computer science today um you know probably because computer scientists are all trained in they're formally trained in maths and statistics and physics and and of course in Computing itself or in coding languages there is a real desire to see language as a codification of the reality of the world and you know this is a tradition in linguistics which goes back to Chomsky you know kind of the role of language is to disclose the fundamental underlying reality that we all experience and to share it with each other right but there's another view of language which is that language is a much more social process by which we construct shared meaning between conversation partners and that shared meaning need not be unified in any you know so Global sense it doesn't have to point to a single unified reality it can actually be quite decentralized quite local and you know I think that's what you're referring to and the challenge is there you know if we have these plurality of meanings for different concepts but in different topics then whose reality should language models Express and I think this is an unsolved question it's really a philosopher it's a question for people working in ethics and safety as much as people working actually at the front end of you know language modeling yeah and it gets to the heart of what soul was saying about this kind of dichotomy between ontology and and epistemic knowledge we spoke with um Chomsky and um you know he even he said as a rationalist nativist that there's still a large Divergence between the kind of epistemic reality encoded in our language and the ontological reality in the real world I think he thinks that um there's a huge gap between the universe as it is and and our understanding of it they calls it the ghost of the machine yeah well I mean you know I'm not a I'm not a linguist um or a philosopher but um I I think I recognize the enormous challenge particularly when we think about important topics like alignment and especially for language models the enormous challenge of thinking about to whom we align our language models and you know of course if there is a single underlying reality then we just have to align our language models with that and then we're done right but I don't think it's quite so simple and I think it's a really incumbent particularly given um the you know the demographic of machine learning researchers who you know tend to be tend to come from more privileged backgrounds tend to be Western actually you know kind of in they tend to be men they tend to be white um I think it's really incumbent on us to to to be careful not to to think that there is one true reality which is the things that that we learned in school and that that's what we should put into our language models really really interesting and we'll get to the intelligence stuff in a minute because you spoke about how we designed the test to be a reflection of ourselves um okay let's let's go on to creativity because this is something that really interests me now um after Lisa doll's match uh he expressed this sentiment and you wrote about this in in your book he said that he thought alphago was based on probability calculations and it was merely a machine but when he saw this move he changed his mind alphago was creative the move was really creative and beautiful he said it really made him think about it in a new light now what is creativity and does it does it need what we call agency what will creativity is probably in the eyes of the beholder um in that case I think what Lisa doll was referring to was the fact that you know kind of the the policy the the actions that um Alpha go in that setting was able to produce were not only you know they were out of not not only out of distribution for a normal go player but they're out of distribution for an expert like himself right and of course it is a property of um generativity that you know what what we assess as good generativity is that we're able to generate things that are new but which are you know kind of also sort of plausible and interesting and reflective of you know kind of they they don't violate basic preconceptions about how the world works but yet they do something that no one has ever done before and that's true for human creativity right when we think about uh you know kind of in the Arts or you know if you if you are composing a new piece of music there are certain you know there are there are guard rails within which you you compose your music you don't just make random sounds but at the same time you also don't just copy what other people have done and so is agency necessary for creativity well I mean you know I think that what we've seen recently with the large um the success of large generative models implies that it's not so for the most part you know kind of obviously those models may be fine-tuned with reinforcement learning but for the most part they're just trained on you know kind of large volumes of data which are then you know they are produced they're generated in a predictive fashion and sometimes the outcomes are really they appear to me at least to be pretty creative so interesting um one one element of it and we'll talk about the abstraction topology space later but it's a bit like with free will it wouldn't really be free will if it was a random number generator so it needs to fall on on the manifold of of let's say the abstractions which you know which can occur which are grounded to the world that we live in but even then gpt3 it's a bit like a chicken and an egg problem so is it creative on its own or is it creative in tandem with human prompting well I mean I I think that that question is a question of taste right I mean you know if you think of human creativity you know there's nothing new Under the Sun right so if I generate if I make a beautiful new painting and everyone admires it well to what extent am I simply an inheritor of artistic Traditions or ideas that I have I have seen I think you know exactly the same framework is how we should think about language models right so they are trained on they're trained they're pre-trained on large volumes of data they're often then fine-tuned on much more specific data or perhaps with reinforcement learning or an equivalent method and so of course what they are capable of doing is a product of what they have experienced just like it is for us okay and and in your book you sketched out various different forms of AI that are truncated in different ways because there's this kind of um it's a bit like good heart's law it's a bit like um what Sarah hooker calls the hardware Lottery which is this kind of determinism of the potential of a system based on the hardware and kind of philosophical design decisions that you make you gave the example in classical psychology where human abstractions are are used to build the models um a similar form of constraining is is things like inductive priors or whether we buy some thinking or or knowing and also whether we train on human data and the extent to which it's aligned and human compatible and so on so there are so many trade-offs on on that Spectrum um give me some thought that's a very broad question I mean I think you know maybe what you're alluding to is a point that I I tried to make throughout the book which is that um when we're thinking about how we should build AI or indeed you know how we should model um human uh brain processes which is my you know primary expertise then you know we really need to think about what the nature of the problem is what the nature of the world is so you know kind of in cognitive science and in Neuroscience people have long understood that in order to understand you know kind of how neurons are functioning or brain areas interact or why people produce certain Behavior behave in certain ways or produce you know certain patterns of outputs we need to start by thinking about what computational problem the brain is trying to solve right so what is the world like such that it requires this Behavior or this neural activity and I think you know kind of it's often overlooked in AI research that exactly the same framework can be really useful right we need to begin by thinking about what the system is actually there for in the first place and the constraints so you talked about you know kind of inductive biases or you know kind of whether we train with human data or with self-play whether we try to sort of explicitly model cognition or just you know let it emerge the answers to What's going to work best is going to depend on the nature of the problem and machine learning as you will know better than me is a very broad Church you know kind of some of our applications are quite narrow and targeted right if I want to build a self-driving car then I need it to do certain things you know I don't need it to be able to complete the crossword for me and I think you know often in machine learning we talk about Solutions without actually talking about what the problem is and you know in the book I refer to this this thing which is you know I think I call it the meta problem which is like you know kind of what is the thing that we ultimately actually want the AI to do and I I'm sure many people do have their own answers to that question but I don't think we have a consensus and I think even you know kind of within the major research companies there is you know kind of there isn't one sort of single answer to that question yeah what what it what interests me about chat GPT is that it's the first instance of an AI which has reached the masses it's in the public Consciousness and it's the first real example of a meta AI I mean it's still constrained in the modality and the interface and so on although it's a very human aligned interface it's very interesting and you can do almost anything with it swiss cheese problem permitting maybe we can bring that in but um so so it is it is a meta problem uh solver well I don't know if it solves The Meta problem I think the the problem well first of all let's let's just make sure we're clear about what we're talking about so um it is definitely true that chat GPT is the first instance of a language model which has been released into the wild and is being used by thousands probably Millions now of people um which is very impressive that's not to say that it's the first machine learning technology which impacts our lives right so our lives are subtly and imperceptibly influenced by Machine learning at every moment right every time you look at your phone or go on Facebook or um you know kind of even or make consumer decisions or whatever these there is um these are influenced by AI just perhaps in ways which are not immediately so transparent so charity beauty is special because it it what it does is it it brings into it brings the experience of interacting with an agent that is non-human in natural language it brings that experience to the general population in a way which hasn't happened before but I think the the mere existence of successful language models doesn't really answer the question of what they're for right so let me give you some examples so chat GPT is a is a prompt and response model right so what that means is that you ask it a question it gives you an answer so it doesn't do what you and I might do what we are doing in this conversation which is to you know kind of to to share information two ways right so I'm learning from your questions as hopefully you're learning from my answers it doesn't do what people do in normal conversation which is that they reciprocate right so chap GPD never asks you anything back the uses of conversation it's very good if you want to ask chat GPD for information it's quite happy to provide it but it doesn't do the sorts of things that we naturally do in conversation um when you know kind of we try to establish um what's the best course of action to take or we seek Mutual reassurance or we try to clarify our own thinking on topics that don't have a right and the wrong answer can't do any of these things most of what we use natural language for are actually exactly those things if you think of the conversations that you have with friends or Partners most of them don't consist of you going and asking for a detailed explanation of what happened in the War of the Roses but that's what people use to actually be or asking them to compose a you know a funny poem in the style of x but that's mostly what people use to actually Beauty for so I would slightly push back on that in the sense that I mean first of all from an intelligence point of view and I agree with Chalet that it's task acquisition efficiency and with um in context learning you can apparently make it do almost anything so I do see it as a bit of a blank slate and and also you can even prompt it to do some of the things that you were just speaking about in terms of getting advice about things or having a conversation and um and also we'll talk about Peter's Triad and you know the um the the most important element of the Triad which is the third one the the abstract symbolic component which is embedded in our language so it's it's the first example of an AI which apparently um can recognize abstract Concepts like Christmas well so I mean clearly you're right that in context learning is really powerful right and the I think the the big the big revelation of the last few years has been that um in context learning with Transformers is so and and at scale is sufficient to allow um emergent meta learning right so that's really what that's you know we we previously have been able to to show that you know by carefully crafting a meta training curriculum or a meta training sequence um models can learn to perform new tasks that fall broadly within this distribution the surprise is that just by training on large sizes of the internet you can get something that approximates that of course you know touch EBT is not just a generative model it's trained with reinforcement learning from Human feedback right so it's changed from Human preferences as well so in that sense you know you're right that we don't know what the limits of this approach are and maybe the limits are you know maybe there are many many things that you can do with this but you know kind of once again that's a statement about the solution not about the problem right my my point is that we need to think more about what the problem is about what we want AI to do and you know that question I think that that question is coming into focus more but it is inextricably entangled with questions of you know kind of um the ethics and safety of AI systems of the the interface between the social sciences and you know computer science and technology right you know what what role should these models have in our society what role do we want them to play and what do we want to use them for and I one of the points that I try and make in the book is that we need better answers to those questions yes and I'm speaking with Luciano for really later and he's got an um a wonderful philosophy of information where I mean similar to Heidegger actually kind of sees it as um a kind of layer between us and ontological reality and and that kind of um derangement goes in both directions it kind of it changes us at a very visceral level um but anyway I wanted to talk about whether Neuroscience can impart useful knowledge to Ai and we can broaden out to things like Linguistics and psychology a lot of folks deride Linguistics there was that famous quote about every time I fire a linguist the NLP accuracy goes up and and you wrote a quote actually from Elon Musk good old Elon Musk no prior experience in Neuroscience is necessary we will teach you everything you need to know now that is is that just arrogance well I think there is an element of hubris in contemporary machine learning and AI research I think there is a sense that you know kind of the the structured theories which exist in linguistics and in cognitive science and in education research and in in the social sciences in economics that these are sort of you know kind of these are these are play things and that actually we can sort of do without that knowledge and you know we can just use large-scale function approximation to solve problems um on the one hand you know kind of there are instances where that is almost certainly true right you know it's certainly true that um the models that we have built especially in the social sciences don't have very good predictive validity so economists aren't all that good at predicting what you know kind of prevention financial markets will actually do behavioral scientists aren't actually all that good at predicting how humans will behave um and it's true that we can use machine learning to make more accurate predictions and I think that's kind of what you know that that that has driven a a kind of belief in some corners of computer science that past theorizing in these fields is sort of not all that important but you know I think that sort of overlooks the fact that when we build models you know we build models for multiple reasons especially we build models both for prediction not not only for prediction but also for explanation and that the explanatory role of models in understanding a problem is you know kind of it's it's a really huge part of what is contributed by all of you know fields in the social and life sciences and I think you know that knowledge can be incredibly useful and it can be incredibly useful especially when it comes to something like deployment right so you know let me give you an example if you wanted to use an AI system to for example a language model to teach someone how to do something then you might say oh okay we can just sort of you know do without you know maybe we can just learn an optimal curriculum from first principles using machine learning but then you know if human students learn in sort of idiosyncratic ways or you know maybe there are aspects of their learning which are you know kind of haven't even entered the machine learning researchers Consciousness like you know maybe you need to motivate them appropriately or you know kind of maybe you need to account for their aspirations or you know kind of their cultural context then if you don't know these things then you're probably not going to succeed interesting could you bring Rich Sutton into this and maybe we can also talk about the neats and the scruffies and um so that there's there's this notion from Rich Sutton that the world is a very complex place and and you said in your book that you know a lot of the the research in Neuroscience is a bit like the blind man and the Elephant they want to have a model with explanatory power so um the Temptation was just to kind of work on on something in isolation and and ignore the messy reality and Rich Sutton said of course just just stop Building Things by hand we need to embrace the complexity lots of compute and so on so um and and that leads us on to the neats and the scruffies I think you know somewhat analogously so um are you a nature as graphic look at my jumper could you introduce it um yeah I mean I think I'm probably a scruffy um both you know sartorially and intellectually um yeah I mean I you know obviously there is no there is no exact mapping from the the original the notion of needs and scruffy's dates back to the 60s and 70s and originally described you know kind of symbolic AI researchers who were primarily focused on on you know building logic systems versus building you know kind of language systems or their equivalents right you know did you need to sort of solve a real world problem and just Tinker till you got it right or did you need to start from you know kind of let's reduce thinking down to three logical principles and build everything back up from there right and those were the the scruffies and the meats respectively and today you know what do we have well you know kind of I think today what we have is um it's it's not it's not an exact mapping but we have people who you know kind of want to eliminate the the messy reality of the human world from machine learning research you know by saying okay you know let's let's try to have as little human um as little knowledge of human uh brain function in rml systems let's try and have as little human data in our in our ml systems as possible and those people are maybe contrasted with um you know another group who say well you know actually we just gotta you know we really just got to learn from you know and from Human data and you know the language modeling that we're seeing you know is obviously more firmly in the latter camp Ridge Sutton I mean maybe I could just say you know I think in Rich Sutton in about three years ago he wrote this incredibly famous blog post called The Bitter lesson which he articulated the point of view that you you just summarized so he said you know we should forget about um you know kind of Knowledge from psychology or Linguistics and or cognitive science and we should just build big powerful systems with lots of compute and you know I think you know he was both right and wrong I think the success of large models with relatively undifferentiated um structure architecture suggests that you know maybe he was broadly right about the fact that we just need to build big models but I think what he was wrong about was the role of human data and I think that what you know you reference to gbt so chat gbt is of course I mean of course it's you know it's trained on the whole of the internet um approximately but it's fine-tuned with lots and lots of um performance feedback from uh human raters right for human evaluators so it's success hinges very much upon human preferences and I think that was unforeseen in that blog post um and I think you know it will be defining for AI research going forward that if we want to build systems that people actually want to use and find useful then we need to ground their performance ground their training in human learning human outcomes human beliefs human preferences because otherwise they're just not going to be all that useful okay but um folks talk about you know super intelligence intelligence explosion the really interesting thing about Alpha zero was its superhuman performance and I don't just mean you know one human I mean Humanity it was just on another level and when we constrain with human data my intuition is it's a good thing and a bad thing my intuition is that it will no longer transgress human capability but it will be more human aligned more human compatible yeah I'm I'm not sure I entirely agree with that right so obviously you're right that you know I'll forget what was exciting about alphago was that precisely that it um it was able to to beat every human on the planet at a game which humans had devised and spent some human spend a lot of time trying to perfect their play on um but of course you know go go is not like the world and the world is not like go right so go is a it's a deterministic game it's a game of finite information it's a zero-sum game the world does not have those properties for the most part go is Guided by a set of like um of of um of rules which are you're fully deterministic the world does not revolve around deterministic rules and what that means is that for the most part the world doesn't exhibit the systematicity that logical or mathematical systems exhibit at least not at the level of that we understand it undoubtedly it's systematic at the level of particles or Quantum phenomena it obeys the laws of physics but not at the level that we understand it and so what that means is that systems which you know use self-play to try to come up with you know policies which are out of the human distribution maybe they're useful for a a relatively small set of problems in which you can in which you can systematize the world in which human human preferences really don't come into it um but I think for the majority of applications in which we actually would find AI to be useful being out of human distribution at either end is going to be worse than useless so it's no good you know if you're a if you're a teacher um in a primary school and you know a student asks you you know what's the answer to you know why is the sky is blue why is the sky blue and you give an answer which you know kind of is a degree level physics you know kind of response then you haven't learned in it the student hasn't learned anything that's not what's required so we need our systems to be to understand what humans want and to be tailored to those beliefs and preferences yeah I love the point discussing the degree to which go and the real world may be analogous and I think when you when you spoke of systematicity I think you mean it in a kind of photo volition kind of kind of way um and indeed the world is not deterministic and we can get onto the probabilistic stuff later but um Sutton and hutter do believe that the world can be represented with a markup decision process and of course you have an agent you have affordances you have action and so on so so that in some sense they they do believe that their approaches could work in the real world yeah that's that's the hypothesis that they call the the reward is enough hypothesis yeah yeah so um I was going to go there later but we're there now yeah why don't you introduce it sure well I mean I guess you know a starting point for this debate is is the mdp itself right so mdp micro decision process I mean incredibly useful tool for you know machine learning for control and laterally you know reinforcement learning um and of course it's the very framework we were just talking about alphago it's the very framework that allowed you know kind of go to be solved amongst many other exciting problems of a similar nature but I think your question is you know can we model the world with that framework and you know undoubtedly you know from a purely theoretical standpoint the answer is is yes we could but would it actually be useful to do so well let's think about what happens what do you actually do if you are a machine learning researcher in RL you know let's say you want your agent to solve a video game of some sorts right a standard application well so you first of all have to specify an objective for the agent so well that's easy if you're in a video game because you can say okay well each video game has a goal which is to maximize your score without you know losing lives right great so we can write that down as the reward that's the r in the markup decision process um and we can we can train our agent to maximize that but what what's the equivalent in the real world so in the real world you know kind of when you take actions we might consider them to be rewarding or not as the case may be but there isn't a sort of magical you know process in the sky that kind of administers rewards to us on the basis of our actions it's not like oh I ate an apple and I got 10 points for that fact in the real world observations and rewards are the same thing so rewards are observations and observations are Rewards and what this means is that you know kind of the the mdp um process for modeling or the mdp approach for modeling the real world is fundamentally at odds with actually how biology works so rewards are not extrinsic they're not administered by the machine learning researcher they are intrinsic when we talk about reward you know the hedonic experience of eating an apple that's an experience which is intrinsic it's generated by the agent itself now if you have agents that can generate their own rewards then suddenly you're in a completely different computational landscape because the first thing that could happen if you give the agent full Liberty to generate its own rewards is that it can just say oh just find everything rewarding and then it's instantly maximized its reward so we don't do that presumably because we're under another constraint which is that we need to survive and reproduce but that survival and reproduction constraint is just not there in reinforcement learning we're not optimizing for that we're optimizing for this weird proxy which is reward and so if you think about it just finish if you think about it you know let's try and think what alphago would be in the real world right you know what's you know what's the kind of or Alpha zero right you know what what is what is World zero right what would it look like well what you would need is you'd need a really patient machine learning researcher to come along and annotate every single action that every single person might take with a score and that score would be need to be exactly what you would you know what would be required for you know whatever we deem to be you know intelligent Behavior so who's going to do that not me yeah so we need the the big man in in the sky to um you know ordain the rewards but there's a there's a few kind of areas to explore with the reward hypothesis um from the angle of behaviorism um from the angle of um uh intelligence from the behaviorism angle you had a wonderful story in your book about the monkey and um how it how it managed to get the banana and the behaviorists of course think that there's just a very um simple relationship between stimulus and response and in this experiment the monkey just kind of sat in the thought for a while and had a flash of inspiration uh it was thinking right and um you know naively I I think of reward as being a weird kind of inverse behaviorism you know this this kind of focus on on the um uh the output signal itself that was the the third thing um teleology so um you were speaking in the book a little bit about how Advocates of reinforcement learning they they they see affordances and actions as kind of leading to a to a form of goal-directed behavior and I was kind of um Bridging the Gap between simple gold directed behavior and rewards and teleology and in itself so what is our purpose and I think folks in reinforcement learning see this kind of natural emergence hierarchy so if only we can place one reward at the top then we get this emergent intelligence we get intrinsic motivation we get this kind of hierarchy of sub-rewards and so on but there seems to me to be a kind of fundamental chicken and egg problem there yeah I mean I I'm not sure I entirely followed your question but let me try and pick up on a few things so first of all yeah I think you're absolutely right that you know kind of at the heart of the reward is enough hypothesis is the idea that you know kind of there there exists somewhere a very simple set of principles which we could write down which would Define how rewarding things should or should not be right so you know maybe we could say everything reduces to Information Gain or everything reduces to empowerment or to you know maximization of one one quantity or another there are many different minimization of cognitive dissonance and you know there is a hope that there is a single principle that we can um we can adopt um so yeah um I was looking at the reward hypothesis from the angle of um teleology so you know the the the you know we we have I'm not even sure if you if it's reducible but we have this notion of purpose and uh cyberneticists and people like cell think that it kind of emerges from biology and um so there's this kind of question of where does it come from and what is it and I guess my criticism of reinforcement learning is it it appears as if we are reducing that purpose or that end goal into one property so so there's that um I'm naturally cynical about well not cynical skeptical of behaviorism because I see it as a dominating Paradigm in how we measure intelligence um most of our tests are that of behavior in our metrics and so on and I haven't I don't completely understand this but it seems to me like um reward maximization is a form of a weird form of behaviorism which means that it's almost like we're not looking at the mechanism we're just looking at the action so is it fair to say the action is the other side of the coin of behavior in some way okay let me see if I can understand or unpack your question I'm kind of thinking out loud here a little bit so sure are we actually recording there even on the Sony A1 yeah oh okay well we'll use that though too yeah Okay cool so the the term behaviorism refers to a movement that dominated psychology throughout the early part of the 20th century associated with figures like Watson and Skinner and it was um founded on the idea that to understand the human mind you really have to understand first how humans behave or understand solely how humans behave and also true for Animals as well so the idea was that we can explain how we can explain human motivation human behavior exclusively through the links between stimulation and response the forming forming of associations between stimulus and response and so um behaviorism um its sidesteps questions around how cognition is structured right so it overlooks you know what what are the role of our multiple memory systems it's sort of it doesn't ask questions about how we attend to the world you know attention psychological construct described since the 19th century of it completely overlooks or or you know kind of reduces it to a very simple notion of salience um it doesn't really talk about language or it doesn't really give explanations for how we learn language or at least not plausible explanations because everything is a chain of associations then you know it assumes that we can we learn all Language by just forming links between adjacent words which then doesn't explain you mentioned Chomsky earlier doesn't explain the the infinite generativity of language so that's what behaviorism is and reinforcement learning as a movement within machine learning grew out of um the study the behaviorist study of animal learning um so the original um the original formulation for example of TD learning um by Saturn nambato um in the in the 80s um was it was originally formulated to try to understand the stimulus response behavior of experimental animals um so you know kind of the its adequacy as a theory of intelligence has been under debate in Psychology and in machine learning for you know the best part of a hundred years right and I think your critique is that um reinforcement learning or or you know the the overall behaviorist approach assumes that there is some specific set of behaviors which are optimal in the world given a reward function which is externally specified which in the lab is like you know does the experimental animal get you know peanuts for or juice for making this choice or that choice and in the world might be you know do we receive reward for you know kind of saying this thing or doing this thing and you know I share your skepticism about that approach and I think that you know it doesn't mean that reinforcement learning as a tool is not an absolutely fundamental part of how biological agents learn and it doesn't mean that it shouldn't be a fundamental part of how artificial systems learn but it's not all there is to the story and in particular you know we need answers to the question of how intelligent systems generate their own purpose right so I think you know there's a very nice linking back to alignment again it's a really nice way of thinking about the distinction between narrow and in general intelligence which I cite in the book which is you know we can think of a narrow intelligence as a system which is given task and executes it very well and we can think of a general intelligence as one that dreams up the task in the first place and so you know kind of that sense of purpose like what am I supposed to be doing and why that's the the really hard Challenge and as we were discussing earlier it's really remains a largely unsolved challenge remains an unsolved challenge partly because humans haven't worked out what they want AI systems for but it's also because you know even if you take our most powerful you know kind of large generative models or our best RL systems They Don't Really generate their own they don't they don't generate their own goals like humans do they don't set themselves objective objectives and then seek to satisfy them what they do is they they do what you know what they're trained to do right so um a lot of this gets back to how the lens kind of truncates our frame of reference I mean even in this discussion with behaviorism it's similar to this concept of a white box and a black box or behavior and mechanism and um uh when I was speaking with Chalmers who he would talk about function and Dynamics and the the subjective experience you know so there's all of these different kind of um uh filters that we use that that exclude parts of the truth a bit like the blind man and the Elephant but um similarly good heart's law so um it's a well-known problem in in machine learning and it's related to what you said a fundamental indeterminacy principle when the measure becomes a Target it ceases to be a good measure so um you said even as a field if we were to agree on a definitive set of tests for General AI researchers would immediately Focus their energies on solving those specific problems so another example of this cone of attention rather than building agents whose generality extends further to open-ended real world settings where we hope AGI May 1B May one day be deployed so this gets us into the whole benchmarks thing what are your thoughts about that yeah so I I guess that one way to tackle this question is to think about what what is the role of optimization in biological systems you know are we sort of gradually optimized towards some objective is that a good description of our learning and in some ways maybe it is right so you know clearly over the course of development we start off fairly random and our Behavior becomes fairly structured over the course of development both you know kind of development and learning right so you know even in adulthood we can learn new things hopefully um so it does seem like that might be a plausible model and of course that's been picked up recently by neuroscientists who view tools from machine learning like convolutional neural networks to try and understand you know kind of object recognition in humans for example on the one hand there is a there's a great deal of excitement around the idea that we can understand human Learning and Development through the lens of optimization but on the other hand um we we I think the Paradigm in which that we use the Paradigm that we use for machine learning is fundamentally unsuited to capturing the nature of the problem that biological agents face in the natural world and that is because you know if you think about it the the basic premise in which you know deep learning is founded is that you know if you have a large and diverse training set and you sample IID from that training set then ultimately your system is going to be able to encode a distribution which will allow it to you know interpolate or extrapolate a small small extrapolation um to new previously unencountered test examples right but in the natural world you know we don't live forever and there is no way given the um the way in which the natural world is is structured that we live in a a Locale which is very different from other places you know we meet people who are very different from other people we study things that are very different from other topics that we might encounter we only get a tiny tiny slice of the distribution that is natural life and so using the principle that we we just sort of you know kind of get as much training as possible and as much diversity and experience as possible and use that to generalize that's just it's just not feasible when you only live for four score years and ten if you're lucky and so humans I think have learned or have evolved very different ways of learning that are not about sort of gradient descent towards one you know sort of perfect version of themselves but rather what we do is we're constantly adapting to our local circumstances we're constantly we're constantly forming new models of the world not not just building one model of the world we're constantly forming and reforming our models of the world using composition which perhaps we can talk about in a minute yes I'd love to get on to maybe we should do intelligence then we'll get into conversationality but um yeah so you delivered a wonderful anecdote in your book about Von Neumann and there's wit and I could tell that you too Chris are off the charts on on the intelligence scale just based on the high resolution and and sometimes esoteric language that that you used um but they shouldn't necessarily mean that that you're intelligent because a book is crystallized I have an intuition that there's something about the kind of information conversion efficiency that that tells us about intelligence but um I must say actually I was reaching for the dictionary quite often and um I couldn't decide whether whether the word desiderata is gratuitous or delicious I do love them I do love that word but I know Orwell would um have something to say about having read his essay politics and the English language but anyway um you said the question of whether there is one intelligence uh or many has been the subject of much research and debate among cognitive scientists you told the story of spearman's G factor and the more heterogeneous version from uh Gardner who like Minsky and how you were just alluding to thinks there's a society of of mind um so um I'm actually interested in this idea of Elizabeth spelkey's work at Harvard she talks about core cognitive priors like you know a agentiveness and spatial temporal reasoning and and so on and and maybe within those those prior there's there's some degree of generality um but um you also said that the intelligence tests were designed to make westerners look the smartest of them all and of course there's the Bostrom Fiasco recently I mean Could you um talk a little bit about that yeah I mean so intelligence testing has a very checkered history as I'm sure you know um you know the earliest intelligence tests were went you know sort of hand in hand with attempts to exclude marginalized groups from various roles in society um and you know kind of whilst today the study or the measurement of the human mind is a sort of a slightly more reputable Endeavor it's still I think you know really fraught with political challenges not least because you know ultimately if you wish to to specify if you wish to build an intelligence test just like you know we were discussing in the context of AI what is AI for what does it mean for an AI to be intelligence intelligent if you want to build an intelligence test for humans you need to start with some preconception about what that intelligence entails so in other words there's a circularity in the building of intelligence tests in the you know the way we build them presupposes what we think intelligence is but yet the test itself is purporting to measure that very intelligence so what in what are we supposed to ground our Notions of human intelligence and this is where you know kind of I think that the opportunity for bias and discrimination have come in in the sense that you know Western [Music] um uh Western educated um probably quite privileged researchers in you know academic institutions have taken it upon themselves to Define what intelligence is and by definition by the way they have defined it that means that groups that are culturally or socially different from them are much like much less likely to meet up to the standard that they have set and you can see this and you know it's still a standard feature of many intelligence tests to include um you know vocabulary um as a measure of intelligence of course you know how big your vocabulary is has to do with you know where you went to school and you know how much opportunity you've had to read lots of books and you know taught the meaning of complex Concepts which is completely different from the intelligence that you need to navigate in the Sahara Desert or you know make medicines from local plants or even you know to organize your Society in a you know where whether you're in you know kind of the Kalahari Desert or or here in Oxford wonderful now um we're a little bit short on time so let's talk about thinking versus knowing and um we can we can sketch out the Divergence and AI research actually with Minsky and rosenblatts they both went to the to the Bronx um Academy didn't they in the 1940s and then they went off on very different trajectories one thinking one knowing and you can bring that in but I wanted to talk about compositionality because when I speak with go5 folks and and I love go five folks so good friends of many of them and um the I think the most persuasive argument they have is are these kind of um volition style systematicity compositionality type arguments so um you spoke about this on page 101 and uh so on the thinking side you said um you know symbolic approaches which allow for Limitless productivity from limited mental Primitives these complex Concepts can be composed from simpler building blocks using a set of formal rules it's a powerful principle that can endow computation with great versatility flexibility expressiveness and of course it led to ideas like the language of thought and indeed chomsky's innate mental grammar so on the thinking side of the of the balance that's very powerful sure yeah absolutely so obviously this debate you know has 70 years of History um you know and and we won't be able to do justice to it all today I mean I you know I think um many Advocates of more sort of symbolic style architectures um focus on the insufficiencies of current deep learning systems and you know they're absolutely right those systems are limited in many many ways um however um you know kind of that just because they're limited doesn't mean that the solution is to try to build in the sorts of reasoning structures that you know kind of we imagine underpin formal logic or maths or you know kind of the the formal languages that we use to to reason about the world and I think that the success or otherwise of approaches that do that fundamentally depends upon the match between the systematicity of the problem you're trying to solve and the system itself so if the problem you're trying to solve is a systematic problem you know like go or like theorem proving then you know of course you can use you know highly structured approaches and of course you know of course you know it turned out that combining deep learning with research was more effective for go than symbolic approaches but of course symbolic approaches are very very successful for zero-sum games before you know kind of the 21st Century but language does have clear systematicity I mean I I know Bob my roommate Bob has a friend Jane who once went to school with someone you know it's it all resolves to that entity Bob well so you know some people clearly you know I'm not a linguist like I said earlier and clearly you know Linguistics much of linguistics computational Linguistics you know is based around the attempt to try and systematize language but you know kind of maybe it's fair to say that those efforts have yet to achieve you know full success and you know can of course Fields like pragmatics and theories like relevance Theory you know I love Dan sperber's relevance Theory right the idea that you know actually everything that we say is really you know geared towards mentally modeling the other and trying to couch our meaning in terms of their beliefs and preferences this is not something which is amenable to the sort of you know kind of the systematization that we might give to a formal language I have to say a real shout out to a book which I absolutely loved which I read recently um which is by Nick chater and Morton Kringle Wilson wrinklesson in which which is called The Language game in which they argue that you know kind of really we should think of language not as a system of Truth Discovery but as one of meaning construction and they they liken linguistic interaction to a game of Charades you know when we start to play charades you know we we will establish sort of signs which we which become mutually intelligible by virtue of the fact you know but they can be very arbitrary but by virtue of the fact that they come randomly to Signal things and that's how language works so they argue to um to that private language game but um yeah exactly like bigenstein hey folks I really hope you enjoyed that conversation today with Professor Christopher Summerfield if you like the work we're doing here on mlst please consider subscribing on YouTube or rating us on your podcast listening app and um yeah just thanks a lot for your support and we'll see you on the next one

Channel: Machine Learning Street Talk

Views: 19,444

Keywords:

Id: 31VRbxAl3t0

Length: 88min 55sec (5335 seconds)

Published: Tue Feb 21 2023