Sean Carroll on AGI: Human vs Artificial Intelligence | Lex Fridman Podcast Clips

yeah you said that we're very far away from AGI I want to eliminate the phrase AGI so uh basically when you're analyzing large language models and seeing how far they from whatever AGI is and we can talk about different right Notions of intelligence that we we're not as close as kind of uh some people in public view are talking about so what's your intuition behind that my intuition is basically that artificial intelligence is different than human intelligence and so the mistake that is being made by focusing on AGI among those who do is a an artificial agent that as we can make them now or in the near future might be way better than human beings at some things way worse than human beings at other things and rather than trying to ask how close is it to being a humanlike intelligent we should appreciate it for what its capabilities are and that will both be more accurate and help us put it to work and protect us from the dangers better rather than always anthropomorphizing it I think the underlying idea there under the definition of AGI is that the capabilities are extremely impressive that's not a precise statement but mean no I get that completely agree and then the underlying question where the a lot of the debate is is how impressive is it what are the limits of large language models can they really do things like Common Sense reasoning how much do they really understand about the world or they just fancy mimicry machines MH and uh so where do you fall on that as to the limits of large language models I don't think that there are many limits in principle I I'm not I'm a physicalist about Consciousness and awareness and things like that I see no obstacle to in principle building an artificial machine that is indistinguishable in thought and cognition from a human being but we're not trying to do that right what a large language model is trying to do is to predict text that's what it does and it is leveraging the fact that we human beings for very good evolutionary biology reasons attribute intentionality and intelligence and agency to things that act like human beings as I was driving here to get to this podcast space I was using Google Maps and Google Maps was talking to me but I wanted to stop to get a cup of coffee so I didn't do what Google Maps told me to do I went around a block that it didn't like and so it it gets annoyed right it it says like no why are you doing it doesn't say exactly in this but you know what I mean it's like no turn left turn left and you turn right it is impossible as a human being not to feel a little bit sad that Google Maps is getting mad at you it's not it's not even trying to it's not a large language model it's not as no aspirations to intentionality but we attribute that all the time Dan Dennett the philosopher wrote a very influential paper on the intentional stance the fact that it's the most natural thing in the world for we human beings to attribute more intentionality to artificial things things than are really there which is not to say it can't be really there but if you're trying to be rational and clear thinking about this the first step is to recognize our huge bias towards attributing things below the surface to systems that are enable that are able to at the surface level act human so if that huge bias of intentionality is there in the data in the human data in the vast landscape of human data that AI model large language models and video models in the future are trained on uh don't you think that that intentionality will emerge as fundamental to the behavior of these systems naturally I well I don't think it will happen naturally I think it could happen again I'm not against the the principle but again the way that large language models came to be and what they're optimized for is wildly different than the way that you beings came to be and what they're optimized for so I think we're missing a chance to be much more clearheaded about what large language models are by judging them against human beings again both in positive ways and negative ways well I I think sort of to push back on what they're optimized for it's different to describe how they're trained versus what they're optimized for so they're trained in this very trivial way of predicting text tokens mhm but you can describe what they're optimized for and what the actual task in hand is is to construct a world model meaning an understanding of the world and that's where it starts getting closer to what humans are kind of doing we just in the case of large language models know how the sausage is made and we don't know how it's made for us humans but they're not optimized for that they're optimized to sound human that's the fine-tuning but the actual training is optimized for understanding uh creating a compressed represent of all the stuff that humans have created on the internet and the hope is that that gives you a deep understanding of the world yeah so that's why I think that there's a set of hugely interesting questions to be asked about the ways in which large language models actually do represent the world because what is clear is that they're very good at acting human the open question in my mind is is the easiest most efficient best way to act human to do the same things that human beings do or are there other ways and I think that's an open question I just heard a talk by Melanie Mitchell at Santa Fe Institute an artificial intelligence researcher and she told two stories um about two different papers one that someone else wrote and one that her group is following up on and they were modeling aell aell the game with a little rectangular board white and black squares so the experiment was the following they fed a neural network the moves that were being made in the most symbolic form like E5 just means that okay you put a token down E5 so it gives a long string it does this for millions of games right real legitimate games and then it asks the question the paper asks the question okay you you've trained it to tell what would be a legitimate next move from not a legitimate next move did it in its brain in its little large language model brain I don't even know if it's technically large language model but it Learning Network did it come up with a representation of the aelo board well how do you know and so they construct a little probe Network that they insert and you ask it what is it doing right at this moment right and the answer is that uh you know the little probe Network can ask you know would this be legitimate or is is this token white or black or whatever um things that in in practice would amount to it's invented the the aelo board and it found that um the probe got the right answer not 100% of the time but more than by chance substantially more than by chance so they said there's some tentative evidence that this neural network has discovered the Aela board just out of data raw data right but then melan's group asked the question okay are you sure that that understanding of the of the Aela board wasn't built into your probe and what they found was like at least half of the Improvement was built into the probe you know not all of it right and look a athow board is way simpler than the world so I that's why I just I just think it's an open question whether or not the I mean it would be remarkable either way to learn that large language models that are good at doing what we trained them to do are good because they've built the same kind of model of the world that we have in our mind or that they're good despite not having that model either one of these is an amazing thing I just don't think the data are clear on which one is true I I think uh I have some sort of intellectual humility about the whole thing because I was humbled by several stages in the machine learning development over the past 20 years and I was just would never have predicted that llms the way they're trained on the scale of data they're trained would be as impressive as they are and there that's where intellectual humility steps in where my intuition would say something like with Melanie where you need to be able to have very sort of concrete Common Sense reasoning symbolic reasoning type things in a system in order for it to be very intelligent but here you're I'm so impressed by what it's capable to do trained on next token prediction essentially that's I I just my conception of of the nature of intelligence is just completely uh not completely but uh humbled I should say look and I think that's perfectly fair I also um was I almost say pleasantly I don't know whe it's pleasantly or unpleasantly but factually surprised by the recent rate of progress clearly some kind of phase transition percolation has happened right and the Improvement has been remarkable absolutely amazing that I have no arguments with I'm that does yet tell me the mechanism by which that Improvement happened constructing a model much like a human being would have is clearly one possible mechanism but part of the intellectual humility is to say maybe there are others I was chatting with the CEO of anthropic darom so behind Claud and that company but a lot of a lot of the AI companies are really focused on expanding the scale of compute sort of if we assume that that AI is not data limited but is compute limited you can make the system much more intelligence by using more compute so let me ask you on the almost on the physics level do you think physics can help expand the scale of compute and maybe the scale of energy required to make that compute happen yeah 100% I think this is like one of the biggest things that physics can help with and it's an obvious kind of lwh hanging fruit situation where uh the heat generation the inefficiency the waste of existing highlevel computers is nowhere near the efficiency of our brains it's hilariously worse and we kind of haven't tried to optimize that hard on that Frontier I mean your laptop heats up when you're sitting on your lap right it doesn't need to your brain doesn't heat up like like that um so clearly there exists in the world of physics the capability of doing these computations with much less waste heat being generated and I look forward to people doing that yeah are you excited for the possibility of nuclear fusion I am cautiously optimistic excited to be too strong I mean it' be great right but if we really tried solar power it would also be great I I think alas discover said this that the future of Humanity on Earth will be just it the entire surface of Earth is covered in solar panels and data centers why would you waste the surface of the Earth with solar panels put them in space sure you can go in space yeah space is bigger than Earth yeah just solar panels everywhere yeah I like it we already have Fusion it's called the sun yeah that's true and and there's probably more and more efficient ways of catching that energy sending it down is the hard part absolutely but um that's an engineering problem yeah so I I just wonder where data centers the compute centers can expand to if that's the future if AI is as effective as it promis as it possibly could be then it's the scale of computation will keep increasing and perhaps it's a race between efficiency and and uh scale there are constraints right you know there's a certain amount of energy certain amount of damage we can do to the environment before it is not worth it anymore so yeah I think that's a new question in fact it's it's kind of frustrating because we get better and better at doing things efficiently but we invent more things we want to do faster than we get good at doing them efficiently so we're continuing to make things worse in various ways I mean that's that's the dance of humanity we we're constantly creating better and better better technologies that are potentially causing a lot more harm and that includes for weapons it includes AI used as weapons that includes nuclear weapons of course which is surprising to me that we haven't destroyed human civilization yet given how many nuclear warheads are out there look I'm with you between nuclear and bioweapons uh it is a little bit surprising that we haven't caused enormous Devastation of course we did drop two atomic bombs on Japan but compared to what could have happened or could happen tomorrow it could be much worse yeah it does seem like there's a underlying speaking of quantum Fields there's like a like a like a field of goodness within the the human heart that like in some kind of game theoretic way we create really powerful things that can destroy each other and there's greed and ego and all this kind of power hungry dictators that are at play here with in all the geopolitical landscape but we somehow always like don't go too far yeah but that's exactly what you would say right before we went too far right before we went too far and that's why we don't see aliens for

Channel: Lex Clips

Views: 94,418

Keywords: ai, ai clips, ai podcast, ai podcast clips, artificial intelligence, artificial intelligence podcast, computer science, consciousness, deep learning, einstein, elon musk, engineering, friedman, joe rogan, lex ai, lex clips, lex fridman, lex fridman podcast, lex friedman, lex mit, lex podcast, machine learning, math, math podcast, mathematics, mit ai, philosophy, physics, physics podcast, science, sean carroll, tech, tech podcast, technology, turing

Id: ThMh0-3JxNg

Length: 14min 41sec (881 seconds)

Published: Sat Apr 27 2024