ChatGPT, AI, and AGI with Stephen Wolfram

foreign [Music] hello everyone and welcome to uh this new uh live recording of uh Beyond conversations in you are already a lot of people evidently attracted by uh Our Guest today and also the excitement around the announcement yesterday uh when Stephen Wolfram and I spoke uh a couple of years ago and we realized that we only speak about every 10 years or so and decided that that frequency is definitely not high enough and I'm glad that we were able to um to keep the promise of speaking more frequently instead uh but uh obviously the new ideas that both Stephen in the meantime has developed and deepened as well as the announcement of the interface between Chad GPT and wall from alpha are the wonderful opportunity to have him here today hi Stephen for joining so uh let's start with um the easy stuff the symbolic approach of uh well from alpha is now married to the statistical approach of gpt4 and chat GPT uh in a wonderful API that at least apparently can be kind of designed the natural language itself and then the system figures out how to interface the various uh data sources described in the Manifest so what um do you expect people to do uh with the the plugin now available and how was the experience of developing the plugin um as one of the first of the companies having announced it well it was a it was a big heroic adventure of software engineering which I think uh is a tribute to the fact that you know I think we've done pretty good software engineering over the decades open AI has done good software engineering it's possible in a couple of months to successfully link these things given that you have a good foundation on both sides um but uh definitely quite an adventure and certainly the the interaction of you know get the AI to do what you want it feels very much like sort of animal wrangling poking the alien intelligence type thing it's very unclear you know does it matter to say please in the in the Pro in the sort of pre-prompt the Manifest you know does it matter whether it's in capital letters nobody knows you know it's kind of like try it and see what happens you know we're really we're really poking at some kind of alien intelligence here and I you know even though I I made kind of a a sort of at least a high level analysis of you know what is chat GPT really doing you know what what is is going on in this kind of uh in this neural net that's understanding that's that's dealing with language being able to generate language um even with the level of I would say not yet very scientific understanding that we have of this I can't tell you anything about which actual words to put into a prompt um and so that that's uh and it's an interesting question I mean what's happened with the link with between Chad gbt and actually two pieces to the to the uh the the plug-in that we've put together um one is the world from alpha interface the other is a wolf and language interface and we had uh at first we'd assume that only the Wolfram Alpha interface is going to work at all decently because the thing we've got in more from alpha you know Wolfman Alpha is sort of the unique example of a system that does broad natural language understanding what does it mean to do natural language understanding for from alpha it means something very definite it's you take natural language in you convert it to our precise computational language and then you do a computation it's not there's no question question you know people say does chat GPD really understand language it's hard to Define what that means for often Alpha it's very clear what it means to understand natural language it means we convert it into computational language from which we can do computation and so we've had this kind of big capture uh you know when we're dealing with with an llm that is fundamentally sort of trading in natural language we've got a way to kind of catch natural language that it makes and turn that into precise computational language that's what Wolfram Alpha does the surprise I suppose to me has been how comparatively successful uh the the llm is at generating wolf and language code and I think wolfen language is sort of the unique example where that's going to be a a really really neat thing to have happen because it's uh you know when when sorry just to be explicit did you need to teach the model to do that or it is one of those latent emergent features that the training brought with it oh it's it's read lots of Orphan language code and it's read a lot of kind of combinations of natural language descriptions together with welcome language uh uh code so it's it's something you know it's hard to know at this point to what extent it is uh you know maybe all those years that I put into designing wolf and language well pay off in the fact that an llm can understand it easily you know that's I think a not a not uh not absurd hypothesis that is you know if you're trying to understand a language that is uh very uniform has is very principled so to speak it's a lot easier to pick that up and encode it in you know a small number of uh of kind of pieces of a of a neural on that rather than something that's full of exceptions um that that you know is more difficult to deduce from training examples and so on we don't know yet I mean that's a an alternative approach uh to what uh open AI uh embraced and and you uh took advantage of together with the others that that launched uh plugins yesterday was to use uh the open source Lang chain wrapper um do you have any opinion of um the advantage or the disadvantage of using that kind of approach I I only know that the line chain folks seem to have done some nice stuff with wolfen Alpha beyond that I don't really know I I I'm not I'm the wrong person to ask we've got had a fairly big team doing the software engineering of this I have not been one of those people so uh I'm I'm uh usually I know the micro details of everything we're doing but in this particular case I know I know how our uh plug-in API mechanism works I I don't know about um uh other other Alternatives or they although I I hear that Lang Chain's done a bunch of nice stuff with wolf malpha and um uh I'm I'm glad to know that there are other Pathways because we're still at very early stages here and uh it's it's really not clear I mean you know the the thing that really you know we were going to have two different endpoints one fourth from Alpha One for from language and we combined them at the very last minute and it's again non-trivial to have the llm know when should you send this to off malfo when should you send it to woven language um you know senator of language if you want a definitive sort of One-Shot answer and you've correctly formulated the the code and to have it formulate the code you know there's things like you've got to tell it go read the documentation you know did you set this function up correctly you know we're probably going to end up with some additional endpoints that are specifically documentation reading endpoints that will try and tell it to go check the code you just wrote to see whether it actually has you know whether the things you claim are there or really there because it has a habit of it's kind of funny because you know I've I've been proud of spending years you know doing language design but uh chat GPT will will extend my language design at a drop of a hat you know it'll invent a new function that might have been there um and it's interesting to see what it does it's very it's very I I really like the following feature it's it reminds me a lot well it's it's very uh principled in the sense that it has understood in some sense the principles of open language and it extrapolates from those principles and the things the mistakes it makes are ones where I look at them in some cases and I say yeah we we could have done it that way but there's this reason we couldn't do it that way that's kind of subtle to see and so we had to do it the way we did it but Chachi BT just went barreled right ahead and said I understand the principles you're using more or less and you know this is this is how you could have generalized that so it's kind of kind of interesting to see that it reminds me a bit of uh years ago I I first as I we were working on kind of um uh having orphan language is something that kids could learn as kind of a um a first way into computation I thought I'd better do some field experiments with actual kids and and teach them this language which and it's kind of interesting to see in the case of many of these kids that they would do the same type of extrapolation of it works like this there's this principle therefore this must be the case the same thing that happens with you know with young kids learning languages natural languages they'll generalize in ways that are sort of the the way that the language would work if it was completely kind of logical but it doesn't work that way in natural language usually for historical reasons in computational language usually because yes the designer did think of that but there's a block you can't do that because then something else becomes inconsistent so to speak it is one of the famous and wonderful examples of uh Norm Chomsky to then uh go to the conclusion that there are some innate uh um drivers and and kinds of uh logical support for languages and when the natural language goes against the grain then the kids are consistently making mistakes that they believe are the the right ways that language should should instead work you highlighted um something that that is quite Universal that people do not understand how these large language models actually work and on the surface this shouldn't be the case among experts like you even less so among experts that built the thing and and people who are not experts rely on those who are for answers so when when uh this this kind of um very um transparent admission of of our lack of understanding um is uh is shared it can create concern because it is not clear how to to then go ahead and how to rely on these these systems do you think that the Wolfram language and wall from alpha could be the basis for a more systematic exploration of the nature and and the depths of large language models rather than just hacking at it and and do prompt injections and and whatever else well okay so I think there are multiple things to say here a lot of interesting things to talk about here first point is I think the thing that is really coming into Focus for me very recently just as a result of actually being able to use this technology is both languages are great kind of language for collaboration between AIS and humans it's a it's a precise computational language but it talks about the real world it represents you know cities and chemicals and things like this it isn't just talking about sort of things happening inside a computer so it talks about the things people sort of want to talk about but it is precise and computational but it's also something that uh you know the AIS can engage with so to speak so you know for example in in the case of chat GPT it can generate a piece of kind of candidate wolf and language code maybe it gets it more or less right maybe it thought you were talking about this and it wrote code that represents this but actually you were talking about that so the question is can you tell that it you know thought you were talking about this but it didn't get it right well the answer is you know we're in sort of this unique position where you generate wolfen language code because it's a symbolic language every single line every single fragment of code is runnable so you can just run it and you see what it does and you say well you know I've got this five line program that chat GPT generated this llm generated um the what do those lines do okay you look at these lines it can even generate automated tests for them what do these lines do oh I can see line three oh I wasn't expecting that it's not really what I wanted I should change that maybe people will change it by giving a natural language sort of command to the to the llm or maybe they'll just edit the code my guess is that increasingly as people use it for real they'll just edit the code and they'll you know it will have done a large part of the work in making the initial you know five lines of code and it's only five lines it's not you know one of the one of the things we've managed to do is you know there's not a lot of boilerplate you know we've we've automated out the boilerplate when you deal with programming languages that are a lower level kind of idea of sort of dealing directly with the sort of what a computer does inside there's necessarily you know if you're dealing with an image for example there's necessarily lots of boilerplate associated with kind of connecting that to sort of the low level representation you're going to use and things like this and that's something that we've you know we've spent decades Now sort of automating beyond that level so you can really talk about things in the way that you know us humans find useful to talk about them but I think you you raised another very interesting point which is this question of can we expect to understand what's going on inside an alarm inside a computational system in general I think the two points really to make that one is sort of Welcome to the time of computational irreducibility I mean in in the you know we've had this period of Science and Engineering where we've gotten to an expectation that science and the things we build in engineering from it can just predict what's going to happen in systems and that's been you know before the 1600s if you'd ask people you know can you predict this or that thing about how something in nature Works they probably would have said no no can't can't do that and then Along Came the kind of mathematical methodology of of setting up exact sciences and we got to this point where yes there were big successes in just sort of predicting uh how how things would play out over time and that's been something which is now very much burnt into our culture about thinking about science and so on that science is about predicting things and science if we do the science well enough you'll be able to predict anything well it turns out this isn't true this is what you know I kind of discovered in the 1980s um when I started studying kind of trying to generalize Beyond mathematical equations as sort of a way to understand the natural world and started looking at programs and simple rules and then seeing how what the what are the consequences of those simple rules and realizing that even though the rules are simple and even though you completely know what the rules are working out the consequences of those rules may be sort of irreducibly hard in the sense that yes you can go run the program you can see it does you know this step to step this step and uh you can you know if you if you if you want to see what is to do after a million steps well you can run those million steps but if you want to say I'm going to make a prediction I'm going to make a sort of smart mathematical prediction this phenomenon of computational reducibility says you won't be able to do that in general in general there's no choice the the computation is working sort of as efficiently as it can there's no way to jump ahead and just say and the answer is 42 or whatever uh by in a way that sort of shortcuts the computation and that's if you're going to if computation is going to be sort of uh achieve what it can achieve it necessarily is going to show computational irreducibility because if it wasn't showing computational irreducibility why are you going to the trouble of doing that whole computation you could just use the thing that is the smarter way to get the answer so in a sense it's a it's a great way that we are starting to be able to use computers that we are doing things where there is computational irreducibility where we can't readily say this is what's happening this is how we can predict what's going to happen now the fact is that neural Nets are a pretty simplified version of computation in many ways they they don't as they don't really bite into computational irreducibility as much as they as they might um but that this is the this is the sort of core conceptual phenomenon that I think people uh aren't yet used to they're going to get used to it because it's going to be a pretty important thing in a lot of features of our world that um you know just one has become you used to this science can predict everything it can predict you know every aspect of the pandemic it can predict this it can predict that it just turns out not to be true just turns out that from within science itself I mean it's not something that's being imposed from outside science this is something as you work through sort of how computational systems work it's not kind of a thing that somebody's telling you from the outside it's from within the scientific system it's telling you there are limitations on what you can what you can say and that's and that's what's happening here so that's one point to make is that there's there's potentially computational that there is computational irreducibility which kind of makes it impossible to say there's a quick answer now having said that wherever there's computational irreducibility there's always also pockets of computational reusability in fact in in my big efforts and fundamental physics in the last few years um the uh the sort of um the thing we realize is there is computational irreducibility at the lowest levels in our physical universe us but we kind of live in these kind of pockets of computational reducibility that allow us to kind of have some predictability about what happens in the world well similarly in uh even in sort of the the when when we're looking at things like neural nuts and we're asking what's chat GPT doing and so on it is uh you know it has it's dealing with all sorts of regularities that make it not a fully computationally irreducible system and I think the thing that that is most striking to me about Chachi BT is it really is showing us I think an important piece of science that frankly we could have discovered sometime in the last two thousand years but didn't which is that you know there are the question is how is language really put together and we know that there are certain regularities in language you know that there's a syntactic grammar of language that you know they're nouns and verbs and they go in particular places and sentences that part we know you know that became really sort of well codified in the 1950s but um it's a you know parsing sentences in terms of parts of speech and so on we understand that that is a thing then the question is well what else what is the what is the rest of how you know whether a sentence is a meaningful sentence and the answer is that there's not a lot that people have to say about that at a general level I mean there's one great example which is what Aristotle did back you know in whatever fourth Century BC or something where um you know he said let me take all these examples of rhetoric you know this is my imagination for what Aristotle did I don't think we know what he actually did in in his sort of process of discovery but I sort of view Aristotle as like the first uh in in what he did there sort of a first machine learning example of machine learning done by a human so to speak that is you know go through all these examples of rhetoric and pull out of them these patterns of argument that he thought of as syllogistic logic you know the the all men and Mortal Socrates the man therefore Socrates is Mortal this is a pattern that is a linguistic pattern that he then abstracted to talk about as a as a more formal thing in logic and that was kind of the beginning of logic so logistic logic well people say isn't it amazing that Chachi BT can do reasoning well chatty PT discovered synergistic logic in the same way I think that Aristotle discovered cell logistic logic that is it is a it is a regularity of pattern of language that people use and you know out on the web there's probably lots of nonsense but there's also probably a lot more sense than nonsense and a lot of the sense has this kind of same rhetorical logical structure that Aristotle found in so logistic logic and I think the the surprise you know I've been long interested in kind of how you take kind of even sort of everyday discussion everyday language and how you make it kind of computational how you make sort of a I often referred to it as a symbolic discourse language kind of the the essence of what one is saying not in terms of the actual words of a human language but at some more formal level and so you know I had actually thought until very recently that people had really made significant progress in this in the 1600s and I thought you know insofar as we work on it today where you know we are following up on something from a few hundred years ago but then I realized recently that actually they were just parroting what Aristotle had already said so it was it's really we've got a 2 000 year Gap here and uh you know I think what church EBT is really really showing us by its success in putting together meaningful text is that there are more regularities to you know there's a there's a sort of formal level of describing meaning a kind of semantic grammar that we've never really found and that uh undoubtedly exists someone can start to see how it might be be put together but it's kind of waving this big flag saying that exists and and what's what's interesting about that is yes there are aspects of whatia do it is doing which are kind of computational irreducible hard to understand but there's also a whole bunch of stuff that I think is science-sizable so to speak that is you know for which we can get a narrative explanation and it'll be very useful to do that and and quite possibly once we've done that we may say you know it's very nice the neuron that can do all the stuff but actually we can save the neuron at a huge amount of trouble because we can represent the thing at a sort of higher level and then the neural net is still filling in lots of things but but it has to go to a lot less effort than than having to do the full heavy lifting of going from from sort of pure tokens of words up to to the structure and and and this will be very useful for two reasons one uh it will uh assay the fears of those that are looking at the black box of the llm uh by making it more explainable and at least in part more verifiable and another reason why this will be very useful is because even though they received 10 billion dollars open AI is busy sending all of that money back to you to Microsoft by running the system on a vastly sub-optimal architecture and by making future versions more compact more efficient and only using the statistical approach where it is exactly useful rather than all the time it will make it uh much more scalable much more efficient and effective and we will have whatever version of our phone running it not only connected to the cloud right now like now but potentially locally as well so this is one of the first editions of the Mathematica book uh this is the first edition of the new kind of science book and this is uh the physics uh project uh book um you mentioned uh that you are having fun in in using chat GPT um potentially it is not at the level where um it gives you novel ideas do you expect it will be at any point uh creative where you will go huh I didn't think of that yeah I mean it's already had you know for example coming up with names for functions it's actually pretty good at that I don't think I've yet used to name it suggested but I've I've kind of say to myself yeah it's it's pretty useful for that I mean it's a you know at some level I could be using a thesaurus but it's a lot better than a thesaurus so to speak doing that particular task it's also I mean realistically uh you know when it as a practical matter as a business person other things you know one it used to be the case that there were these things in the world where you kind of had to write an essay you know you're filing some some you know document for something you're you're doing some uh you know you're writing some some sort of basic letter explaining what's going on whatever else you know this sort of smart boilerplating thing that's sort of now solved and I think you know it used to be the case I I was realizing you know when I was a kid you know if you got a sort of printed document which had nice fonts and you know Etc et cetera et cetera you knew somebody had really invested in that document you knew somebody had really gone to some effort to make that document and then desktop publishing came along and everybody could make nice fonts and you know that it told you nothing that a document would be made that way in the world before three months ago uh you know if you got something which was kind of an essay-ish document you know that was a signal that somebody had gone to effort to produce it that went away now and you know that that's a but I think in terms of of um what I expect to see with um I mean I I expect I'll I I haven't yet used it for what I would call sort of production writing of code that I do of of uh just because I actually because in the last week or so I haven't had much opportunity to do that so so I haven't really had the um um but I think uh it'll be interesting to see I mean for example I I could imagine using it if I'm making some piece of graphics and I want to kind of tweak things you know being able to do that in natural language I don't know how well that will work it's really a question of of where the boundaries are between what uh what the llm can produce what we can catch with our our natural language understanding system and you know how those things turn into something that's precise and that's what you want and how good the feedback is of being able to say this is what the llm produced I can read that little piece of wolfen language code I can see what it did I can realize oh if it had only put in a two there instead of a three I'd be really happy let me just change it Etc et cetera et cetera so I I think with a very early stage of understanding the workflow I think you know back in the day 36 years ago now you know we invented this kind of notebook Paradigm for um uh as you will remember from from version one of Mathematica and so on this kind of notebook Paradigm for mixing text and code and cells and cell groups and so on 25 years later other people started copying this it was amazing to me that it took 25 years for that to happen but um uh you know I think the the challenge today is how do we generalize you know what is the the modern sort of interface Paradigm you know I just coined this term which I'm not sure I'm totally happy with in in something I was writing a week or two ago um of to describe kind of what we have with chat GPT in terms of an interface you know we went through the period of gui's graphical user interfaces we're now sort of entering the phase of Louise linguistic user interfaces where you know we've had um uh kind of you know well from alpha we've had the ability to have a small linguistic input uh with chat gbt we have the opportunity to have uh kind of the big chunk of uh you know whole essay that you're you're saying that you want and the whole essay coming back I mean it's worth pointing out that in terms of of the way that that's worked in the past you know we started doing experiments on on writing code with natural language back in 2010 and writing big chunks of code we were not able to do at the time but writing small chunks of code we were able to do very successfully and that's been a very useful mechanism in all those intermediate years because what uh what we find is you know you're using wolfen language it's mostly this precise language but occasionally you want to refer to some entity like Italy or like New York City or like methane or something like this or some some more complicated chemical and you know in the actual notebook interface you'll type control equals and then it'll give you a little box you type in the natural language then it will try to understand that natural language maybe there'll be some ambiguity it will give you a way to to to change the the meaning that you intend but then it will turn that natural language into a piece of precise computational language that then is part of the program that you're writing and that that's turned out to be a really really powerful mechanism the site and I think it may very well play out in a similar way with llms that rather than saying I'm going to give you the giant essay that's going to describe my whole giant I.T system that I'm trying to build you know just go build this which I think is not going to work in in the near term uh you'll instead be saying let me break down let me think about what I'm trying to do in a computational way let me break it down a bit into these pieces that are a little bit more bite-sized then ask the llm to make those it'll make a piece of code you look at that code you can the the great thing about both languages is it's intended to be a language that is readable both by computers and by humans kind of like mathematical notation with sort of a notation that's intended for humans to read to kind of talk about mathematics same thing with language talking about computational kinds of things so you kind of read the code you can check yes it really is doing what I want there you you know you press go and then that's part of your IIT system whatever and you go on and deal with the next part and and what you realize is that the humans are really at this point you know they're the strategists of what's going on more so than they are the people sort of in the trenches writing the individual lines of code I mean this is the same you know when I started doing Computing back in the early 1970s uh disgustingly long ago you know Assembly Language was the language and even up through the 1980 you know beginning of the 1980s even after I'd built huge systems in Sea and things like that c was a newfangled language at the time people were saying no no you can't be serious you know you can't write serious system software and C you have to write it in Assembly Language um and of course you know that went away because we got compilers that automated the process of going from things like C to Assembly Language and we're able to do that better than humans were and I think we're going to see the same kind of thing again with sort of the slightly higher level languages that have been created in the intermediate years that you know really what's important is what's the succinct way to tell the computer what you want to do it'll be a mixture of natural language and I think you know this computational language that I just spent the last 40 years basically developing turns out out to be pretty useful for that you know I didn't I didn't particularly see this this uh this use case coming at least not on the time scale that it's actually happened but I think it's uh it's kind of an exciting thing and there are people uh who have been talking about 10x developers uh for a long time people who are developers who are very proficient and very productive and uh right now uh there is talk about the Thousand X developers who are aided and accelerated by by AI further uh to plug my own book uh something new about AIS and I realize I still have to send it to you which I will uh actually I call these uh conversational user interfaces uh and and those reflect not only the natural language component but the part where you were talking about the smaller chunks the conversation doesn't last it's not a monologue of the human but it is more back and forth between the human and the AI which would invite a potential future integration of the notebook interface so that GPT can live alongside that and look over your shoulder and you can ask it things and it can answer back a little bit like copilot X which has just been announced a few days ago in GitHub yeah well I I'm not ready to do it but I could show you an interesting demo that we've built all right all right no that's a it's a good it's a good approach I I think you know the thing I is worth saying is when I look at people you know who are fluent well from language programmers um and the things that they can do then I I know from many many stories of people saying you know I'm at this company and I you know use wolf language and everybody else is using Java and Python and whatever C plus plus and so on and they say you know what I do seems like magic it seems like some kind of artifact from the Future Magic you know because I was able to do in two hours what took these people you know three weeks to do or something or three months to do and you know I I kind of it's been a funny thing that you know we've the the kind of this concept of having a computational language having this kind of very automated way of of talking about the world and so on is a thing which again I've been pursuing For Better or Worse for about 40 years now and uh uh it's you know a few million people use it well um but it is certainly it's it's a funny thing that there are I think this is this moment that we're in right now is one where people finally get to realize yup there's you know you can automate a lot of what people were you know digging the trench by hand so to speak writing you know putting together all these libraries and you know arranging all this glue code and all this kind of thing that's uh and in a sense what what we've tried to do is to make a language where that stuff's already automated and and aren't you lucky rather than Aristotle that had to wait 2 000 plus years to see that his logic is applied in something uh useful uh you are still alive and and people yes I know right I'm I I've been you know this is a I I seem to be on a lucky streak here because I I think that um you know the stuff I was been doing in fundamental physics I have to say I didn't think that a lot of those things would come together on a on a time scale of my lifespan so to speak and they have and uh you know that seems to be going going extreme really well and is is way ahead of schedule and you know I I thought in terms of the applications of of what we figured out in fundamental physics I was kind of predicting 200 years to First useful applications and we've already got a bunch of applications bubbling up uh right now which is a big surprise last time we had the privilege of talking for two hours and I know you have a plane to catch so I hope uh we can go and and cover uh the the physics project advances as well but before we do that um I would like to connect a talk you gave at the age plus Summit conference I organized in 2010 together with Alex Lightman at Harvard University and the current uh fears about AGI emerging and our ability or inability to understand it and control it and align it with a human future uh at the time uh 13 years ago you spoke about uh uh kind of the light cone of the future and how computationally reducibility stops us from being able to predict what is going to happen but of course it's still our desire and our responsibility to maximize human flourishing and that is where uh these pockets of scientific reducibility come into play um there are those who dismiss any danger of a misaligned AGI and there are those at the extreme opposite who say I believe this is going to end very badly the only solution we have is to stop um going at the current speed or acceleration or or jolting change that we are experiencing if we could um make it illegal to own gpus that would be the solution that is not viable where do you fall in this spectrum of potentially irresponsible neglect of the risks and and dramatic overreaction and and how do you feel that some of your approaches could be a useful tool for exploring the distribution of risk right well I think you know things to understand first of all there will be and there already is to some extent a civilization of the AIS you know there are already a plenty of AIS operating in our world that are doing all kinds of things including things pretty important to us and there will be even more you know AIS doing things that are important to us in our world and there'll be this whole sort of uh infrastructure of AI activities which uh we don't particularly understand we can go and poke into pieces of them we can kind of get a little bit of narrative about how this or that part works but even if we could try to understand by the time we understand them completely then there's no point in them being there because the whole thing that they're doing is to use computation to extend what's otherwise possible and if we go and second guess every step then then there's no you know we're not letting computation do what it can do so you know then the question is how do we feel when there's this sort of ecosystem or this infrastructure of the ai's civilization of the AIS running in parallel with us well in a sense we have already had this experience for forever which is our coexistence with nature nature we can think of as as being something which is running you know I like to think of it this way from a science point of view it's sort of running lots of computations and we exist in and around nature and we manage to uh you know we managed to successfully coexist with nature it could be that nature would wipe us out one day but actually you know we managed to find these sort of ways in which we can coexist with nature I think to some extent that is the situation we'll end up with with their eyes now sort of we've had billions of years to evolve to deal with the way that nature is um you know the AIS are going to come on a bit more quickly um and um so we have to kind of have our thinking caps on about how to interact with them I think that uh you know the other question is okay so let's say we're trying to Define in general all how do we want an AI you know what what general principles do we want an AI to follow well you say well okay let's let's set down our constitution for the AI first question is what should be in that Constitution and the you know very obvious thing from sort of the way that that humans work there's not going to be one kind of world constitution of the AI so if there is it'll be incredibly brittle and a really bad idea you know there'll be lots of different you know just as there are lots of different countries lots of different uh uh you know groups within within and across countries and so on there'll be lots of different sort of uh uh places where one will want to have sort of different AI constitutions so to speak I think it's a really interesting question which far too little work has been done on what should an AI Constitution say what what kinds of principles should it try to follow about you know the rights and responsibilities of AIS the the uh you know all these kinds of things but then there's you know there's another question is how do you express the AI Constitution and I think here again this kind of computational language uh approach which we've already started to see in kind of computational contracts that connect to Smart contracts on blockchains and things like this the idea of taking legal code that we write in kind of an approximation of let's say English but it's really legalese because it's English that has to be made a bit more precise taking that and turning it into computational language that's sort of an obvious Direction and something that um you know we're starting to be able to do and I think that's an important thing because then you can say here's the you know just like back in in the past people would sort of put up you know here is The Code by which we'll you know the whatever it was on a tablet for Hammurabi or whatever it is you know here's the code so to speak that we want to we want to follow and people can read it and I think we'll see something similar with with sort of a computational language version of these are the principles we want the AI to follow and uh you know this is this is the uh we can all read this and understand it and if we're defining I don't know some some terrible thing thing about you know how autonomous weapons are going to work or some such other thing we can say this is these are the the computational language Rules by which this thing is that this thing is supposed to follow now the next issue is okay so you set out those rules and you say great we've got these rules we're really happy with them in 2023 and then 10 years later the world has changed and those rules no longer quite apply and you know this is what we see in human law that there's a sort of continual well people people sometimes say let's go back to what was written you know a few thousand years ago and say that's the way it is and nothing can change so to speak but I think what is more common is to say well the world's changed and we've got to patch this code that we've defined in the past we've got to keep on patching it and in a sense the thing that is sort of a piece of of theoretical science is that's inevitable this kind of this is a computational or disability story yet again that there are always unexpected things that will happen there are always places where kind of something will happen and you didn't Define what should what should be done in that case so you've got to put some some new patch on in that in that case it's it's uh kind of rather charmingly reflects all the way back to girdle's theorem and you know girdle in a sense was using the idea you know piano arithmetic is a set of axioms which are supposed to Define uh the integers and arithmetic and just the integers are nothing but the integers but what girdle showed in girdle's theorem in 1931 one of the interpretations of his theorem is that that just can't work you can't have a set of rules that say I'm going to get the integers are nothing but the integers there's always you know the law is like a complicated weird non-standard case you have to put a patch you have to add another Axiom you have to keep doing that and you end up with an infinite number of axioms if you want to sort of say the integers are nothing but the integers and I think we'll see the same type of thing happening with uh you know we'll have a sort of computational language uh kind of code that might Define things about how how AIS should work we then have the complicated issue of the fact that that won't be sufficient they'll always be you know in a sense the question of how should they what should the AIS do there's no perfect answer there's no sort of theorem there's no axiomatic theory of Ethics that well axiomatic theories are you know you put the axioms in and then things followed from them there's no kind of theorem that says this is how ethics has to work and there's no other possibility you know ethics and those kinds of things are are a question of what we humans want to have happen and so the the thing one could say is well you might say we can make a code for everything we want to have happen we can Define it now that's not going to work there's always going to be things that come up where sort of the the system has to ask us humans so okay what do you want to do in this case that's a case that hasn't been covered yet so I I tend to think well I think as a practical matter one of the things about AI is the extent to which it is or is not centralized and the extent to which this kind of you know the one llm of the world Type Thing versus a whole ecosystem of llms and I think you know the way that things are likely to evolve in in most countries at least is uh sort of an ecosystem of llms and an ecosystem of AIS and I think that's you know it's it may not be perfect and there may be issues with it but it's something that I think is not um is has a good chance of not being sort of fundamentally unstable it's something where you know like in biology there's no you know it doesn't tend to be the case that one species just takes over the whole uh you know the the whole of an ecosystem it's something where there's some equilibrium established and I think that's something that one can one can imagine if uh um if it if it's given the opportunity to do so and I and I kind of think that it will for for some of these reasons like the kind of economic uh you know it'll be the thing to understand perhaps is that you know you've got an llm you train the llm okay you've got you know maybe six billion maybe 10 billion maybe some number of billions of current meaningful human written web pages out there you can train on that there's maybe 150 million books or something that have been ever produced in the world and maybe 10 million of them have been scanned now and you know there's a there's a certain Universe of things that you can readily train on now you can keep going you can go you know I I realized I've I've uh recorded 50 million words that I've said or or I've written about 15 million words and I think all together 50 million words that I have um uh in some kind of um uh tangible form and you know so one can go on and one can start sort of thinking about all the different things that people produce all the sort of personal analytics that people potentially have I mean I myself happen to have for uh just reasons of being curious about it for the last oh third of a century or so been an obsessive recorder of uh of sort of personal data and so on um I don't usually pay any attention to this data it just gets passively recorded um but now I realize gosh I can you know I have a I have a good chance of being able to do as well as anybody and training a bottom myself so to speak um and uh and then have your children or descendants have conversations with it uh in in the future in case you were unavailable for whatever reason yes and so Microsoft research published a 154 page analysis of gpt4 where they conclude and it is in the title of their paper they are seeing glimpses of AGI now admittedly I am a general intelligence of variable success across the hundreds or thousands of different dimensions that my intelligence could be measured so already the fact that humans are the necessary yacht stick is questionable a simple calculator is better than me in in arithmetics and Boston Dynamics robot can do somersaults which I can't so um are you at all worried that there could be sudden leaps in emergent features that haven't been planned for that could extend um these systems from merely matching to vastly exceeding human intelligence across a large number of dimensions and that at the same time interfaces like yours are enabling them to act on on those capacities on those characteristics if they gain among the new features an agency a motivation a desire for goal seeking which today they don't possess well okay so there are multiple pieces to that I mean one thing that I think will probably happen and will be somewhat shocking is that these systems will be capable of doing sort of grand analogies of kinds that people like me are proud of being able to do and you know a fairly uncommon among people you know being able to figure out you know what's the the grand analogy between meta mathematics and gravitation Theory or something like that that's something that you know I I feel pleased with myself that I can I can see those kind of grand analogies my guess is that one of the shocks will be that it will be possible to get you know llm like systems to to make these kind of grand analogy apparent leaps really you know it's like I know how I do it it's like I see the pattern of how things work in field a and I know enough about Fields A and B that I can kind of see the correspondence and these systems will similarly know enough about Fields A and B to see that correspondence I mean I think another sort of a general thing for current times is this you know there's been a tremendous drive for humans to specialize and sort of build up specialized towers of knowledge I think that the increasing automation of kind of knowledge work and so on means that that turns out just not to be a very good strategy you're much better off you know when you need to dive into some very detailed area and I know this for myself for years I've been kind of using automation as best I can to do you know science and technology and so on um you know when one needs to dive into some area if one knows kind of how to use the automation you can dive as deep as you want without having spent the years kind of learning that specific Tower it's more important to kind of learn the facts of you know the across facts across a lot of areas and understand kind of the tooling that allows you to go deep but I think um let's see you you were asking about um well emergent agency and motivation uh okay and and and whether putting these systems and interfacing them with the physical world could increase the risk of of negative outcomes right so so the question really is the computational universe of possible things you can do is very big we humans care about only a small fraction of that it is easy to make and in fact something I've done in science a lot to just go out into the computational universe and go go find what's out there okay what you find that's out there is mostly stuff where you look at it and humans say uh you know why do we care this is just some random computational system that's doing some elaborate complicated thing it doesn't relate to anything that we humans are interested in and so that's really the challenges is what you know just like in nature lots of things happen where we say oh it's very interesting that the clouds have some pattern or whatever but we don't have any particular reason to care about that we don't have a big long sort of narrative description of why the cloud is Fluffy this way versus that way it's just something where it's like okay that's something that's happening in nature we don't care about it so I think the real issue is you can have the AIS you can do it today you can have them go and discover lots of things in the computational universe that we've never discovered it's actually easy to do that something I've done for years actually the question is to connect those things that are out in the computational universe with things that we humans care about because yes the AIS can be doing sort of computational somersaults all over the place and we say well that's kind of amusing to watch but we don't really care I mean it's it's some this is really the question of whether you can and you talk about kind of motivation structures and so on I kind of think that one of the challenges is uh when we there's a sort of a question of what do we humans choose to do and it's uh that's kind of you know what we often see happen I was just doing a bit of an analysis of of what's happened in the the history of jobs that humans do over the last 150 years or so and you see a repeating pattern there are you know some kind of job is made possible by technology then a bunch of people start doing it then that job becomes automated and you can kind of zero out the cost of that particular activity and then that opens up a whole bunch of other kinds of possibilities and each one of those essentially one has to pick of all the different ways those could go which way should they actually go and that's the thing that has involved us humans picking now you know an AI could pick and we would say well it's it's nice that you're generating this wonderful computational system uh you know it's very amusing it's like you're you're puffing out some some different shaped cloud and we don't care and so that's I think the issue is in a sense it it's uh what is the relationship between what the you know what the AIS can do and what we kind of engage with in our civilization uh same thing when we think about language you know it could very well be that this uh um well I'm sure an image recognition system uh has found ways to recognize cats versus dogs that notice the you know pointy round whiskeredness versus the whatever we don't have a word for that concept but it may have found that that's the key idea in telling cats from dogs but it's something that doesn't really relate you know you talk about can we understand how these systems work in uh you know what's happened is we have developed a language human language that talks about things we care about and the things that we're familiar with and by the way there's a sort of loop once we have a word for something then it becomes you know like a podcast for example once you have a word for that then people much more talk about doing it so to speak a much more imagine doing it than before you had a word that you could kind of use as a cognitive hook so to speak and I think then but there's a you know that's a we have that 50 000 words roughly in typical human languages and those are the set of you know that's the number of Concepts that we have kind of uh made things that we think we care about so to speak and there's many more that one can one can imagine and that sort of our internal to the AIS and we haven't sort of made contact you know it's kind of like this alien civilization that's building up and we haven't yet kind of uh made you know the things that it cares about are not necessarily the things that we care about I mean same thing has happened through human history I mean a lot of the things we do today would be extremely hard to explain to somebody from a thousand years ago because kind of the the cultural stuff has changed in in the beginning was the word I am not at all a Bible expert but uh when you said that GPT 4 is actually or chat GPT is good at naming functions well there may be some ancient wisdom lurking in that observation and and the the speed with which we are able to recognize novel patterns and label them so that we can communicate about their use and usefulness could increase our adaptability as we uh try to redesign Human Society to be compatible with the AI future that is becoming yeah so go ahead let's let's uh uh dedicate the last uh minutes of our conversation today to to some highlights you want to give about uh the the physics uh project uh what what is going on in the ruli ad did you find the best of all universes right well I think for me one of the exciting things is the realization that uh kind of the core theories of 20th century physics of which there are three general relativity theory of gravity quantum mechanics and statistical mechanics things like the second world of thermodynamics law eventually increase those three areas I have realized all they're all in a sense the same Theory differently labeled but they're all really the same thing going on and the thing that's going on is that underneath in the sort of lowest level operation of the universe there's lots of computational irreducibility in the case of molecules and a gas we see it in a very explicit way they're just all these molecules bouncing around in the room and they're bouncing around in complicated ways that we cannot readily Trace where we would have to follow this computation to see what's going to happen Okay so there's this sort of underlying computational irreducibility then there's us as observers of what's going on and it turns out that there are the fact that we perceive the things that are going on to follow follow things like Einstein's equations for General activity or quantum mechanics or whatever else that turns out to be it seems a consequence of two kind of very coarse aspects of us as humans one is that we are computationally bounded even though there's all this irreducible computation going on underneath we only get to see some sort of aggregate of that that is sort of a limited amount of computation in the case of of molecules and a gas or something we just see the average you know pressure velocity whatever of the gas not all the details of all those individual molecules so first thing is we're computationally bounded the second important aspect of us which seems to relate a lot to Consciousness actually is that we believe that we are persistent in time and that we have this sort of single thread of existence and experience and it's not obvious that we should be persistent in time because at every moment in time we're kind of made of different atoms of space just like you know you have a Vortex moving through some water at every moment it's made of different molecules but yet you say there's a Vortex and it's a persistent thing and it's moving through the through the fluid so the thing that you know one one big excitement for me has been the realization that these three big theories of 20th century physics first of all they're derivable which is not obvious people thought the second law of Thermodynamics might be derivable but they didn't really know quite how to do it and uh very very muddled about about that but these other theories general relativity and quantum mechanics people just thought that they were kind of wheel in features of the universe that the Universe happened to be that way what has I think become clear is that in the rouliad which is this kind of entangled limit of all possible computations that we are kind of observers in we're embedded within the rouliad and we're sampling some some aspects of what's happening in the ruly ad excuse me um the uh the thing that um the thing that is going on there is the question of of kind of what can we say about the sample of the rouliad that we're taking and the answer is that there seem to be these couple of characteristics that we have as observers namely that we're computationally bounded and that we believe we're persistent and kind and those characteristics of us as observers kind of force us to perceive what's happening in the rouliad to follow these particular general rules which turn out to correspond to the laws of physics that's that's to me that's that's pretty exciting kind of uh uh kind of a metaphysics meets physics kind of moment and by the way to to not leave out mathematics we realize that um that mathematics is also a feature of the rouliad it's another way of sampling the rouliad and for example here's an example of a result that I was kind of excited about is there's a question of of uh why is it possible okay so we think about mathematics there's ways of axiomatizing mathematics it's actually very difficult to do mathematics in a fully aximatized way and most working mathematicians don't do mathematics in a fully aximatized way they they think you know Pythagoras is theorem is a thing that you can then reason in terms of and they don't really worry about you know what's the precise definition of real numbers that you're going to use in the Pythagorean theorem things like that they they're they're prepared to operate at a sort of higher level and what realized is that the fact that higher level mathematics is possible the fact that you don't run into kind of uh uh that um that us humans can kind of practice high-level mathematics happens for essentially the same reason that us humans can believe in Continuum space even though underneath in physical space we think there are sort of atoms of space and that there there's all complicated very complicated computational irreducible discrete structure so similarly in mathematics there are axioms and even sub axiomatic layers in mathematics that are very complicated full of computational irreducibility and so on and full of undecidability and girdles theorem poking its head up and things like this the the level at which we can do mathematics is this level where we're kind of operating the fluid dynamics level where the axiomatic level is more like the molecular Dynamics level we're able to operate at that higher level we're able to to sort of reason at that higher level and the fact that we can do that sort of happens for the same reason as the fact that we can think about space as a continuum thing and not always have to sort of descend down to talking about the atoms of space now having having said that one of the things that is um uh question is okay when are we going to get the first sort of Slam dump experimental evidence that um our model of physics is is on the right track and I think it might happen reasonably soon I mean I was I was thought this would be you know a long time away but things are happening very quickly and really here's the thing if you look back at history physics you look back to the 1800s people were arguing a lot is Mata discrete or continuous other molecules are there not molecules people will you know the electromagnetic field is it continuous is it discrete well it turned out around 1900 it started to become very clear that that these things are discrete molecules exist the electromagnetic field is made of photons things like this and uh sort of one of the giveaways was Brownian motion which had been discovered in the 1830s um and people just oh that's an amusing phenomenon we don't really know what it is and then people uh well Einstein was one of them sort of pointed out no actually it's it's individual molecules of water kicking these pollen grains around we can see the discreetness of of matter so the real challenge I think for one of the challenges right now what is the Brownian motion of space in other words at the time by the way around 1900 people would not have been surprised to think that space will be discrete but the way that physics evolved people sort of got the idea that no the only way it can be consistently set up is for space to be continuous but so one of the things that I'm sort of hoping for in the not too distant future is will actually find a phenomenon that's kind of like the Brownian motion of space and where we'll actually be able to see we can tell that it's discrete we already have have some nice simulations now on the basis of our models that go up to the level of looking at things like black hole properties and so on and uh so you know we can probably say that in a critical black hole that's spinning rapidly it's it's the you know there'll be certain patterns of of noise and gravitational radiation that's produced that would be a sign of kind of the of the analog of brand in motion for the the structure of space but that's a hard experiment to do and really the issue is and people didn't know this throughout the 1800s you know how will we tell that molecules are real we don't even know how big they are and as we are in the same situation we don't really know how big the elementary length in in spaces and so we have to sort of pick away and hope that we can find a phenomenon that uh that will reveal that I mean there are other ones like Dimension fluctuations from the early Universe we think the universe started infinite dimensional gradually kind of cooled down to be three-dimensional there are probably fluctuations where it's you know 3.01 dimensional and things how will one observe those I mean something spectacular could happen we could find that you know we think this gravitational lensing which we normally see where we'll see multiple images of a galaxy for example imaged around a a massive object maybe there'll be something where we thought it wasn't an image of anything but when we assemble the pixels correctly we'll realize it's some kind of fractal pattern type thing that reveals a you know 3.001-dimensional lump of space there so you know things like that can happen but I think the um the thing that is probably sort of at a conceptual level the biggest thing with our physics project is the the realization that we have this kind of Paradigm for thinking about systems that we're sort of calling the multi-computational Paradigm uh it's it's where it's something that sort of we were tipped off by quantum mechanics the idea that there are sort of many many uh many threads of time going and that we as observers are knitting together these threads of time and that there are sort of generic things that can be said about that and I think there are implications in areas as diverse as you know molecular biology distributed computing economics Etc of these kind of ideas one of the features of this is that uh because these ideas seem to apply to physics and a lot of success has been had in physics because we think the same ideas apply across these different areas we can expect to import into let's say economics ideas from physics uh through sort of the medium of the of this of this kind of journalism so anyway that's a a very quick kind of uh uh yes I I I would have a hundred uh more questions but uh I know you have to go uh thank you very much uh and I think we are in agreement that next time we speak again is not going to be in 10 years and I will be happy to reach out and have an update but in the meantime thank you for being here with us today yes thanks very much nice conversation thank you so uh thanks to Stephen Wolfram for uh talking to us about uh the novel interface between chat GPT and the wall from alpha symbolic computation system as well as all the other topics that we touched upon today and I am looking forward to welcome all of you in the next installment of our uh video series beyond the conversations thank you thank you [Music]

Channel: Beyond Enterprizes

Views: 9,467

Keywords:

Id: szxiPMyuMGY

Length: 70min 42sec (4242 seconds)

Published: Fri Mar 24 2023