Emergences A Talk By W. Daniel Hillis

[Music] my perspective on this is maybe closest to George's time coming out and I liked his introducing himself as being interested in intelligence in the wild and I will copy that I think that is actually what I'm interested in - but it's with a perspective that maybe makes it all in the wild I see my interest in AI really comes from a broader interest in what I think is a much more interesting question which I have no answers to and can barely articulate the question but it's how simple lots of simple things interacting emerge into something more complicated and then that then creates the next system out of which that happens and so on and so like it's the phenomenon for instance chemicals organizing themselves into life or single cellular things organizing themselves into multicellular organism or individual people organizing themselves into society with language and things like that and I suspect that you know there's more of that organization to happen I think AI is AI that I'm interested in is is a higher is a higher level of that and I suspect that like like George I suspected not only it will happen but it probably already is happening and we're going to have a lot of trouble perceiving it as it happens and and and partly I think we have trouble perceiving it because the notion that end so beautifully described if you know this sort of the Golem or the that's such a compelling idea that I think we get distracted by it and we imagine it to be like that and that kind of blinds us to being able to see it as it as it really is emerging not I think such things are impossible but I just don't think those are gonna be the first the first to emerge and you know that there's a pattern in all of those emergencies which is really interesting which is that they they do start out as analog systems of interaction and then somehow like the chemicals you know have chains of circular pathways that you know metabolize stuff from the outside world and turn it into this circular path ways that are metabolizing you and then what happens always is is the going up to the next level is somehow those sort of analog systems invent a digital system like DNA where they sort of abstract out the information processing so they put the information processing kind of in a separate system at its own and then from then on the interesting story becomes the story and the information processing rather than the story and in some sense the information processing system starts to drag along the the the complexity happens more in the information processing system and so that certainly happens again with multicellular organisms the information processing system is neurons and they put you know they eventually go from just you know a bunch of cells to having this special information processing system and then that's where the action is in the brains and behavior and you know that allows it drags along makes much more complicated bodies much more interesting once you have behavior and and and of course it makes you know humans much more interesting when they emit language and can start talking to know but that's a way of externalizing the informations processing in fact you know as we build communications like writing first and you know that's sort of our form of DNA for culture in some sense it's a this digital form that we invent for encoding knowledge and then you know we start building machinery to do information processing as systems and you know everything from legal systems to communication systems and computers and things like that so I sort of see that as a repeat pattern I wish I could you know say that more precisely and but I I think you all know how I'm talking about when I wave my hands in that direction I'm I think it'd be you know somebody will someday make wonderful progress and finding a way of talking about that that more precisely but if you apply that to artificial intelligence and look at it let me take your bringing off point of things beginning I think there's a worry that the singularity people are is this somehow the artificial intelligence will develop welcome super powerful and develop goals of his own that aren't the same as ours and so one thing that I'd like to convince you of is is that I believe that it's actually starting started to happen already and that we do have intelligences that are super powerful in some in some senses not in every way in some dimensions much more powerful than we are and other dimensions much weaker but but the interesting thing about them is I think they are already developing emerging goals of their own that are not well aligned necessarily with our goals with the goals of the people that created them with the Gulf with people that they influenced with the goals of the people that feed them and sustain them and so on you'll goals the people that own them and I think that those early intelligences they're probably not conscious so the examples are in in may you know it may be that there's one in working inside Google or something I can't I can't perceive that but that but Google is one so corporations are examples nation-states are examples and so corporations are their hybrid Intelli their artificial actually artificial bodies that's what the word means right they're they're their artificial entities that are constructed to serve us but in fact what happens is that they don't really you don't really end up serving exactly the founders they don't really serve the shareholders they don't really serve it's not really the employees that they serve it's not really their customers that they serve they kind of have a life of their own in fact none of those entities that are the constituents actually have control over them and I think that that's a very fundamental reason why they don't I think it's you know Ashby's law of requisite variety that you know he basically says you know in order to control something you have to be sort of have to have as many states is the thing you're controlling and and therefore the super the super complicated super intelligences by definition are not controllable by by individuals and certainly you know you might imagine that you know the head of yo Google actually has you know gets to decided what Google does especially since there's a founder of Google but you know if you talk to Larry and Sergey you'll realize you know they're pretty out of control of the whole thing yeah and they're not unique in that respect I mean when you really talk to the heads of state or things like that mostly they feel powerless and even if they're in the positions of great power they don't feel as powerful as they are imagined so yeah they don't feel in control they certainly don't feel in control of of what's going on and and they constantly expressed frustration that people imagine that I can solve this problem I see this problem but I can't I can't solve it even though I nominally yeah yeah so that they're not power lot purely powerless that was it that was an overstatement yeah you know and of course you know as shareholders try to influence and do influence corporations but you know they have limited influence so people inside companies try to influence them and so on but one of the interesting things about the the emergence of them having goals of their own the emergent goals as the emerging goals often tend to successfully thwart to see those influences as sources of noise or something like that so for example before Information Technology one corporations couldn't get very big because they just couldn't hold together so you couldn't have sort of world spanning corporations that as effectively I'm sorry so I would say that East India Company did not as effectively hold together as an entity and stay coordinated I mean they're they can be big but I don't think that they were as tightly coupled information technology you know I think certainly it made it certainly made it much easier I won't quibble with you whether they were edge cases but certainly yeah certainly you know having the Telegraph of the telephone and things like that you could have skyscrapers full of people that did nothing but hold the corporation together by calling up other people in the corporation and and so these these things are hybrids of technology and people but one of the things that they do is is you know before the technology could enforce you know to as they shift it over to have more and more the decisions being done by the technology one thing they can do is prevent the people from from breaking the rules so it used to be that you know an individual employee could just decide not to apply the policy the company policy or something like that because it didn't make sense or it wasn't kind or something like that then that's getting harder and harder to do because more and more the machines actually had the policy coded into it and they literally can't solve your problem even if they want to on so I think that we are we've actually got to the point where we do have these super powerful things that do have big influences on our lives and let me an example of sort of emergent behavior and we just had so and and they're interacting with each other so the stories that are happening like let's say the Facebook is a great example there's an emergent property of Facebook that it started it basically enabling conspiracy theory groups now it wasn't that you know Zuckerberg decided to do that or anybody at Facebook decided to do that but just kind of emerged out of you know what their business model was and so on and then that actually had an impact on you know this sort of other emergent thing which with the government which was designed for dealing with people not corporations but in fact corporations have learned to hack it and and they've learned that they can use their super human you know ability should track details to things like Lobby and track details of bills going through Congress in a ways that no individuals can and so they can influence government in ways that individuals can't so they're more and more the government is responding actually to the pressures of corporations more successfully than the pressures of people because there's sort of superhuman in their ability to do that even though they may be very dumb and in some otherwise well but that's a that is an example though of their success which is that one of their one of their successes is is their their ability to gather resources to get food from the outside world and so they're they have been extremely successful at gathering resources to themselves which gives them more power so there's a positive feedback loop there but you know lets them invest in quantum computers and aoi and so on which gets them presumably richer and better and and so I think we may be already in a world where we have this kind of a runaway situation which is not necessarily aligned with our individual human goals and I think you know people are perceiving it in they're perceiving aspects of it but I don't think it's widely perceived what's happening is that what's happening is that we have these emergent intelligences so so for me I think when I hear people talk do kind of the hypothetical Henry any of these super intelligent eyes that are yeah I'm going to take over the world I'm like well that might happen sometime in the future but we actually kind of have a real example now and we actually we if we could just figure out how to control those rather than thinking hypothetically how we ought to you know design the five laws of robotics into these hypothetical general AI sort of human like things let's let's actually think you know how can we design the five laws of robotics or computers into corporations or something like that we ought to be able to do that ought to be an easier job if we if we could actually do that we ought to be able to apply that right now yeah or something like that but I think it's I think this is not just a metaphor I think this is actually a serious literal problem the Supreme Court recently said they have the right of free speech which means just because he was giving a talk at nips who's a historian at Cambridge that's made exactly exactly this argument exactly about corporations and nation-states but he's been during which I think it's quite convincing that this is from the origin of corporations and nation-states that it's it's from industrialization that that's when you start getting these these agents and then there's some questions you could ask about whether you had analogous you had analogous super individual agents early so maybe you know just having a forager group is already having a forager community is already having a superintelligence compared to the individual it's fairly clear that that kind of increased social complexity is very very deeply related to some of the things that we more typically think of as being as main intelligences so we actually have a historical example of those things appearing and those things changing the way that human beings function in important and significant ways and you know for what it's worth at the same time the the data is that individual human goals got much better on average right I mean you could certainly argue that there were things that happened with industrialization well they stopped having accidents they stopped being struck by lightning you know someone like crosslink has these has these long lists that are like that so so we do have a historical example of these superhuman intelligence is happening and they could it could have been what people thought at the time was that the effect of that was going to be that that individual goals would be frustrated and you know if you were if you were trying to graze your sheep on the Commons then the those individuals individual goals you weren't better off as a result but you know it certainly isn't it certainly doesn't seem like there's any principle that says that what would happen is that the power balance and so certainly humans aren't powerless to influence those goals and I think one of the things though that we may be toward tipping the balance because a lot of technological things I think have helped enable the power of these very large corporations to coordinate and act and gather resources to themselves more than they've enable the power of individuals to influence them [Music] books things that were pretty against I mean I would say the East Union company did some pre inhuman really just in common he said he viewed the Constitution is a program written for a distributed computer it is a really interesting comment that if you take what you're saying seriously to think about through what is the programming language are a huge part of the problem you know we're puted echo chamber okay magnify the significance piercing news which is a kind of the schizophrenic model this hyper connectivity right good everything connects to this conspiracy theoretical model so the monthly as I learned from Wendy Chung is at its core of the programming language like baguettes like as distinguished from the parallel study in the 50s of you know birds of a feather don't flock together the difference attracts right so these were two models in the 50s that were at the core of this game theoretical you know algorithmic thinking and everyone went with white that gets like which produces the echo chamber so the first question is about hybridity so the DNA model has been radically by translocation right so it's not the case that they're perfect ones right you mentioned nine out of ten you call it but there's the 1/10 note which has information from the chimeric gene that I have floating around me from you know my son whatever right there's translocation going on all the time so there's not this in other words do we have a resource there in this ongoing hybridization of the program do we have a resource of a point of inflection and to to Bob's rights comment we also are giving rights not we but the Bolivian Constitution is giving rights to the ocean to a tree to cetaceans right so can this dialogue with others with other life-forms with other sentences you know somehow break the kind of horrifying picture of the corporate super intelligences are there other trans locatable you know informational streams that can kind of be magnified or the algorithms be switched to proliferate differences and dialogue external influences rather than the continuous proliferation of the self-same so firstly we're gonna say I don't think it's necessarily horrifying because I don't think we have no influence over this so I agreed that this has been going on for a long moment being put in place algorithms the question is I think the trend is very much in the direction of the next level of organization which is corporations nations say some things like yeah actually taking advantage of these effects like right symbiosis or acquisition of genetic material which is done by acquisition and you know they have lots of ways of taking advantage of hybridization that is actually better than individual so in fact I think the technology has has kind of hurt the individual interactions as you point out but the way that it's played out in many ways Hartmann it has helped it in some ways it's been a mixed bag but I think it's definitely enabled the corporations because the corporations before were limited just by logistics of scale so they became more and more inefficient and except in very special cases they couldn't hold the other as they got bigger technology has given them the power to hold together and act effectively bigger bigger which is now you know we've just gotten in the last year the first twenty into our company because they were designed from the beginning to take good advantage of do you think do you think that there's a characteristic difference between the kind of research that goes on in the corporate under the corporate umbrella then university umbrella say I mean is there I mean I can I know people have lots of views about this and there are things you can do in university you can't do it the University and in one of the other but how would you characterize in particular areas of I think corporations are much more rationally self-interested how they focus their research you mean there are resources more efficiently so they're choosing better effective at promoting promising research there's a lot research areas that increase their that our alignment with their emergence cream very open intellectual delivers as a way of attracting steerable to what they do so Google and Facebook both extreme freedom they have I think that's a great example with them being very smart channeling the energy toward their emerging also Steven knows I'm sure says look at the emergent goals and corporations and so on I mean it's difficult to map how the goals and humans have evolved over the years but I'm curious whether you can say anything about what you think the the trend of emergent goals and corporations is that is what what you know beyond kind of you know if you talk about human goals you can say something about how human goals evolved over the last few thousand years I think I mean some goals have remained the same some goals have changed you know the trade as well there's a lot of corporations in the same niche they're competitive whether get to be one well when it get to be to they often become uncompetitive so they instead of trying they might trouble if one is substantially bigger might try to destroy or gobble up the other one but otherwise it might try to cooperate with the other one against the interest of the consumer let's call it yeah I trade and that's what when they get as they get bigger they also want to control their broader environment like regulations they have a small company is not good so a restaurants not going to try to control the regulation of restaurants but if you have a huge chain then you can try to control your the governmental context in which you are you could also try to control the consumer side of it - I've been advertising as a simple way to do that so as the corporation's get bigger I think there's an unfortunate tendency that they the industrial competition goes down and we see this in big in high tech it's really very extreme I mean there's there's only you know five huge ones and they're doing different things manufacturing a factory and that's likely to continue not just in a high-tech areas but in others and I think it's very worse than that the corporation will get more and more resources to shape their own environment and then are--and their goals are not just I mean at the lower level and you know the restauranteur say you you have two goals one is to make money for your owners and the other is to survive but when you get much bigger it seems to me that often the goals are to protrude beyond those two or can also control as much of your environment as you can the purpose of stability for further book but both for both and I think and and I think it and there's another trend that's correlated with this which is the concentration of capital at the individual level you see a higher and higher proportion of the wealth of a country it's in the top 1% maybe I think it's a symptom of the returns on capital greater than the growth of productivity which doesn't depend so much so the corporation the corporation's are likely to have more control over resources and I think that's unfortunate corporations do you think the corporation's will emerge with with the same kinds of virtue and vice humans well you got I'd say that Volkswagens cheating is very much the world we inherited from the 1940s this brought the first basic conferences the huge competition was faster computers to to break the code within 24 hours to speed what thinking machines is trying to get more instructions per second and but there's another side to me there's slow computers actually in the end survival of species and that's like where the immune system is so good because it's a very long term memory we need that too we don't just need the speed you're talking about the scale not only in size and distance but in time and that's a good thing because corporations or it can be a bad thing - you can have a dictator the last four thousand years but it's that even when they get bigger they seem to have they seem to have a very predictable they seem to have this very pretty stable life right that's what people like Jeff was like Jaisal said but anyway it's very important possibly good function of these systems we're gonna get a longer term you know longer term computing where you look at look at the very long time series in that I mean one question is you know historically we have places like AT&T IBM Xerox it had really world-class labs that over time deteriorated I mean AT&T Laboratories is nothing remotely like what it was like in the nineteen sixties and fifties and they expelled a lot of research eventually because it wasn't shooting up short term enough for them and they figured they offload that to the universities get and then take the fruits of it and do things that were more short-term so one possible outcome is that even the places where they're hiring people at a high level of giving a tranche of the research relative freedom as a sort of cover and attractor you might not that might not even that one outcome is said that could expand but it could also pull back and you could end up with you know pulling you know wrecking parts at the university and not having a lot of freedom in the corporation I don't know I just think that I mean it seems to me an open question what's gonna happen with this concentration of research at a few companies and it's worth your while to do basic research because whatever is paid out you see that the US government did you your Frank Stroup comment that eighteen T was required by the government so they spend well so much on trees you know will never regularly [Music] you know it's worth our while to do basic research and things where we are the only distribution channels basically and the same thing is happening with with a bunch of AI stuff that's being done in the in places where the only beneficiary is but I think there are cases where you can tie this very directly to a I so the best example of this think is the Facebook feed management algorithm so that's a situation so Nick Bostrom has this thought experiment about you know machines that you know you make an AI where the goal of the AI is to manufacture paperclip so that it consumes the entire manufacturing paper clips Tristan Harrison pointed out that the Facebook feed management algorithm is essentially that machine but for human attention where consume an attention makes money as a consequence of doing so that's fed back into the mechanism so you make you man attention it gets better and better at consuming human attention until we've anybody who has teenage children knows that there's a attention attention deficit I mean I don't I think that I think that idea is is highly exaggerated and and let me give you the reason why I think that if you think about walking down a street or driving down the street and having billboards are not if you were in a first generation literate culture what you would say is there's this terrible problem as you go down the street every second what you're doing is having your attention distracted by having to decode what this is right there's all these symbols and you sit there and you decode the symbols and you're not actually paying attention to anything that's going on this trait for attention is terribly divided and we know even straw you mean even neurologically that what actually happens is when you are deeply immersed in illiterate culture you end up with strip effects where you're decoding of your decoding a print is it attention demanding in the same way if you're not doing it by serial attention anymore in fact you're doing it completely automatically and in parallel so that I mean not it's just not it's something that we all worry about because we're in the position of the of the preliterate we're in the position of the pre literate person it's not at all obvious that this is somehow an intrinsic character right now if you look at where artificial intelligence is being deployed on a large scale people are spending a lot of money paying the power bills for you know doing the computation things like that they are mostly being done in the surface of these kind of either either corporations or nation states mostly corporations but patients are rapidly catching up on that and they are making those more powerful and more effective so I think that when we think of these runaway eyes I think we should think of them as not things off on themselves they're actually that the brains of these runaway things that are already these hybrid eyes and so the artificial brains of the artificial nervous systems of these things that are already hybrid AIS and already have emerging and wait and so did this is to say why I disagree with you about this I mean all right because these days you know back in the nineteen citizens they're watching TV five hours a day might have no this Tarble I know I enjoyed preparing for the grumpy old man changed my life and I like practice event is to get people's attention to webpages attention or dollars or the designers will tell you that they're using the lowest brainstem functions anything valuable that is valuable to them they will use this power to good and you know there will be you know there'll be problems with that and there'll be limits on that and so on and you're pointing out some of the limits and getting attention and there'll be limits in their ability to get money and there'll be limits in their ability to get electric power and you know and so on but they will use all of these tools to give as much of it as they can my challenge would be is that any different than it was for design which would in 1780 yes I mean you could argue that was much more of a tip in power if you considering you know 1710 you consider the difference between being around in 1730 and 1850 because for example for the East India Company they couldn't establish a policy and monitor that everybody did that policy I mean that's exactly what people like were to do that was the whole part of the whole point of inventing industry inventing factories was exactly I had to translate itself to a language of the army which was the British Empire yeah so there are measures between corporations and governments that we really have to worry about like the one we have right now so you don't have to worry about them or there isn't power what I'm saying is we have a historical the question is why is it that you think that this is a tipping point all right so you know it looks like there's this general phenomenon which is that you've developed these trans individual super intelligences and they have certain kinds of properties and they tend to have power and their goals that are separate oh that's true but we have a lot of historical evidence and it might be that what's happening is that that there's more of that than there was before but why do you think that somehow it's a tipping point that this is a point at which this is gonna be different from it could be a tipping point I'm not sure exactly now what I am saying is that there's a rapid increase there's an explosion of their intelligence so there's a so these explosive technologies which are driven by Moore's law and things like that are being used to their advantage not necessarily there very few examples where they're being used to individual advantage there are lots of examples where they're being used to the advantage of these of these hybrid emerging that's that's a very good example because of between 1730 in 1850 the the life expectancy and degree of nutrition and height of the average person in England declined because they would be taken off of the crap off of the art of you know the countryside and locked into factories for 90 hours a week this is exactly there's historians there was historians here that's why I think thinking about these historical examples is actually is actually helpful so if you think about the dip the scaling difference between let's say pre telegraphic train right so if you think about the difference in scale between the communication that you could have before you have the Telegraph and afterwards and before you had the Train and afterwards for all of human history right the fastest you could go was the faster the speed the fastest communication you could have is this being the passive learners and then suddenly you're have communication at the speed of light it seems to me there's nothing that I can see in what is happening at the moment I think that is now so so I'm saying this is happening now I'm including railroads and telegraph I mean this moment in history includes all of that so that's the thing that's happening right now so that's industrialization that's essentially industrialization that's essentially I think realization focuses on the wrong aspect things happen at once and you categorize them it's the particular thing that I think is interesting which happened at the same time as industrialization was the the construction of an apparatus of communication of symbols and policies and things like that that was outside the capacity of a human mind to follow I think that's the interesting thing not the construction of that not there many other aspects of industrialization but I think that's the thing that's happening now and I think computers and AI things like that or just that going up and that's what I mean I think seeing this moment of increased poverty and stagnation of wages for a big sector of society an enormous increase of wealth within a concentrated group and the consolidation of the industries like Amazon is something that's that's that's that is that does represent the sharp edge of that increase you know it's not just a simple linear continuation of what went before I mean in the post-war era post-world war ii era you know there was a sense that people were able in families to go to college for the first time to get loans at least if they were white and that meant you had a big class that had increased expectations and increased income and that's you know we're seeing that echoes of what happens when that stops when you're basically not bringing new people into the college system you're not giving them increased stakes and homes and real estate and things that increase in value and one argument it's often made by the technology companies is we're not doing anything different right you know this is something that's being done in the past and we're just doing it better but I think there actually is a case that you can make that doing it better is different it's that you know the objective function is the same but you're doing a better job of optimizing it and one consequence of that is that you get all of the unforeseen consequences of doing a good job of optimizing that objective which may not have been here when you were doing a bad job of optimizing so you know machine learning we talk about regularization right regularization it's forces that pull you back from overfitting on your objective and you can think about you know not being able to do a great job of optimizing as a form of regularization is helping us to avoid all of the negative consequences of really optimizing the objective functions that those those companies have defined for themselves so they say we're doing the same thing but they also say we like to break stuff and breaking stuff up it means breaking the social it's not doing the same thing better is the thing that then reveals why it's bad to do that thing right yeah you move everybody into half the people in the geek economy and take away their benefits but let's just fine so why should they well and since is let's say [Music] I mean the training the Telegraph is the inflection point rather than you know the last ten years is the inflection point I think it's pretty hard to make quickly it provides everything you know people are off of off of the Commons and and there's pollution and tuberculosis and all the rest I think the data are that if you think of the train and telegraph is an inflection point that the individual achievement of goals got well I mean didn't just get better but God exponentially better I'm not saying there's an inflection point I think that we're going through a transition and so we're in the middle of a transition from going from one lovable organization what if we are sounding less questions but but for instance individual cells had to give up the ability to reproduce they had I think we will lose some things I think will loosen things that process will gain some things in that process but really it is we don't have to - I think we're spending too much time worrying about the hypothetical I mean the description of the state of things but then to ask what is to be done and what aspect of humanizing let's call them robots they I would have been like would be to tax them as humans especially when they replace workers in factories or accountants or white-collar jobs in recognition professions then we would all have a stake the corporations are now politically powerful enough to keep their tax rates level and not only that but the billionaire's are powerful enough their tax rates low for example inheritance tax the point at which perhaps in 50 years time vast sections of the population I need to be working 10 or 15 hours a week and we might have to learn from aristocracies how to use less in other words it's perfectly possible in anyone who speaks of retirement we were talking about this great how busy you can be doing nothing distributing wealth and function [Music] but this is we've lost them and they now have more power than individuals do influence in the political system so there's an example of you know where the train has left the station we're now in a post individual human world we're now in a world that is controlled by these emerging goals of the corporations and I don't think there's any I think we are now in that world

Channel: Seung-Tai. Park

Views: 739

Rating: 5 out of 5

Keywords:

Id: 4Bm6A_BxMgU

Length: 47min 51sec (2871 seconds)

Published: Wed Sep 04 2019