Jaron Lanier | Gray Area Festival 2019 Keynote

very weirdly once upon a time I made up the terms virtual reality and mixed reality and I had the first company and I did a lot of other stuff we did the first headsets commercially also made the first color when it all did a bunch of the first apps we did the first car prototyping app my favorite was the first surgical simulator later on some friends and I had a company call in the 90s called I'm addict where we pioneered the kind of stuff you see on like snapchat where you can add stuff to people's faces or this ridiculous aging app that's going around we pioneered this sort of face face related software that's sold to Google the old VR company ended up with Oracle weirdly they don't seem to be doing anything with it but I guess right now it's so long ago anyway so I've been doing this a long time but I want to talk to you about most is this sort of cultural framework so you might even call them mythologies that we create to think about technology and there are two primary ones in my understanding that have somewhat competed with each other and I realize this isn't absolutely true and it will be untrue for some people so this is a very approximate sort of a pattern to notice it's true for me anyway the two the two mythologies or paradigms or contexts or frames or whatever you like are artificial intelligence and virtual reality both of which get renamed periodically so when we speak of AI we could be calling we could be speaking of deep learning or machine learning and Bob Loblaw there's all these variations and similarly with virtual reality where there's mixed reality and more recently all the different actually this whole thing about terminology goes way back there used to be actually quite passionate Wars about what things should be called in the 80s it's funny dream for many terms he won't even have heard probably now I I think these two compete in the sense that they suggest almost polar opposite approaches to the relationship between people and technology neither of them is really about any particular technology or any particular scientific question for instance the face related software which was the first to be able to track features and transform a face we also won the early Myst competitions for even finding and and recognizing faces and by the way we had the we discovered that our face didn't our face finder didn't recognize black people in the 90s we'd that problem is old and shouldn't be a surprise anymore oh my god all these things that happened but it anyway that wasn't a I at that time at that time nobody thought of that as AI that was part of VR and it kind of flipped over to the other camp and now face stuff is considered AI now the idea of AI can be found going very very far back in the literature of humanity if you care to look for antecedents there's a wonderful thing in Plato about how someday there'll be robots to replace the annoying musicians and the Agora but when we think of the AI cultural framework most of what we think of was really formulated by just a small group of people in the 40s 50s and the 60s we think of Alan Turing and the Turing test of course but really the person who most authored the the sort of cultural world that defines this was Marvin Minsky now Marvin was a mentor of mine and in fact he was my most important mentor and he was infinitely generous and infinitely sweet my debt to him is indescribable and we used to love to argue about the top I'm presenting to you now where I always thought from when I was a teenager in the 70s that there was something screwy about the AI narrative and I was much more the Softee humanist who believed in consciousness as this thing apart and didn't believe it's all a giant computer and blah blah blah and Marvin loved to argue with me whenever whenever I talk about this stuff I always think of Marvin the last time I saw him just before he passed away when if his of one of his students called me said listen Marvin's very frail when you go there don't engage when some big argument and so I showed up at the house and he was like oh you're here I really want to argue I'm so sick of people who agree with me so we had this but to some degree I hope this doesn't sound boastful but I'm probably the principal author of the cultural framework that we call virtual reality now and indeed I called it virtual reality and it was something that I and I don't mean to say that I'm the only one there many other extremely like just in terms of this cultural presentation of it there plenty of other people but the the the way it was present in my mind and the way I talked about it in the 80s which was the first wave of VR mania was somewhat as an anti powder and a reaction to artificial intelligence an alternate framework so I want to talk a little bit about comparing the two I guess the first thing I want to make clear is both are based on illusions and it's just that their illusions presented in a different way when a professional magician a legitimate magician if you like presents an illusion they have a proscenium arch around it and they say here's my trick and that's quite ethical because if they don't outline where the trick is they might be deceptive they might be charlatans they might be crooks and so that the proscenium arch has a very particular ethical role in magic perhaps more so than anything else that happens on a stage to me virtual reality is like that vert reality is saying here's the illusion enjoy the illusion it's it's it announced as the illusion it's ethical an honest about that now in my view AI is also an illusion or at least it's legitimately perceivable as an illusion it gets a little subtle so let me explain this my perspective which I used to argue with Marvin about in the old days is that information this thing that we can encode in bits or qubits lightly is alienated experience so what that means is that when you perceive something from bits if you don't have a cultural framework in which those bits are meaningful then there is no meaning it's dependent on the person who the bits are for if you I'm you know just their various ways to see this it might not be immediately clear but for instance the thought experience experiment alien a Martian or somebody comes down and - such a Martian who's never seen written language maybe they come from another dimension they're not used to 3-dimensional perception maybe they're not used to two dimensional perception but to them they see this thing which is your smartphone or for that matter your your VR headset and to them it's a lava lamp it's a thing that creates heat as it makes patterns and they might enjoy it but they might miss everything else so the idea of this is very much as you can store potential energy for instance I can lift something that that is heavy and then if I drop it the potential energy that I stored by lifting it is released right and so in the same way when i encode information into bit in a program or an example set for a machine learning algorithm whatever it is I think I am entering in information and it's only decoded when somebody can perceive it now the interesting thing about this way of looking at information is it gives people a special role it says that it's the person who's kind of this mysterious end of the line that actually has the meaning right now a lot of people don't like that they're like oh no don't put people on a pedestal so this is an argument I've been having with all kinds of people for many many years not just Marvin lots and lots of others now the thing is this question about whether people are special or not whether with their consciousness is real whether people have some kind of special ability that is different from special status that's different from computers it is complicated and it's bound up in a lot of it has a lot of baggage it has more baggage than anything oh my god how can I start one thing is that there are a lot of people who've spent their lives battling against people who are mired in religious superstitions that often cause them to be ridiculous that's not to criticize all people who believe in religious ideas but those who oh I don't know believe that it's immoral to be gay or that evolution can't be right or vaccines must be bad or bla bla bla bla bla just like and so you talk to somebody like Richard Dawkins as a great example who's been battling religious people all their lives and to them to suggest anything special about people is to suggest an alliance with those who are superstitious those who want to use beliefs in order to manipulate others and they are often correct that can happen but the thing is this business about whether people are special is a kind of a figure-ground inversion opportunity if you look at people interacting with computers you have the freedom to perceive that as a meeting of equals or as a meeting of those who are different those who have the special thing and machines that don't this business of where you draw the line isn't science it's an example of something that's really pure philosophy or pure belief from a strict truth test point of view all of the places you can draw the line are equally valid I used to propose a way of thinking about this which is called the circle of empathy and here's how it works you draw a circle around yourself and you say those things within the circle deserve empathy and those things that are outside of the circle do not so for instance perhaps rocks are outside of the circle perhaps there might be rocket Aryans or something who would disagree and hold rocks sacred and and it might be only certain rocks maybe there's certain sacred rocks but maybe other rocks aren't sometimes the circle is hard to place there plenty of people who would not include all humans within the circle racists do not include all humans within the circle homophobes and xenophobes do not include all humans inside the circle and so on people come to blows over where the circle should be there are those who wish to place the fetus in the circle there are those who wish to place animals in the circle and others who do not and then the this debate about AI kind of amounts to whether computers go inside the circle or not or at least some hypothetical computer now the thing about the circle is it's ultimately a moral choice it's not scientifically a real thing there's no circle position meter there's no meter that measures consciousness as taurine pointed out at the dawn of all this in the Turing test there's no absolute test for something like humaneness or consciousness or any of these things it's subjective it's all up to people perceiving one way or another and so the Turing test is based on a judge not on and in these days when when people are trying to judge whether they're deep learning algorithm is good or not they compare it to people performing the same tasks it's all based on comparisons not any kind of absolute standard you could say that about anything in science except it's not the same because other things in science are based on ultimately on universal constants this is not this is cultural now the advantages of making your circle as big as possible is you're less likely to be an by mistake the advantage of making your circle smaller is you're you're more likely to avoid certain kinds of incompetence so for instance if you decided to make your circle large to include bacteria and you say I hold bacteria sacred I will no longer brush my teeth you end up at a point where you're not helping anybody not even bacteria and so there's a there's a way that making a circle too large creates incompetence or they're things that can paint manipulation for instance if you pretend to speak for something that can't speak for itself you might actually be overriding people who can speak for themselves which is what happens when the advocate for the fetus overrides the woman who's pregnant and so you can actually become abusive without even realizing it so this business is very it's a kind of a competency morality puzzle and I have sympathy for those who have difficulty figuring out where to place the circle and I don't I don't believe that any of us have our circles precisely correct I don't even think that's possible but I think we can get pretty good at placing our circle now in the case of the AI idea the circle is expanded to include computers but I think it's important to remember how this notion first came about and instead of talking about Plato and his suspicion of musicians and or many other writers before Turing let's talk about touring as one starting point when I was taught computer science we rose told that this one of our founders our principal founder Alan Turing had come up with this thought experiment and it was foundational very much like Einsteins thought experiments about bullets and elevators and all of that were foundational to relativity and we were never told anything at all about journals life which is now better known there's been a movie and whatnot but I was absolutely gobsmacked as they say when I discovered the circumstances under which Turing came up with the Turing test it was very close to the end of his life it was in the final weeks Turing story for those of you who don't know was that he was obviously one of the very first computer scientists he and von Neumann defined the abstract model we used for classical computation he was one of the first programmers he had that he was one of the first people to apply computers to anything and he's most celebrated for using an early computer to break the Nazi secret code that had been considered unbreakable and giving Britain access to secret Nazi communications is thought by many to have saved Britain from an invasion it was obviously bombed but it wasn't invaded it was better after the war he was celebrated as one of the great heroes of World War two but there was one problem which is that he was gay at a time when it was illegal to be gay in Britain and so he was forced to endure a bizarre treatment that was supposed to I was like their version of converge therapy was supposed to cure him of his homosexuality but they used chemicals now for us for my present-day perspective it's almost impossible to put ourselves in the mindset to understand what these people were thinking because it's just so macabre and weird but remember in those days the computer was not yet the model for the mind there was no this was before there was this idea that computers and minds are similar so the thing that was the model for the mind was the steam engine that was the metaphor and steam engines are all about pressure building up releasing pressure if there's too much pressure and it's all this pressure stuff and you can see that metaphor at work in Freud's work for instance it's all about too much libido and things building and all that and so the idea was that if you're gay it must be that you're oversexed so the way to correct it is to inject you or force you to force you to receive female hormones to balance you and so touring against his will was forced to receive these and he started to develop the physical characteristics of a female body which he did not want he became terribly depressed and so far as we can tell he committed suicide by eating an apple laced with cyanide by his computer so the first computer was part of a suicide ritual for its inventor and yeah so so what's going on here that the Turing test is something that he conceived of in the last weeks when he was undoubtedly starting to plan his suicide because it was a suicide that took planning he proposed a thought experiment in which an old Victorian parlor game in which a man and a woman each are shrouded behind screens and can send little messages similar to today's tweets I suppose to a judge and they're both trying to fool the judge potentially in the judge is supposed to tell which is the man and which is the woman and Turing said let's get rid of the woman get that woman out of there we'll put a computer in her place and they have a computer and a man each passing notes to judge and the judge is asked to choose and when Turing says is if the judge cannot distinguish the two shouldn't the computer be given some sort of recognition some sort of status some sort of rights some sort of empathy something know outside of the context you might look at this and you think well this is a nerd argument for computer machine for computer human equivalents or something a challenge of the bar you'd have to meet to declare computers as not being the same as people and of course we can always claim the Turing test has been beat it has been in a multitude of little ways so we can get rid of that distinction but I think in context you see something totally different in context you see a moral challenge to the society that was killing him I think what he's saying is I have devoted my life to fighting the Nazis the very definition of a Nazi is someone who kills others because of their identity and now you're doing it to me and I think perhaps he imagined a more pristine existence in the computer that didn't have all these issues of identity maybe he imagined a future in which other people are also judged not to be fully human maybe but you know what when you read him it's funny I've never found even the slightest touch of anger or revenge or resentment in Turing it's it's almost incredible like he was gentle and he has an amazing line in a footnote to the first statement of the Turing to history he says that you know what even if we consider computers to be like people they'll still ultimately have been made by God and it shouldn't change any of ideas our ideas about spirituality which is a remarkable thing to say right at the start isn't it because people who are facing death naturally wonder about Souls they naturally wonder about survival and that makes this whole thing even more mucked up and the problem with the AI mythology is that it has started to become abusive of people's fears in the same way that medieval religions were or unfortunately many contemporary ones where person is afraid of dying and then the high priest says oh I get your consciousness to survive I can back you up to the cloud and the person says okay I will fund you and of course this is all once again a magician who hasn't acknowledged the proscenium arch they don't know none of us know now meanwhile the AI I mean the virtual reality a virtual reality idea was was conceived to demonstrate through example that there's another way to think about computation something different and I want to try to give you a little bit of a sense of why I believe and I want to emphasize what's VR what isn't VR is not absolute I mean all kinds of things are called VR that build seems so to me and whatever you know and it's a it's a way of thinking more than any there's no precise delineation of which hardware which software is is VR or isn't or is a mixed reality or whatever I mean might by the way this thing lists me is Microsoft when I speak I don't speak for Microsoft believe me I say things that horrify them all the time so it should be under there but if I was having my Microsoft hat on I'd point out we saw these things called mixed reality headsets that are actually classical occlusive headsets you might ask why none of us know we don't know why we did that but anyway this is just to say that these definitions are not absolute so [Music] the when you interact with a computer that's been designed by somebody who believes an AI the computer will pretend to be a person there'll be a thing how can I help you it'll it'll pretend it'll pretend to be a serial Cortana whatever and then this happens enough and you start to think you're kind of just one little cog and the giant cloud brain one little one little cell it starts to hypnotize you into believing it and indeed the way the computer interacts with you is the way you interact with it and it creates the illusion for a moment that this whole thing is just information exchange in a sense it doesn't matter if you have internal experience you can still use your smartphone it still kind of would be sensible I suppose it's a funny thing about internal experience or consciousness or interior a or awareness or just the existence of experience whatever term you propose we can experience it ourselves but as far as others we only we only know their consciousness by faith right you have faith that your lover or perhaps your dog or your child is actually conscious in there I once wrote a comic paper in the 90s in an academic journal suggesting that the only people for whom one could tell if they were truly conscious or not were professional philosophers where their philosophy would reflect whether they experience consciousness and this sort of daniel dennett heart AI kind of philosophers were just dead inside they just didn't have it to which and and I used to have fun I used to do public events with people like Richard Dawkins and Daniel Dennett and and Marvin for that matter and they didn't end it with say oh yes absolutely I have no consciousness inside that's correct and that's a great you know this actually works it actually is this total consistency we agree we're happy I'm conscious you're not great let's eat but VR is a different sort of thing though so here's so let's say you put on a classical occlusive headset and you're in a virtual world and obviously the whole point is that the world can change so maybe you're in some weird world of butterflies on an alien planet or whatever you know all the sorts of things one does and as you undoubtedly also know you can change your body's map you can turn into some different creature and still control it and feel like you embody it so maybe you turn into a weird scorpion and jellyfish thing so there's much more variable than there would be in normal everyday life which is of course the point so you varied all these things once you start learning a bit more of the science and the craft you realize that you can change the perception of the flow of time you can change the way people perceive colors you can change the way they hear you can change the way people communicate with each other you can create merged avatars where the sense of body connection becomes different than it would ever be in the physical world you could just do all these things and I'm sure there are many more tricks that are yet to be discovered but the question is as all these things are variable there's still something that doesn't change which is you're floating in there there's somebody perceiving all this and that is a remarkable thing which is easy to miss as virtual world experiences change you remain there floating that's you that's your consciousness you never notice it otherwise because usually it's just continuous with everything usually your consciousness is just part of the flow of physicality but in fact reality because the exterior world is variable that you are still there even when everything changes there you are that is you floating there that's an angel or something now this observation is not a scientific observation you can formulate arguments against it you can say it's an illusion but as I used to say consciousness is the only thing that is introduced if it's an illusion that's one definition of it do you see that huh so there you are floating and VR is the only technology that highlights that it's the only consciousness illuminating digital thing now I think there are other things that folk that also do I mean I think it's musical instruments do it and why not but you know to do a digital thing that does that I think it's it's VR that would be one of my definitions of VR anyway I have a lot of them I suppose so the thing about artificial intelligence is that it's an arbitrary way of parsing the world and so is rich reality but I'd propose that they're opposites I'm not suggesting they're the only two options perhaps but they're the two that we know about to any degree of detail and experience and culture for technology virtual reality can be thought of as a spatial illusion in the sense that your body can change the world can change your proximity to other people can change they're all things that have a little bit to do with space whether it's a visual or haptic or audio or whatever it stills things changing in space or space-time I suppose if you like AI to me is an illusion that takes place in time because what happens if you gather all this data from people mash it up and then present it as if it's a live person at that time but it's really a crowd of people from before so AI as a time illusion VR is a space solution primarily it's a rough once again one can argue especially around the edges of that one but where the the distinction really becomes important to me is economically the one of the problems with AI is because of the arbitrariness and the ambiguity of recognizing when something is an AI it's very easy for AI to become abusive now it might be with wrote reality to perhaps somebody can use virtual worlds to train people to become horrible or something I'm worried that might happen but so far the great abuser has been AI and what will happen is somebody will call an algorithm AI where when what it's really doing is its manipulating people it's really allowing the people who own the computer to screw with the other people this has a great history there's which I wish was better known the very first person to propose and design what a network experience would be like was named Ted Nelson who's still with us and lives in Sausalito and he was imagining a super creative artistic thing that would be very democratic and respectful the first person to actually implement a networked experience was a behaviors name BF Skinner have you all heard of BF Skinner so in the early 60s Skinner had the Skinner was this person who would methodically train animals and people and what was called a Skinner box and in his publicity where you control when you got reward and Punishment in order to change behavior patterns and he was this was his thing and he's one of the people who documented how teasing people how making reward and Punishment a little random and unpredictable actually makes the mind lock on to it more and is even more powerful he's one of the pioneers but one of the things he believed is that eventually there could be a digital network that would addicted and put them under behavior modification and then that would create the perfect orderly Society because he didn't like all these disorderly hippie people he would have hated Turing he was he was really into creating a conformist society and so he actually started implementing these things with the one of the pre-internet networks in the early 60s and that this was all way before ARPANET this was before those packet switching if you know what that is so there's this problem that AI because AI can be anything and because AI doesn't usually really work reliably there's always that little glitch at the end of it it's never quite right so it's not ever quite quite there so what it really turns into is a way to dress up behavior modification which is what's happened with our good friends at Google and Facebook and I love them I mean like I was mentioning the face software companies was sold to Google they're all friends of mine and stuff but I just think the business plan is so horrible but that's a whole other story but what we call AI has become almost exclusively about giant businesses tricking people it's become not exclusively but almost exclusively there's also once in a while some algorithm that's good at playing a game or something like that and those things get a lot of publicity but in the field it's all about tricking people so the thing is if you're starting mentality is that people and machines are the same you program machine's why not program people then it all seems perfectly normal and natural and why not and yet there's something terribly wrong about it it seems to me that you've fallen into a trap that's rather similar to the person who says who cares about the woman I'm gonna protect the Phoebus fetus and what you're actually doing is saying I want to control women you know that's the actual effect and we're sort of saying you know there's there's this way that you start controlling people through computation which i think is such a horrendous idea that I don't think we can ever find sanity again until we cease it which so this is a whole other talk I give about the future of internet economics but what I want to propose here is that the most crucial thing about virtual reality isn't just that it can be beautiful it isn't just that it's a window into our cognition it helps us understand ourselves it isn't just yeah it's fun it that it can be radical none of that is as important as the idea that you can't do it unless you believe people are real because otherwise what's this whole thing for you know there's a there's an intrinsic message in the VR system that the person inside the headset is the recipient of all this the reason for it that is not true when you're connected to a big cloud on your smartphone then the center is the cloud it's part of this giant you're becoming part of this giant global brain something or other some that hypothetically supposed to turn into the singularity or something you're part of this other thing but there's no reason to center all these gadgets around a person unless you believe in people do you see the difference all right so I would argue that that moral centering of the devices and the software around the person instead of the big computer is the crucial definition of our reality and anything that does that that wants to call itself virtual reality or some similar term fine by me then I think it's totally earned that that designation now one of the difficulties we've had in virtual reality is oh boy I want to talk about some things that might potentially offend some people in this room and I'm apologize if that's the case I don't know you and maybe I have it wrong so but there was as we all know this fresh boom of interest invert reality five or six years ago now when there was suddenly a lot more investment in it and a zillion startups and all these things and the problem with it is that it was overlapped very strongly with the gaming world and the gaming world was going through the gamergate style convulsions so unfortunately the particular timing and nature of the VR the most recent VR revival brought with it this ugly energy that I was repulsed by which is kind of why I sat it out I wasn't really around during the boom with oculus and all that stuff because the culture was ugly and that's not to say that the VR culture was always ugly but it was always it's kind of like gaming in itself isn't ugly but gaming culture can be really ugly the game itself might be fine but then the board's associated with the game might be horrible in that sense the art was kind of turning horrible and there were a lot of huge egos there were a lot of people who were acting out of weird fears of other people that made no sense and just all this garbage and I feel like it's cleaned out a bit at this point it's a little bit like blockchain like all the people wanted a quick buck have finally gotten bored and they're sure gone and the people who are left I think like better so I'm kind of wanting to get more involved with it again the yeah so that's my short take on it so the problem though is that because this huge period of investment where many billions of dollars flowed into virtuality rather suddenly was so focused on a certain mentality and that mentality was kind of stupid I mean it was sort of like well I'm gonna go to a game and I'm gonna be like God and everybody else sucks and all those people are stupid but I can program better it's like this whole stupid thing I just I hate it and you know what I'm talking about once again maybe I'm being unfair but I don't think so I I really I mean the obvious thing about it was the misogyny but that wasn't the whole of the problem there was just even leaving aside the misogyny the nature that of what was left had a had a shallow quality and so unfortunately we still have a legacy that was shaped by where those investments went so the investments went more into hardware than software in that period and thus we have great headsets and barely passable development environments which is just weird so and I don't I run into the danger sometimes of being like the old the elder who says my day and you kids don't get it and maybe that's true like I mean I might be falling into a fallacy like that but in the old days which for me was late 70s into the 80s it was considered obvious and necessary that the most important thing is the development experience because this was an expressive medium the whole by the way it just to let you know that the timer screen just went out so at this point I might speak forever for those who are running the house so the it was considered and now it says no signal that's good it's probably Microsoft software Tina right so all right so it was considered obvious that the whole point was for to be a communications medium the first do you know the name of the first commercial headset that was sold you might not it was called an iPhone but it was a ye phone that was the first commercial headset that was ever sold and they were actually pretty good they were too heavy but the optics were actually excellent and the screens were better than you think and they were okay there but they was called an iPhone because the whole idea was that this is a phone this is not like a solitary thing in fact the term virtual reality does I should explain where that came from the the person who invented the tracking headset where you move your head around in the world seems to stay still was ivan sutherland who's also essentially the founder and inventor of computer graphics and is also still with us he's living in Oregon now and I even used the term virtual world to describe what was out there and he in a virtual what you'd see through the headset and he did that in a paper when I was three years old so who's before I was around but he got virtual world from an art theorist from the 40s and 50s named Suzanne Langer so the term virtual world was actually coined way before the technology worked and oh yeah it is Microsoft and then Oh God that might be Apple actually I'm not sure no it's like yourself okay so no it's Apple it's an apple thing that looks like a Microsoft thing what is going on in the world anyway okay so Suzanne Langer was a really fast I mean art theorist who was thinking about many of these same ideas but not in a digital context obviously because it was too early and so when I came along my thought is what we really should be talking about is how people can communicate through these things how do they connect and so the social version of a virtual world would be a virtual reality so that's where it came from and so it was obvious that if the point was for people to communicate if the point was to connect if the point was to explore if the point was to create creative variations in the world so that you would notice that your consciousness your center of experience was still floating there no matter what changed if that was the whole point then the experience of creating was more important than the experience of what had been created and therefore not just us but everybody who was involved in early VR tried to create worlds where you would invent while you were inside where you could program while you were inside where you could model while you were inside and today we have this totally different paradigm where you can do like a little bit of object placement but it's kind of weird and you have to kinda know how to program and you have to cobble these things together and there's hardly anything you can use to really make serious stuff while you're on the inside and that's just strange it's just weird it would be I don't even know how to describe how strange this is it would be like designing a car but it wasn't worthy and you could only use it in this little closed circuit you'd have to hire these people to change the closed circuit all the time I mean it's a very weird thing to me and it still strikes me as weird and but the thing is that there's such the the gaming culture is one in which there's all this investment in a gaming studio that's they put a ton of money into making whatever it is a Halo or something which is to name a Microsoft one and and then you present it to people and that's a different thing the gaming world is not it just kind of glommed on to virtual reality as this idea at a certain point and the thing about vert reality is it does have this kind of sexiness in the way that AI does it does have this forward feeling like this is really a paradigm change or thing I should say that in both AI and NB and and in VR the supposedly radical new ideas have been the same for 30 or 40 years you know I mean it just keeps on keep on repeating the same things but they always sound new and so this evergreen feeling that there's this radical new thing is a way to raise money in fact in the old days when I used to argue with Marvin and I used to say oh but I mean you do know that to say I think is right and he'd say well you can look at it either way but it's gonna be so great for getting funding so just shut up and go along with it and then we'd go to DARPA and get funded you know and and so I get that I understand that it's true for both AI and for VR but the heart of VR has to be creating within it in a way that other people like so this is an idea that in the early days I called post symbolic communication and so what that means is that most of the forms of communication were used to are mediated by some system in which we're used to things referring to other things like for instance words refer to certain things or notes on a page refer to what you might do with an instrument and so on and this gets back you remember us talking about how information is alienated experience the information is the encoding of experience right so we can call that symbolic if we like so posts about communication would be something that would combine the fluidity of symbols with the experience of the concrete world it would be a concrete kind of fluidity so I used to have a lot of ways of talking about this and I wish I could channel one of my talks from the 80s because I used to be very profit like in sort of intense and I I don't think I can quite do that now especially after all that's happened with the internet practically destroying our world so I have a much more measured feeling about it now but the way I would talk about it is um there's a bunch of ways but here's one there's one about young childhood so when you're a little baby you don't understand the difference between imagination and reality yet or so I surmise I don't really remember it so I'm not sure but I think that's true and then there comes so the thing is if you're not quite sure what's real and what's not it's as if you're a God all the time like I guess with the gaming wienies aspire to or something and you you know like you might imagine some crazy creature and it's as if it's real sometimes imaginary friends or creatures are very real to children and then there comes this moment and I think this often has something to do with the terrible twos and the difficulty of toddlerhood where you start to get clear oh actually I'm not the Lord of the universe I'm this helpless little wet pink thing that wets itself and is absolutely dependent humans have this particularly humiliating childhood because if neoteny we're we're more helpless than almost any other species for a long time so it's this incredible demotion it's the greater you're from God to this helpless thing that wets itself it's like this gigantic fall on it I get on a cliff right and it's hard to deal with that and then a little later you learn about mortality and then it's like things really suck and like I thought I was god of the universe this is this is really hard and so the thing is that gradual you start learning that there's some tiny parts of this world that you're embedded in this physical world where you do have the freedom to create about as quickly as you do in your imagination and there are little parts of your body that you can move quickly and and with facility to your fingers but mostly your tongue in your mouth actually to your body to a degree and that becomes your definition of who you are physically it's a part you can control as fast as you think as far as fast as you feel and then you learn you can manipulate these things in order to refer to all the stuff you can't actually do so I can't I can't actually build a sudden mile-wide bespectacled octopus flying above San Francisco with its tentacles stirring the fog but I can say those words as quickly as I thought of them you know and so what it is is it extends the little part of the world you actually can control as fast as you think and feel which is mostly you know your mouth and your fingers but more generally your body in order to refer to all the things that you cannot another thing it's so so the channel by which people can understand you as fast as you think and feel is most often mediated by this kind of symbol layer like words but and not always like with music music I'm playing here it isn't except this is only for him there's no content to it I don't know how to if if I was conveying something about a giant octopus with this it would be through symbolic layer in it so with reality potentially if people could improvise with in virtual reality if you had a good enough interface that you could start to make things as you think of them I don't know how to build that interface but I do have some thoughts but if you could start to express yourself by making things in it then people would perceive something like you're dreaming but without the symbolic layer they'd actually perceive the stuff directly right so the idea is that vert reality has the object the objectivity of the world the fluidity of dreaming and so I would think of it as being like an extension of early childhood into adulthood that isn't presently possible there are many other ways to think about it but this is this to me is a truer North Star for the future of art reality than the people who are saying oh we're gonna make the illusion better and better because who cares I mean this is the thing this is an agenda that just mystifies me I absolutely understand if somebody says I want to make a virtual reality interface where surgeons can train better or where musicians can perform better or whatever if there's some human thing you want to achieve with it and you need to make it better and you need better sensors and better a better software control loop and this and that that's a sensible agenda just creating the illusion is stupid because the whole I mean perception is so imperfect anyway like what the hell are you doing it's a very weird agenda for me and in fact one of the great illusions the technologists perform to their own detriment is to pretend that they're not changing people so for instance it's amazing to me I talked to my friends at Facebook all the time and they still believe after all that's happened that they're only measuring that they're not creating that there's nothing I at regenexx going on that if you use all of these techniques to get people's attention and to steer them around you're only measuring their hard hard desires that they had from some other place you're not actually programming them which is really stupid I mean it's like laughable in the same way trying to make this super realistic VR will just change the way people perceive it's because it's such an abstract goal it's not to any purpose so it doesn't have a grounding it's a loop that's taken off on its own it might seem like it does but it actually doesn't because people are evolving all the time people are changing and this also becomes a problem with AI in the sense that both the RNA I have this problem if those who do research in these areas believe that people are unchanging and there's some fixed bar where you pass the Turing test and you make the perfect VR that feels real or something like that it's presuming an untruth which is that people don't change one of the things that I believe about VR is that as it gets better it teaches us to perceive the difference between vert reality and reality better and better so in the old days once again as before the era of computer-generated racist horribleness and before before all of that before all of that we had the first demos of shared virtual worlds of vert reality we had the first ones in color we had the first ones with your hand and then like there was a whole bunch of stuff about our stuff that just didn't wasn't available in fact we had the first ones that weren't wireframes and so it was a very much sought after demo and actually one of the things about those years that I don't miss is how the politics and the sort of heavy-handedness that people use to try to get in the demo queue which was very uncomfortable but at any rate whenever we did do a demo who'd never experienced for someone who'd never experienced it before I'd always try to sneak something into the room in front of the person that was an interesting physical object usually a flower I think we have a geode sitting around or something like that and the reason why is there's this amazing moment that these days I think it's just kind of wasted because people aren't thinking about it this way but there's this moment where you've gotten used to whatever the quality is of the VR system of the moment whatever it's tracking is and its resolution and it's it's the way it represents your hands and all that kind of stuff and how much freedom of motion you have and blah blah blah all of that becomes your normal after a little while when you're in the demo you take the headset off and then you look at something from the real world and it's hyper real for a moment like like what you can do is you can treat fort reality not as an alternative to reality at all because it's not it's a medium it's a it's a thing we make it's an artifact it's a performance it's a magic act but you can treat it as a palette freshener you can treat it as a way to perceive the usual world with greater alacrity and that to me remains the most beautiful moment in all of our reality is that moment when you take the thing off in the case of mixed reality where you're still seeing the real world the the analogous way to do that is okay so I had them one of my steep my student interns which one did this Judith amorous from MIT Media Lab did this she did this this weird vandalism program with the hololens and so what you do with it is you you mess up a space you can graffiti the walls and you can throw this gloop it's tough and the gloop covers things and you can what else was there I think you maybe you can spray like foam and spider I don't think she had toilet paper in it but anyway it's just like you can basically be and you can be a vandal and it's really fun it's actually kind of cool but the thing about it though is and this might sound like sacrilege just might sound like the horrible the most horrible thing in the world to do but it's an amazing experience to go into the redwood forest with that program and a hololens and and harmlessly start adding artifacts to the trees and then suddenly you see the difference and you really see the tree that's the thing that is so amazing that you should say why do you need some fancy expensive headset to see a tree maybe it's pathetic that one might need it but whatever works you know and I just find it astonishing like it opens my perception and I for all of the applications of VR that I've been involved with and there have been a lot over the years well I won't say that that's my very favorite my very favorite is the surgical simulator which I'll say something about in a second but my second favorite is this palette freshener thing like just being able to see reality by comparing it to something is the heart and soul of VR from my point of view so in the 80s an Alaska and I and a surgeon at Stanford named Joe Rosen built the first surgeon simulator in VR and it's evolved and evolved and gone through many generations a few years ago my wife had cancer which he has recovered from and she's doing great in her principal surgeon used a procedure that was designed in a direct descendent of original VR simulator he trained for it and it turned out his teacher had been a student of Joe's the original collaborator so these things come around and that's my very very favorite VR application story

Channel: Gray Area

Views: 24,069

Rating: 4.9239545 out of 5

Keywords: +Gray, Area, Foundation, for, the, arts, Organization, +Art, +Tech, and, Technology, Theater

Id: lsNF4KfmwkY

Length: 57min 20sec (3440 seconds)

Published: Sat Feb 22 2020