IAN CHENG DMA LECTURE SERIES 2017-18

hello okay I think we're ready to go so this is very exciting we were so lucky to have in Chang and his super busy schedule he was just giving a talk at Birth UC Berkeley last night and earlier today and then zipped down here and now we've got him at UCLA so thanks so much for coming just I'll give a brief introduction and some time I was gonna ask you in if you if people are asking questions would you prefer they sort of hold off till later or ask as they come up but I don't know if you have a preference okay that might be good it helps us to because we we stream our talks so other people remotely are looking and and so we it helps us to have a microphone when people ask questions we'll be coming around with a mic too so in Chang born in Los Angeles and lives and works in New York currently he graduated from UC Berkeley in 2006 having studied cognitive science though he told me he did sneak some art classes in there but then he went on to Columbia University where he graduated 2009 with an MFA and visual arts this says your professional practice began in 2011 and that was a really good time to start with this work too because I think the art world was just becoming more aware of this kind of work and has shown extensively in Europe and in the u.s. it's got a few shows up right now and then the just a little statement here there the romance between science and art can be traced back centuries Ian Chang maintains the relationship by marrying an innovative technology and visual objects in a dynamic and interactive manner Chang uses a combination of cognitive behavior recognition patterns and computer simulation to create an evolving and dynamic aesthetic experience the artists body of work is in a constant state of metamorphose never repeating itself the simulations consists of a group of heterogeneous virtual objects taken from a number of sources by bestowing behaviors to the virtual images Chang explores a complex nonlinear notion that relates to the neurological and psychosocial shifts happening with the human mind so get your head around that and we'll hear more from an cheng thank you very excited to be here thank you for having me you can look at this later so just really briefly I graduated a grad school around 2009 and I was in New York at the time and in grad school I didn't really make any art I didn't know what I was doing with my life I didn't know what to make now surrounded by a lot of peers were very prolific especially painters and it kind of became this sort of moment in the woods where it was only after I'm sort of the beneficiary of really good mentors I think it was only after graduating from Columbia I had the chance to work for two artists who I really respect and love Paul Chan and Pierre week and what I really loved about both of them more than their work was how they structured their day what their schedule is like what they ate how they spent their time and one of the amazing things they did with their time was that they really kept themselves alone up until like 11:00 a.m. and I didn't have to come into work until 11:30 12:00 sometimes when they're on a creative streak like sometimes 1 1 p.m. and this kind of freed up a whole space of time for me to in a way sort of mirror their schedule I became really free in the mornings and turns out that seems to be for non nocturnal people one of the most creative times in the day after I don't know if you ever feel this but this is kind of like post-lunch slump and then afterwards maybe 5:00 p.m. you kind of start to recover and get better ideas again but the mornings really important and I really learned this from Pierre and Paul so around 2011 I started making my own work and first I was making a lot of animations and for any of you who do animation it's like extremely tedious and you know that it can be a real perfectionist and you're a perfectionist here like really close so animating a hand could you know you could be doing this forever breaking down every every frame and it's kind of perfecting it and I'm a control freak with my work but I won't really want to escape this perfectionism I wanted to be a control freak from like 50 feet up from here and I started to think how I could really save energy in a way and save my time and sanity making animations and I thought about how an animation might be able to animate itself and I kind of stumbled on the idea of these game engines like the game engine I are currently now I use and have been using called unity which gave me the idea to sort of make a video game that plays itself and if a video game could play itself than eyes and artists could be free to really think about broader things that aren't as tedious as the actual animation but maybe about behavior or about a group of organisms or about how movement changes depending on environmental conditions and social conditions and this video game engine unity really freed me up to do this and it's a tool I used to this day and I just want to emphasize that it was really important for me to find this tool I think there's a just we're in a media program here there's a real temptation for me always to like grab onto the newest tool but I felt that was something like unity it had a kind of ecosystem around it there was a lot of developers doing stuff major games were being made with it and I felt really supported when I didn't know what I was doing with it this will come up more I think as I show you some the work really feel that in some ways I started making these simulations in a moment where all the plumbing was really done in a way like this it's a dumb thing to say but we have the internet it's really done I mean not done it's like really active but the plumbing is there unity this game engines there the way to see this kind of stuff online the way to present it and very packaged and easy way it's all there and in a way I feel like we're in a golden age of content where once the Plumbing's done you can start to play with the water and so I really want to play with the water here and my initial simulations were sort of meaningless I think they were very like me experimental I won't show them to you right now I kind of want to skip forward a little bit but around 2015 after making like 1012 different simulations I start to feel that I needed help I wanted to make more complex works I wanted to make a simulation that was kind of about something I felt that their meaninglessness wasn't a problem but that it could benefit from an alternate perspective I really wanted something that was meaningful but I didn't want to work to be burdened by meaning and so I thought oh a story is really meaningful I need to put a story in there I need to put a story in a simulation together and if they started to sculpt each other or fight each other then we would get this pretty interesting productive conflict and I realized as an artist that in order to make something more more complex I myself had to become complex and is around that time I started working with my producer Veronica so and together we started making this body of work called emissaries I'm gonna quickly show you the first one sort of talk of talk around it and from there I think I don't start open mic negs I thought I'd just show you like my hard drive and we could kind of walk through maybe some recent projects like I've all these files that never see the light of day so I thought maybe I should show you guys so this is called emissary and the squad of gods I made it 2015 with my producer Veronica [Music] that's not right so all the characters you see in this landscape there really wanted to make this simulate a kind of narrative premise the idea of a kind of ancient community and they're living on the side of a volcano but the volcanoes never erupted before but it's starting to tremor and so there's a threat of an eruption threats a really interesting state because that's right it is a kind of moment where you don't know what to do it's really stressful do you stay or do you leave and I wanted to simulate his ancient community so all this these characters here they have an internal family structure so someone is someone's daughter or son or parent and within that there's a kind of sense of authority especially for the little kid characters and all of the characters here their model of mind their very basic AI was borrowed and inspired by the Sims and so the Sims had a very beautiful idea of AI it proposed that each kind of agent would have a series of needs like being hungry or being thirsty or needing to play or socialize and those needs were decaying over time and the objects in the environment including other people advertised those needs how to satisfy them so this water ball here would advertise to me and drink me if you're thirsty and I would drink it and get the reward of having my thirst replenished and this simulation was structured in a way where all the objects you see parts of the landscape and other characters had are riddled with invisible advertisings advertisements that kind of announced themselves to the needs of the agents and I found this really beautiful as a model of mind I mean it's it's it was very easy to implement but the idea was that you know cognition is just not in your head it's kind of distributed between your head you and all the objects in your environment all the people environment the cognition is a kind of relationship with everything and yourself and within that what was so fun to make this was that you have enough objects and have enough characters and you have enough agents all kind of advertising to each other kind of signalling to each other you get a very emergent kind of behavior you get kind of emerging group behavior you get your weirdo is to kind of become individualistic that go off the edge and amongst all these characters there's one character called the emissary which I'll show you here this girl and she has a very different AI her a eye is entirely scripted she's not won't even call it many eye but in a way I think it's the most advanced kind of AI in the sense that relative to other characters the emissary has its got like a Shakespearean actor she has the entire script that I'd given her and she's trying to produce this script in the landscape she's trying to act it out it's kind of like when you wake up in the morning you make a plan for the day you try to kind of you know I try to get here from Berkeley make my plane get a new burr get to UCLA and so many things have gone wrong along the way and I really wanted to use this one character to embody that you have a story and the idea of a narrative and something very deterministic and if she were to get through her entire journey and we were to watch that it would kind of feel like a movie it would kind of feel very meaningful but because it's within the plate of a simulation countervailing forces are helping against her all the time and sometimes with her all the other agents are often in the way or kind of inspiring to do other things distract her and sometimes she in her script is trying to corral them into doing stuff to do with her story and so these two forces combined allow the story to kind of fracture out in ways that are unpredictable to me I was really inspired by these kind of animal cams on the internet where you see a panda and it's really unremarkable for a really long time I really wanted my simulations to there's narrative but it's not the center of gravity and when you watch a panda on the panda can man is sleeping for like 12 hours and on a 13th hour you know rolls over gets up like breaks a bamboo stick and this is like revelatory and I really wanted to capture that sense of boredom or boredom was really kind of exciting and at the same time my wife Rachel Rose she was extremely extremely obsessed with the Real Housewives of Orange County and I in turn became obsessed with it and watched all seven seasons with her over her shoulder and Real Housewives is really special because for me I mean not to like overly elevated but this is really beautiful because what you see what when I was what I read it when I got really into it I read about it the producers the initial idea was to gather like really fierce and strong personalities put them in a room and throw some alcohol in there and I let the drama kind of just emerge on its own now seven seasons later I can tell you the narrative art a story of each of the Real Housewives there is a narrative there but when you watch the texture of Real Housewives it's something closer to like the sims or even like the Panda cam where you're watching to sort of a observation of behavior and this in itself is thrilling and the center of gravity is there and yet in the background there is the larger narrative the farallon's lives there really one of my simulations to embody that characteristic I mean they're definitely not movies and I really wanted it to apply this other kind of register of narrative and so for this particular simulation I present it with two views so there is what we see here I kind of call it the the overview or the ecosystem view this is like the Panda cam view this is the Real Housewives view and then there's a more kind of cinematic view there's the story of this emissary character if you were to watch this screen alone you might be convinced for a moment if her narrative doesn't get diverted too much that this is a kind of animated film and the other aspect of narrative I'm really interested in there's this amazing software called dramatical it's also a theory of narrative and the gist of it is that is a the idea that narrative is actually the work of one human brain trying to solve a problem and it's artful because it allows a mind to weave a narrative that takes place objectively and subjectively when you watch a really great movie you're really into the point of view of the main character and yet the movie cuts between the main character and the larger kind of activity or problem of the film and I really wanted to I really wanted the simulations to embody this two kind of perspectives and within that triangulation of perspective you kind of get something called meaning it's a little bit like a war movie where you see a soldier in the trench and that's one perspective and in itself that experience alone it's pretty meaningless the soldiers is trying to survive but in a war movie you get that perspective and then you get the perspective of the general in the field looking at all the soldiers like it's a map and it's and that in itself is pretty meaningless it's kind of like cold-hearted but together that triangulation and maybe even you get the perspective of the opposing side all three of those perspectives start to become something more meaningful the chain of events isn't just a chain of events in the universe it's something that you can kind of attach a meaning to you can say this counts of something you can see this kind of kind of resolution from multiple perspectives that you can't get in our normal daily life I mean we get it through looking at things but just the experience of being alive is very pretty meaningless when you're only stuck in this point of view so blah blah blah I really wanted to do that for this simulation and I'm gonna stop this one now and actually I'm gonna skip ahead a little bit to stops so and this this was the first demos it's a trilogy of simulations and I'll skip the second one I'll show you a little bit now about the third one which is the more recent it's called emissary sunsets the self and I want to just back up a little and say a little bit that these things don't just appear this way I don't start working that way I try to actually find the easiest way to work and that's often with drawing try to find the most facile way to do things to kind of express an idea and to not get off-track and so with emissaries especially the third one I start to start from the place of drawing I just kind of started drawing a lot and art if you've spent enough time trying to make card you realize art start some pretty stupid places you know it's kind of just what you feel the time but then if you take that feeling seriously and you get over how stupid it is you kind of stick with it it becomes complex and I sort of expands itself and it's emissary's the third one started from I mean this drawing it's a drawing of like a group of characters kind of like mob like and they're confronted with not a mob but kind of super organism and organism composed of smaller organisms and just had this idea this thought in this drawing of like a group of characters meeting another kind of group or organization these two organizations coming together or meeting and their differences and their similarities get amplified in their juxtaposition and how they kind of behave and act against each other with each other and I started doing these kind of drawings to sort of worked out who like who are these guys and then who are these guys no one knows I don't know so this kind of activity of drawing trying to figure this out kind of mucking around dirt goes on for I don't know two three months there's a moment where my producer Veronica just tells me you kind of move on you got to have a real idea we got to kind of you know start programming something we got to kind of get something on the screen but this is this occupies a lot of imaginative space because it's so easy like drawing so fast and easy so can just try so many ideas so quickly so here I'm just trying to work out what these this mob-like character looks like I start to think about meerkats and when meerkats get scared when they're threatened they actually pull together they group together and they become very packed like they're going to rear up and tried to scare away their threat so I thought maybe these characters kind of resemble a meerkat more stuff and I thought about the other character which had it be more alien I think it had to be more nonhumanoid and I started composing a vocabulary of almost like Lego parts or kind of plant like parts I started calling them the worm leaves are the worms and I don't know there's a story geometric shapes at first I mean it starts really stupid you just kind of go wherever again like from 7 a.m. to 11 a.m. like you can I just feel like and be stupid it's like that partner brain that tells you this is stupid it just hasn't woken up yet starting to fashion what were they where then I thought oh they got to start wearing the plants that they're all around these other kind of worm leaf characters they gotta start wearing them why not right oh so I started making like this vocabulary for them I thought again out of kind of energy conservation laziness maybe or just I I thought like oh if I made a vocabulary like maybe just twelve different parts of it fifteen parts it could be like a little mini language and by composing it together you could get a lot of procedural variety like hair - no mmm burnt okay so here are like this is the vocabulary that I ended up with which I'll show you what came okay like more fashion the world was trying to think about how they are ting to think about how they might move but then really thinking about how I can save myself from having to be the animator okay so drawn a lot and Veronica's like being great drawings move on so what the meerkat like character ended up looking like that human they look they ended up looking like this and here was the call this the mascot like I made one whom in I just made one and then for those of you who are familiar with 3d animation there's a kind of inner skeleton and kind of rigging to help you animate it I added like three times more rigging than I needed in order to distort the face in different ways in order to distort like the chest the arms the legs the torso so I could get a whole population of women for free like so I could get you know teenage in for saruman these like bored short throws like kids you know kids it's important kids then again like the human I start as I was kind of drawing this and designing this and ultimately 3d modeling it I really wanted to make it it's a subtle thing but it's also obviously I really wanted to make the way it looked reflect its inner behavior and if it were to be meerkat like it had to be sort of animal-like kind of cute but then so quickly I could turn into kind of nastiness like when it feels threatened it becomes not cute anymore and so we eventually stumbled or came like settled on this guy and I don't know it's kind of cute but then you get like I don't know like this is kind of jangled teeth one human in particular so I mean we're really like in a way that's like kind of embodiment of like a group with a rush like kind of a prejudice I started they were gonna be prejudiced against mutations in landscaper overly active mutations in landscape and with some generosity I start to think about well maybe they're prejudiced about the mutations landscape because they really want to protect the landscapes their home it's both the naturalistic aspects and also the culture they wanted to kind of maintain it no fair enough but then one human who I called the rancher sort like the tree hugger of the group the ranchers like prejudice and the flip foot and the opposite like the rancher fell in love with these worm leaves this kind of plants that starts to become mutate and start to come the super organism and in a way uses applies the same personality trait of this prejudice to protect the worm these against it's like fellow human here's like here's the ecstasy of the Ranger I have to say at a certain point when you have enough toys like on the table it becomes really fun to make like costume design their answer was just a blast here's the Power Ranger shot sure love this the dissimulation generate this pose and just like on the right track what else can I show you guys I did not use motion capture for this third one I did for the first time Astaire's you saw I worked with Madeline Hollander who helped me motion capture the idea of an ancient community and how they might move I want to get into the kind of it was really inspired by Julian Jaynes and like the sharing of voices just really briefly Julian James crazy psychiatrist from 70s who's hypothesized that people didn't have consciousness 3,000 years ago and said they heard how they into voices like voices of authority figures like their mom and their dad or eventually gods and that helped them solve problems it's a little bit like when I see myself and then like the walkway here at UCLA and hunched over I hear my mom's voice she's like the stand up straight and I know it's my mom's voice it's my memory but he Apophis eyes that ancient people just assumed that mom was there just like can see her so there's just like a kind of spirit or God and I worked with Madelyn to kind of try to develop a way of moving for the characters and the first emissaries that really embody the sort of lopsidedness of of hearing a voice on the right side your left your left ear but your right brain but this this one with the human this was totally not choreographed in in a pre premade way we can get to this in a second well let's get to now why not so the AI for this third emissary's I really wanted to have this kind of threat this is kind of threat response I really wanted to do sort of the stuff it did the first dynasties and I thought I you know I'm not Google I can't really think of a all-encompassing model to mind I came across this beautiful book by Marvin Minsky called society of the mind and and and he kind of proposes the idea that within a mind you have a lot of like sub brains that a a brain in mind is the emergent effect of many little brains I don't know if this is neurologically true but it felt it felt kind of true and so I thought the AI has to be like many brains sub brains and they compete with each other and it's thought that the first brain that gets the most attention is the threat brain what I call the threat brain the threat brain kind of scans environment for threats and when it encounters a threat and has a kind of fight-or-flight response but when there's no threat in the environment then it kind of graduates to what I call the limbic brain the brain that's sort of like the zooms to do with like feeling hungry or feeling some kind of burning desire that gets more and more urgent and then if an amine is satisfied with its hunger slept enough like played around other characters enough there's no threats then it graduates yet again to a third brain - called a narrative brain which allow us to plan ahead one or two steps and planning one or head one or two steps allows the human to then get into more trouble basically to start to reactivate other needs and then start to discover our inadvertently uncover new threats so these three brains constantly are competing with each other this isn't totally representative of it but this is so I make these diagrams to try to tell some of the engineers I worked with on this project when I was thinking but it starts like kind of like drawing starts with this really easy diagramming this easy diagramming is pretty illegible to anyone until I kind of Whittle it down to the easiness of what I just told you when I'm working with them different ends of Engineers we ended up with something that looks like this this was a graphical representation of what I just described those three brains are constantly I say updating amongst themselves and unified by something called utility theory where every kind of action that could be taken by the human agent amongst those three brains is weighted with a score it's almost like a very primitive neural net and those scores depreciate or appreciate over time as the they're more actively achieving like getting food and those those scores then result in a winner and that winner is sort of like who gets the spotlight the person gets spotlight the brain that gets a spotlight gets to act on that and it results physically and virtually physically in an action that you've seen the animation skip does well this is testing a threat brain yeah here again here just again to the other side of the equation the the other characters these worms started from this basic vocabulary and from that again the impetus was to try to find as much to squeeze out of it as much expert civet E as possible so we can get different kind of morphologies from you sort like a plumber's nightmare diff furniture kind of like you know through the Moomins start to make yeah do they start to make art like bad art here over like here's like you know a sculpture garden starting to put things together here the really fun part about this stage and making a simulations putting the kind of ingredients together it's at this moment I feel like and you kind of finally be almost like a chef like I can finally start to cook here here's like a group of Oommen I'm just kind of testing out their different you know different levels of urgency like they're all in different states here I found this one guy Veronica really likes this one guy yeah yeah this is like this is the intersection of AI and art just so happy like to show you a few making other videos I constantly screen record stuff as I'm making it just to remind myself that it's worth making what I should say is that this whole process is really hard and there's like a lot of doubt involved in making a project like this and again it goes back to that dumb drawing like you know I go back to that drawing to remind myself the feeling as dumb as it was and it's just instinctive as it was to make that drawing like that is my true north that is the kind of thing that will get me through all the difficulties of this project because I can always go back to that it's like it's simple enough to know what that feeling is again I try to look for that every single moment try to look for that spark and every single moment the drudgery of trying to make a simulation like this here what we should talk a little bit about the the worm character so the worm characters I describe the different parts that I kind of made a count language of bodily language for to compose together but what's holding them actually together physically is a it's called a cot that's called a cabal spring physics system that we developed in unity on top of their existing physics system and with ball springs physics has existed for a long time but what's so beautiful about it I think is that there's a kind of bounciness or squishiness to it that you don't normally see in computer graphics you know computer graphics usually it so they kind of pre-rendered mesh that has a kind of rigidity to it and I really want something to be the office of that the human were kind of these meshes that had this sort of mm armature human-like structure that one of these other characters to have this kind of squishiness to it something more alien I'll show you so what that became is something like this so this is the you know the synthesis of there's different body parts mixed with this underlying physics system I mean for me like just very beautiful shapes like movement that is I don't know where this comes from and every moment like this like making a screen round like this is just energy it's like just energy for the project because it's again it's just so hard making things that to have something like this like ah this is like a Red Bulls like it just keeps me going just show you a little bit of like the human how they start to develop in there in terms of their movement I first started was a human start with something who's this running around shame in this room Oh like okay yeah different ages cool oh I mean oh this was such a nightmare for a little bit's in the final project but for a while we try to create our own genetic algorithm for how the the worm Leafs would move and we gave a very basic neural net that existed modular lis within each part and then aggregated across the whole organism and in this case we were trying to reward if we're just getting past this group of blocks see the orange guys kind of succeeding and saw the guys sort of a loser Bravo yeah a little bit this made into a final project but it was such an idea at the time to kind of integrate all that that was a whole money trail we went down putting them together touch is really important I think and the simulations I've made I've discovered that touch is really important because often when you see a video game or when you play a video game characters are really touch each other sort of like shoot each other or like throw intermediary objects at each other or hand objects to each other but they don't touch there's often very little contact and they thought thought like touch would give this a real sense of aliveness there's some of the other touch stuff I'll show you touch stuff touch touch touching I think seeing touch is very empathetic and this is trying to pick up a box the massive things definitely matter so obviously he does not succeed and so it comes of all this is something like like this is a good example so we got our Moomin characters we got our worm characters in this video is screen recording when the worms is physically attaching touching the head of the human in the movements going to go one way and the worms gonna get another way and what you get are these crazy movements and so it comes of all of this is well when you guys walked in that was the emissary census itself I'll play a little bit of that but I think first [Music] [Music] I mean like really boring keep let's just we'll keep that let's let it sit for a little just keep it be boring here talk about some other stuff oh I'd like to show you how this these works are ultimately physically installed so 2017 and did a show at ps1 moment and I should all three emissaries oh my god you guys are gonna like see my tax return or something so I want to show you a little bit how inspired by art oftentimes that when I see an exhibition when I think of my own exhibition as a viewer I want to go and see inspiring like I want to see inspiring art and I try really hard to create a feeling I think often is almost easier to create physically a physical feeling of magic and kind of portaling and so for example this is the first demo Cerys these are two different views but what we really nerdy doubt on was like these doors these kind of archways and they seem really like whatever who cares their doors but this kind of curving kind of fatness of this door feels more like a portal for me at least and this then led into the following room to the second emissary's which I'm not going to show you right now it's called emissary forks at perfection oh I just want to say that these exhibitions are entirely designed and like Sketchup initially you really really kind of feel that like you know like this is obviously like I made some block and I extruded it out of this and I thought that looks like a good idea hopefully it'll just work it's definitely exhibition design has definitely become the product of like Sketchup for me at least [Music] okay so then behind this screen you get that uh another portal and then into this more intimate room and this is its very sense of the self and I really wanted to be square I really wanted it to feel like a portrait of this kind of mutation and all that kind of other characters around it we're congregating around it bothering it collaborating with it and that it would essentially be it's a simulation there's a narrative agent there's an emissary in there that is possessing that mutation but it would feel like a portrait of this emissary and I really wanted to make a giant like Instagram but the crazy thing about Instagram of the square is that squares look really bad in real life they look like kind of narrower and but when down your phone your phone is like really rectangular and when we have a square inside a rectangle looks really great but when you have a square in real life it looks like and so I really murdered it out about making the ratio so that it felt like a nice square so this is a really nice square for me this is a seven six ratio square and I love this detail I don't know why I'm just telling you this because I think these sort of details matter in terms of the magic okay I mean something's happening here I'm gonna stop this and I'd like to just really briefly show you a newer project so this last year I started start trying to reappraise what it meant to make a simulation and I thought I really want to focus on just one character I really want to focus on one creature I really want to explore what it might mean to be within the container of a creature so before is kind of using this container of an ecosystem and all the characters in the ecosystem I really now wanted to think about a creature and the complexity that's inside just one creature and I kind of thought of a creature as a kind of compositional space and it would you know in the way that for a filmmaker a movie allows a filmmaker to insert ideas about production design set design actors lighting cinematography sound design music a creature a thought could allow me to contain ideas about like metabolism life death like when something's born when something gets sick when it decays and of course ideas about AI and how the body affects the brain how the brain affects the body and you know like a painters usually say you got to mind the edges and when I think about the container of a creature mining the edges then means like thinking about and having like the pleasure to think about how something is born how something might die and I call this creature start as I was trying to make it I call this creature Bob and it stands for bag of beliefs and again it started from a place of drawing this I'll show you some drawings the very very first the most important drawing Veronica says this isn't the most important drawing I think she's right yes it's this guy here this is Bob this is essentially what Bob became this was definitely the the energy I was talking about the kind of feeling that really sustained the whole project the idea of Bob's body was that it would really directly the kind of structure of its mind in that kind of cartoonish pop psychology ways so like there's sort of the left brain which is very much focused linear kind of the ambassador of who we are and I this is sort of that's kind of head that you and I would just immediately recognize like oh we're gravitating to faces like this is this is the head of the creature but then it snakes in and terminates into this sort of fractal izing said tree roots in a way and I thought I was Bob gained new memories as Bob gained new pardon personality traits and behaviors parts of its body would literally reflect out would grow out of this and in a really like cartoonish way this was sort of embody the right side of the brain which is much more intuitive and open-ended and absorbing of all kinds of stimuli and sort of the cold storage for memories like you've suppressed or deep down there and to come out in dreams and that this body would literally be a conflict between those two parts of the mind and it could then therefore play out for us I think often art is really like an exercise in spatial izing things because we can only really understand things in front of us when their spatial eyes and their physicalized you know like a film director will always say like you got to show it you can't just tell it because you got a spatial eyes it's so much more potent when you see something out there and so I thought oh to kind of think about AI in the mind I got really think about the body maybe the body could do double duty as representing and embodying the different changes in Bob's mind and the AI for Bob I'll talk a little bit more about the body real quick so then with that body plan very quickly you could start to imagine different kinds of postures a bob would take [Music] for example you can imagine the kind of personality that is really a homebody someone who was kind of you know grown up enough but decided not to you know seek new adventurers kind of past that and sort of starts to literally set roots somewhere starting to turn into a tree I was really inspired by this mollusk that crawls around on the sea and it's constantly it's like looking half its life for a home and finally finds the perfect rock it sticks on to it but then it can't find food anymore so then it starts to eat its own brain literally the cost of finding a home has become brain-dead and I was really inspired by this thought that maybe that could really be embodied and inside the body of one kind of Bob a bob that has whose personality went that way its body would then reflect though on the flip side of that you get something like I call it Starship Bob it's sort of inverse of that there's this more adventurous Bob whose head is maybe leading it but then almost tendrils starts to kind of be like a little mini army you kind of flocking along and then when Bob gets really unconscious those tangel start to take over they become the more dominant leading force call this bob bob and then through all this there's this idea that if you create a creature the most beautiful thing about creature kind of sentient being potentially a sentient being is that it would have some persistence that you know when Bob starts life its life when it's born it will just keep going and you turn off the simulation you turn it back on it's still there as it was and this sense of permanence was really important and I thought how would this the life of Bob really kind of be enriched and it sort of stumbled on this idea that like mmm the creatures I really liked in when I was growing up in like in the media that I see creatures like smog from lower the rains are like Tama Toa this like big gay fabulous crowd from Milano is that they have a life outside of the narrative that they're placed in when you when the hobbits first encountered Smaug what was so beautiful Tolkien did was that Smaug is laying on giant pile of gold and immediately infer smog has a real life this guy's a busy guy right he's living this whole Real Housewife life collecting gold and then these hobbits like try to corral him into their third act narrative and this is how we think of Smaug is a fierce dragon but Tolkien just shows us a glimpse of Smaug in repose like kind of relax just being at home and I really wanted a character to have this duality this kind of repose and it's sort of like active a kind of fierce character and I thought the only way to really give a character like this life in by extension given a aiya life a life that was more interesting than just our momentary encounter with it but that had adventures of its own was to start to make it maybe more on our plane more interactive all the previous simulations I showed you were kind of video games that play themselves I really wanted to make or at least try to you know crack the code a little bit on making an artwork interactive but interactive art gets a really bad rap I think and so then I thought mmm maybe more interesting or fun way to think about this is try to make an art with a nervous system and if an artwork has a nervous system then any person it encounters would be subject to influencing that nervous system I worked with a viola got off from lux loop to develop a kind of interface for communicating with Bob was like skyping with Bob through an iPhone interface and Bob could read your emotional effect and this would be a data point in Bob's AI which I'll talk about in a sec this was a whole other bunny trail to chase Bob just show you briefly some of the making of videos so kind of like the third emissary's it started with the ball spring Network I just thought we were not done with this thing yet and project a project the thing I really liked to conserve or save or regenerate is the kind of tools that I built on the back end on the side of the project that can become overtime like a kind of mm toolbox for me that's very specific so this ball spring physics excuse me this ball spring physics became the basis of Bob's body here we see Bob this is the beginning of Bob trying to like trying to figure out how its body could become interested in something this is going to green or I can skip this okay so there's a whole bunny trail it's gonna say real quick who's this guy so I thought you know Bob would have no dignity if Bob was just there for you as a person there's just waiting person-to-person to have social interactions and that was the extent of its life I thought Bob would have a lot more dignity if you had a little friend and you know when you watch like Pixar movies the success of I think of a Pixar movie is that all the characters like it's always two characters I was like Buzz Lightyear and it's woody it's like finding Nemo his dad and dory and those two characters are faced with the same reality but they see it from very different points of view and when you have these different points of view they have different problem-solving styles and when they solve it they have these different perspectives of addressing the problem and hopefully working together or not to get to a better reality and so I thought Bob really needed a little friend to kind of be as dual sidekick so there was a there's a whole era of making Bob wear this Buffalo was in the picture I can only briefly say why this Buffalo didn't make it in but it's a lot it's a lot to do the sense of scale I mean on one hand a buffalo allows you to really recognize something legible but then I think you think I thought you started to sympathize for the Buffalo because that's what we know Bob is like it was such a weird dude right the Buffalo is something that you then think oh it's the Buffalo Bob like maybe we're supposed to watch the Buffalo and also the Buffalo I think gave away a little bit Bob's scale you kind of know what a Buffalo is sizes so then you think Obama must be this size and I think this kind of took away a little bit the magic so we axe the Buffalo but not without having some fun with the Buffalo first some of the tips of all of the end points of all Bob again I wanted to create a kind of sub agency within them so that they would be themselves a kind of mini group but the kind of organization that's alive so we applied a flocking algorithm on each of those sub tip those those tips of Bob as if they were their own agents but they were constrained by the connection to this unified body and from that very magically we got these I think very beautiful movements that were very unexpected this is Bob's physics system struggling to flock at the same time so for example when Bob is or when the flock when the flocking members of Bob's body are oriented upwards you get something that looks like an enemy or plant or kind of mm-hmm like yeah kind of a plant that would suggest this one kind of personality and then when it flips the other way that plant sort of becomes like legs like a like an ambulatory animal let's see what else oh so Bob's face here's some of Bob's expressions again like so Bob's face we're sort of the representative Bob it's what you and I recognize they really think you know our can get so complex you should take I feel I should take one should take the easiest route to getting some kind of thing that's legible to a viewer and I thought if the face is going to be sort of like the left brains or ambassador it needed to you know let's just borrow like from the anime playbook and let's just make it really emotive its board embarrassed surprised flirty mmm let's gonna show you guys death okay so Bob's body is a small spring that worked and I think the previous simulation was like there's no consequence to that really but with Bob I really wanted a situation where if you stretched Bob enough it would be painful and that pain would only be relieved by being broken my technical director Sam being said this was like really emotional in this video for him and I there's something to that because I think the choice to make Bob is sort of stringy ball spring theme the consequence of that isn't obvious until you start stretching a stringy ball spring thing until it breaks and then you start to feel sad for it I think trying to find them emotive quality of something as technical as a ball spring physics Network was really important then we started to try to give Bob ability so this is Bob eating it's the first time since Bob throwing something this is Bob throwing the Buffalo Oh so bob has the ability to self prune it's a very neutral light kind of ability one hand it can kind of take care of it's like you know decaying bits on the other hand it could be kind of this obsessive behavior like picking out all the hairs in your eyebrow and so somehow this bobbin we're testing it turned into this like evil snake form I think again suggestive of maybe the underlying personality or emotional state of BA the tongue this is maybe Bob pruning more splob chasing its own tail I'm gonna show you Bob let's look at Bob five minutes okay perfect yeah [Music] [Applause] bingo [Music] [Music] [Music] [Music] that's one speak really briefly about the a and Bob so Bob has those three layers I've spoke about in emissaries but in addition to that we really wanted to think about an aspect of the AI like what it might mean for Bob to feel something and the bigger the more we dug under the surface of women to feel something we realized that feeling isn't one thing it's a kind of a comparison and so about every 10 20 seconds Bob is sampling about 25 different data points the state of its body its metabolism how old Bob is the time of day and when there's the iPhone interface the emotionality of the viewer and these 25 data points are pretty meaningless individually but together we kind of bundled them up and we called it a memory and we stored it in this this data structure called TKD tree and that's a kind of data structure that's n-dimensional it's not bounded by any fixed dimensions you can stick data in into an in dimensional array and within that n dimensions one of its properties is very quick to search and so we devised this idea of what feeling is where Bob has all these memories littered inside skatey tree and at any given moment when bob wants to feel something bob samples there's 25 data points of the current moment and compares it to the close searches the KT tree for the closest memory that matches the current moment of course the closest moment that matches the current moment is somewhat different it's a little bit like blossom as a UCLA is visiting a friend who I didn't really like that much and so there's a very negative experience and today I'm really happy to be here and so comparatively my emotionality my feeling about UCLA is really positive things are getting better at UCLA and it's that Delta that betterness that pretty subjective but nonetheless real comparison that gives Bob a sense of feeling things are better or worse now taking that Bob then reapplies that to all the different stimuli around Bob so if things are getting better and Bob's getting fed like full feed Bob when Bob is getting fed and things are getting better Bob then associates a positive feeling toward food but the office it could be true - like things could be getting seemingly worse it's feeling is bad and then it gets fed at the same time Bob will associate food with a kind of negativity and over time if there enough coincidences of being fed and a negative Delta Bob will be this kind of well Bob will have problems with food Bob will have a really problematic relationship with food and to me this felt I mean again it's a model of mind but it felt somewhat truthful I think - how things feel and part of the interest in making AI for Bob is to really get out of maybe the the labels that we find in cognitive sciences in trying to approach a I like Bob's and motions have labels we have names for emotions we have names for feelings but really if you think about our own brains these are really like n dimensional spaces and we conveniently have labels for them to make it easier for us to talk about but really it's this kind of n dimensional space that is I think sort of at the heart of how something might begin to feel I think I'll stop there and thank you for listening and happy tech questions [Applause] hey thank you I just wanted to ask a quick thing it's so nice that you showed some of the behind the scenes and how you make because I think a lot of people might be interested too and I just wondered how did you form a team like how did that happen as an artist to put together a team how did you find the people to work with are they people you've just been working with for a long time or yeah so the the broader story is that well I just just came back from Berkeley where did my undergrad and just my memory was jogged I did this one class in Berkeley and a computer animation where we actually had to form teams as the nature of the class I mean to make a short film a short animated film I made a short film called midwife crisis about a midwife but we had like a team together and this really sparked the idea that you know making art or making kind of something creative could be a team effort could be not everyone has to have their hand on the pencil and then around 2015 my wife introduced me to Veronica so and she was producing video games at the time and somehow the idea of emissaries seemed better or something and so we started working together and you know Veronica for me you know the most pragmatic sense you know helps me find much more talented engineers and I ever could be helps me find three modelers sometimes helps me find a concept artist you know sound designers and only the kind of a kind of existential level she's like my gym coach you know she's she's the one that reminds me that drawing was good in the beginning just trust that feeling go back to it thank you other questions and microphone to be able to record it I thinks I just have a couple of questions one is about aesthetics and sort of the visual look of what you're doing because it's not I mean if I think about what unity does most sort of directly it's either kind of much more shaded stuff or 2d stuff yeah and you're somewhere different and I'm just curious how you think about that and then I'll just ask the second question too about sound and whether you take the same approach to audio in terms of algorithms and neural nets and so on or if the audio is more a compliment sure so the the aesthetics of it and initially with those earlier simulations I really tried to pare it down to something that was simple enough where you could more focus on the behaviors rather than maybe the realism of excuse me the rendering um another artist said to me recently that when you look at I don't know like revolutionary posters from like I don't know like 1920s Russia or something you don't remember all the politics surrounding me you look at you think oh that's a cool poster and somehow the things that are left in history are sort of the aesthetics of something and maybe a few of the compositional laws of it and then if you're really kind of a scholar about it then you understand where it comes from blah blah and somehow the aesthetics like really Rises surface over time and for me this yeah this really resonated this was like the aesthetics the look and feel of this is like super importance as important as the AI not more in a way and the sound the sound world for example above the sound we worked with a sound designer called Jeremy yang and Jeremy he does sounds for a self-driving car company trying to find sounds to be a motive for how being English like you know a car comes up and says you may pass but house has kind of a moat that it sort of like when you drive by a couple freeze like arguing and you don't care the English we understand their emotionality and he was very good at this so I've worked at Jeremy the sound sources are all stems but they're applied to different parts of Bob's body and can be triggered in that kind of a very interactive way the voice of Bob it's very inspired by this famous author called Nellie the talking otter I'm gonna show you Nellie this is important let's look at the Nellie for a sec again this duality this kind of proposed chill but then this is kind of fierceness I think gives an animal some dignity what is your what are your thoughts on like simulation ethics like is there ever a point where Bob's neural network is so complex that like pulling him apart and like playing around with him and like constantly changing him and playing with his emotions becomes like unethical in your mind I love torturing Bob such a pleasure what you can't do in real life you can do in simulation but ya know out a certain point of fuels I feel bad for Bob and at one point at the serpentine where this was first shown there were six Bob's there were six Bob's so that we could see almost like twin separate at Birth they all start from the same place and some Bob's flourished and grew and this one Bob in the first week picked out all tie brows Oh speak it like whittled it pruned itself down to a little nub and there everyone in the serpentine who works there just felt so sorry for it I felt so started for it and it was stuck in this somehow its body couldn't accommodate when he got so little it couldn't grow more and I'm just it's currently still just this little stub of a person it's sad but it's beautiful I mean and we keep it we didn't kill it we didn't kill it we just kind of let it live it's sad life you said your later where it gravitated more towards thinking about meaning do you ever stress about how this is received by your audience or do you ever think about whether that meaning is conveyed or do you kind of just like let it play out um definitely stress about it but there's the feeling that you know I tried to make the simulations so that when you come to them you could watch them for five seconds or for five hours and the way they're installed for example that I'll show you real quick the way Bob is physically installed is meant to evoke kind of like a reptile house or nature sanctuary and the hope is that in terms of its meaning in the most basic way you would come to see Bob as although artificial kind of artificial creature and artificial life form that you would give the same status as a real life form and so I really worked hard to try to create that feeling and I think feeling is sort of like a Trojan horse if for me when I'm making an artwork the feeling has something has to come through really clearly and then it's sort of the Trojan horse again you get a more complex things like the meaning of it or the AI of it or all the other interesting things I'm interested in it should feel like a clear thing I think and so I really obsess about about feeling but here here's how it looks at the certain time yeah so it's a those are flat screens tiled together inset into the wall and in the basic way the echo the floor and the echo the kind of sort of grid prison-like structure this window on the outside the serpentine but the hope is that when you walk into it you don't feel wow that's a tiled flat screen do you feel that's Bob could you talk a little bit about like the relationship between the users emotional effect and how that affects Bob in the simplest of ways it is just one of the data points that Bob is assessing whether this moment compared to the closest memory is trending up and down but what it does do is it does we try to infer the emotionality from the iPhone X notch camera but again it's just one data point the hope is in the future to really expand upon that to really make that more meaningful but right now it's just all stored into its memory and considered amongst all other kind of trivia in his head cool thank you [Applause]

Channel: UCLA Design Media Arts

Views: 2,079

Rating: 5 out of 5

Keywords:

Id: gBOfkWTUnvI

Length: 76min 57sec (4617 seconds)

Published: Tue Apr 10 2018