Brett Jones (Lightform): Projection Mapping - Shared Augmented Reality for Out of Home

you [Music] great I similar to the last talk this is going to be a string of videos so this is kind of like having the substitute teachers in for the class so kind of sit back and relax and enjoy what we're going to talk about is projection mapping so projection mapping used to be called spatial augmented reality and it's a technology where you can go in to hit play where we take everyday projectors just like the projector that's displaying this video I think something I already oh all right so projection mapping is where you take everyday video projectors like the one that we're seeing today and instead of pointing it at a flat screen you point it at something that's not flat that's a 3d object why should you trust me because I've been doing it for about nine years at Disney at Microsoft and in other places why not use a headset well the biggest reason is if you have a lot of people you need a lot of people to experience magic at the same point in time and so that's why I use projection mapping this used to be called spatial augmented reality then about 2008 artists in the media decided it was projection mapping the first known instance of this was at Disney's Haunted Mansion ride that was in 1969 using analog video projectors then in 1980 Michael may mark put a analog film projector on a turntable and then filmed a room spray painted everything light and then reprojected the room back onto itself this was then picked up by Ramesh Hrothgar at UNC who kind of invented the math behind projection mapping so scanning 3d objects and then using 3d models to do the projection then about 2008 this started to be used for building projections so this is producting on to the Sydney Opera House from obscure Digital this is Bon Jovi concerts this was on Twitter this morning so I threw it in this is called so net Lumiere shows which is really popular in Europe this is Redwood City so you know two towns over there's projection mapping every Tuesday night to draw traffic into downtown Redwood City besides projecting on buildings this is being used in the and live performances so we have American Idol and we have the Superbowl with Katy Perry two years ago it's you know San Jose Sharks have production mapping it's used in Broadway theater productions it's used for viral YouTube videos this was bought in dollies box and it can be used in retail so besides just doing events you can take projectors in here they're in nice little white boxes and then they're projecting information into stores so you have little birds hopping around a store you can take physical printed things and you can animate them we call this glitter for products so the idea is attracting attention to a product someone's going to walk up and touch that product they touch it there's 70 percent more likely to buy it you can just change the price for every product so same thing you can do with the TV you can do with projection you can do this for different product categories so consumer electronics fashion perfumes Nikes done this about four times you know nail polish you can use connects to make it interactive so when you pick up a product the visuals change besides inside of stores you can use this in storefront windows so this is Faberge egg at Harrods that's in the front window using I think six really expensive projectors so it works during the daytime this is another version of tennis shoes done by buck feet and besides projecting directly onto the product you can project onto the set that's kind of surrounding that product for visual merchandising you can also make it big so you can do what you used to do with caves and where you can just project onto you kind of arbitrary surfaces this was a cool project by Microsoft where they built this giant playground with Microsoft products and then you could sit inside of it use the products and then everything was kind of subtly animated this was for Spotify they took a wall of products they painted them white and then they projected animations on top of them so like there's a butcher knife that's animated on top of the actual knife it's used in architecture for like pre-visualization techniques if you guys haven't seen the sand table there's a sand table used for kind landscaping and studying water flow in the other room so this was Google doing more architectural visualizations you can use it in retail for more ambiance application so not just on products but on entire kind of wall displays these are paper cutouts that are then pasted onto the wall and then subtly animated with projection you can display information around a product that explains certain features and then you can do that by combining printed graphics with the projected light you can do this in rear projection in addition to front projection so this is a rear projected version of a virtual assistant you might see that in like an airport or train station and then going to where we're at today Expos and trade shows basically every single car when it comes out its launched with with projection mapping so this is a Jaguar when we're about to see a Land Rovers version of this you still don't see this technology in dealerships right but you see it in trade shows it's also used in other kinds of events so this is New York Fashion Week or launching the new Starcraft or you go to Disney World and you pay three thousand dollars to have a magical wedding cake so you have Tinkerbell that flies around your cake or you have the happy couple's initials that get drawn onto the cake or my favorite version pac-man cake it can be used in restaurants so the first example of apt chef is in the other room as well so this was done by skull mapping and they have a little chef who cooks a virtual steak on your plate while you wait for your dinner this technology is kind of replacing TVs so if you think about Dunkin Donuts Dunkin Donuts has four TVs of the menus Starbucks and every other cafe does not that's because they don't want to look like Dunkin Donuts so what you can do with projection is you can make menus that seamlessly blend into the aesthetics of the space you can use this in museums so you can educate and inform you can take existing sculptures from Michelangelo's day and animate them you can take you know models of cities and put information on top of them and this can be interactive so you can use depth sensors to enable multi-touch on any surface moving to art so we're going to look at a bunch of examples of combining printed things with projected things so not just using a white screen but using a colored screen so here they took a bunch of cutouts that they printed they pasted the model wall and then they did the subtle black and white animations what you get here is you get really high resolution because printed things have really high resolution but then you get to animate them with projection you can go really big so instead of printing you can paint you can paint a mural on the side of a building and then you can animate it you so as they're saying you can combine printed elements with projection that are very high-resolution the other version of that is wall paper so you can just put up vinyl decals and then subtly animate them with projection digital wallpaper that's connected to the Internet you can imagine a version of this where you have your Twitter feed on it as well so besides being ambience you can have practical information this is one of my favorites this is a bar and they have a magical enchanted forest in the bar that's subtly animated this is a version by joining the mercier where he use markers to draw like a mountain and then animate it with projection you can do it on paintings this is you can do it in corporate lobbies which is what this is going back to general Mercier and if you're feeling ambitious you can rent a generator and you can take your projectors into a forest so you can project onto trees you can project onto mushrooms you can project onto frogs this project was done by three hoons it's two guys from Germany and they spent a month in a forest shooting us setting up little IKEA tables and projectors and generators but the effect is quite magical this is a project we at light form did with G monk who's the animator that did the Tron opening sequence in the default windows logo we created a sculptural element out of it was a cnc'd material and then we animated that with projected light and this was at a gallery in San Francisco you can have a painting of the bay lights of the Bay Bridge that's animated via an ultra short throw projector this is actually a demo that's in our office you can also animate people so this is a cool project where they took people's existing tattoos and then they did motion graphics on top of them the one downside is you have to wear very little clothes when you make this effect work and moving into the future you can make these experiences completely interactive so this is a recent project from Disney where they're tracking an actor's face and then putting makeup on it in real time this project from Microsoft where you wear a projector on your shoulder and you can turn any surface a wall your hand into a phone and so you can dynamically takeover anything in the real world and turn it into a screen this was my work at Microsoft called the loom room the idea is you take a projector you put it on your coffee table and you pointed at the TV you already own now the game comes out of the TV and it comes into your living room so you can have bullets that fly out of the screen and into your living room this actually got a 3d model of your living room so you could do things like project on top of your existing furniture and compensate for the existing color and geometry so you turn your 50-inch TV into a 15-foot TV you can do things like you get shot in the videogame and your living room get shot it wobbles shakes and explodes you can have it snow in a video game and the snow collects on your bookshelf on your plants using a physics simulation in your living room and moving to a bigger version of that we did Microsoft's room you live project which was instead of one projector we have six so now every square inch of your living room becomes a game so here we have whack-a-mole like the arcade game but the moles pop out of your walls your floor your couch and you literally run around and hit them with your hands with your feet or with a gun which was an infrared LED that we were tracking there was six projectors and six connects so every person within this room was tracked you could do head tracking you could do view dependent rendering from any person in the room and we could also detect multi-touch both from your hands from your feet or you could even use objects like a pillow so you could grab a pillow to you know throw it at another player and that could be actually part of the game some really wonderful acting in this video by the way here we have a soldier the soldier is climbing down your wall across your entertainment center and across the floor and this effect was head tracked so without wearing anything on your face you can render a 3d object in the scene because the system knows where the person's head is we called this one indiana jones so the idea is you're searching for an idol and you're going around your living room and there's traps and the traps get sprung based upon your location in the room and you have to dodge them and have really bad acting as this character is running across the room we're using the normals of the 3d geometry to figure out where he should be oriented you can also do things like toss a tennis ball back and forth between two people so here we're tracking the hands of the person in real time this is cool work by Ishikawa's lab where you're projecting onto a super high frame rate capture of people and then reap rejecting on top of them a little bit about how you do projection mapping well how you do projection mapping today is you use video based tools like After Effects and you can use projection mapping tools like madmapper d3 etc usually have a like a grid and you have to warp your content onto the kind of the building that you're projecting onto basically it's a horrible process which is why we invented light form light form is a camera and a computer all-in-one it turns any projector including the one we're using today into a 3d scanner you take light form you point in any object and within seconds you have magic that exists in the real world so we use those same kind of effects we did at Disney at Microsoft and you can within a single click make the edges of objects glow and pulse to music for instance where this technology really becomes interesting is when projectors go from being ugly to really tiny which is going to happen within the next three years if you want to learn more about projection mapping go to our blog projection - mapping org and then we end on which future do you want to live in which is do you want to be surrounded by screens or do you want to be able to through projection turn any object in the real world into a display but more importantly be able to turn it off and with that I think we're exactly at 50 minutes is there a question I yeah talk to me afterwards now I can just give it you yeah so light form works with any projector so it actually just talks to the projector over the HDMI cable and so it's a smart add-on for a 200 dollar Pico or $100,000 projector for projecting on the side of the building all right with that we'll get to the last talk [Applause] [Music]

Channel: AWE - Augmented World Expo

Views: 46,663

Rating: 4.9886847 out of 5

Keywords: Augmented Reality, Virtual Reality, AWE2017, Augmented World Expo, Lightform, Brett Jones

Id: EGw2JOx11IM

Length: 16min 3sec (963 seconds)

Published: Thu Jun 08 2017