The technology that’s replacing the green screen

[Music] you know the green screen and blue screen you know it becomes the missed covered planet deserted jump background [Music] what happens when the imaginary planet is already there [Music] the moment you step in the middle of the volume you're just you're just there the volume is the epic sounding name for the combination of high resolution led panels imagine awesome huge tv screens wrapping around a stage the physical set design matched to the panels and 3d models plopped into an environment the same way they do in a video game then it can respond to camera movement to simulate the real world disney pluses the mandalorian a live action star wars tv show used this technique my name is charmaine chan and i'm a lead compositor at industrial ida magic is it physically like confusing being on on this set oh absolutely i mean the thing is like you're you're shooting all day let's say in the same exact scene and like you're at that location it doesn't feel like it's something fake it just feels like an extension of a regular set stage you know you got to be careful because there are times where people don't see where the end of the stage is and where the leds are you have like wiley coyote and roadrunner situations where somebody's like running into the wall or something yeah we definitely made sure that no one's running in that stage because of that reason charmaine is credited as part of the brain bar the group of visual effects artists that operated this system one might adjust models like a rock or a spaceship in the panels while another might tweak live animations like a burning fire charmaine often adjusted color it was funny because it looked very much like you know back in the days when you would have telephone fundraising stuff like on pbs it was just rolls of people with phones ready to go but instead of phones we had computers in our walkies as mandalorian vfx supervisor ian milham tweeted the set crew and brain bar operating the panels let them radically change environments in just a few hours or beyond set as they launched it into hyperspace [Music] my normal working life is very much behind a computer in a dark room somewhere in the corner now i'm actually in there with the gaffer with the prop designers with the set designers most people that we would never see because we're in the post-production process it was very accelerating but sets like this one weren't just fun for charmaine they helped to remove creative roadblocks as a compositor we're the ones who kind of take all the renders take all the cg elements and put them together to make it look like it's a seamless integrated photo so think of it as like advanced photoshop but for dealing with moving imagery charmaine worked on this scene in the last jedi we get this footage of kylo in front of a green screen if you're lucky this green screen will be evenly lit with no seams and it's a piece of cake that's never the scenario spending the time almost frame by frame making sure we can remove that green screen so that we can put kylo on top of that removing a green screen is actually still pretty hard for one it doesn't work with green characters yoda's green removing one solid color or keying can look good but you still need detail work see how these fine branches just disappear perspective of the background also doesn't naturally change that has to be designed into the final composite ditching the green screen and projecting or playing the image behind the actors gets you closer but not quite there [Music] [Applause] you can get detail in an illusion of depth and better light instead of green screen spilling on the actor you get blue sky and red desert actually lighting them that basic technique has worked in everything from 2001 to oblivion but you missed the proper perspective shift or parallax in the background since it's just a video playing on a screen the volume tackles some of those problems you can also adjust light and objects on the fly and the reflections actually work because they are reflecting the other screens instead of a green screen which was especially important for mando the show's main character his whole armor was reflective from head to toe whether it be his pauldron or his helmet it was just like you can't avoid seeing things being reflected [Music] so creating this volume where we literally could close up the whole thing into one giant circle and have an environment all across these screens we were getting exactly what we wanted to out of his helmet the brain bar could focus on details that made the final product as seamless as possible which was still a lot of work i would go in and whether it be a rock or a barrel or something i would try to color correct it to match what was on the set but where color correction was more important was when we're dealing with the bigger parts of the set so whether that be the dirt on the ground versus the dirt in our digital scene and the lighting from the scene affected the dirt on the ground so then we would have to like because we had a blue sky and suddenly now there's all this blue on this rock we would have to color correct the ground and the rock to also have just as much blue as the blue that we just introduced before they started shooting i would have five to ten minutes to get that all lined up and ready to go i can imagine that there are some creative breakthroughs that this makes possible for for your job what would those be with this technology i'll be honest i would not be mad if i never have to do a green screen keying or extraction ever again now i get to be a person who's doing the shot and i can help basically finalize a shot in camera it just makes it a more um cohesive filmmaking process and this puts us right in there next to everyone else who's creating these shows or films it was great to talk to charmaine and learn a little bit about her work and some of the amazing things that she's worked on this video is actually from a sponsor which is verizon they just turned on 5g nationwide so with 5g nationwide and in more and more cities you get the unprecedented performance a 5g ultra wide band it's really fast the world's fastest 5g you can download an album in a few seconds it's not just going to change your phone it'll change everything i'm guessing that a lot of creative breakthroughs will come from it too so this is the 5g that america has been waiting for and it's only from verizon verizon doesn't directly impact our editorial but their support makes videos like this [Music] possible

Channel: Vox

Views: 4,946,424

Rating: 4.9270911 out of 5

Keywords: Vox.com, vox, explain, explainer, green screen, mandalorian, the mandalorian, yoda, mando, charmaine chan, industrial light and magic, ilm, vox almanac, almanac, phil edwards, vfx, visual effects, green screens, blue screens, stagecraft, led panels, led walls, computer graphics, cg, cgi, unreal engine, star wars, disney, disney plus, lucas, technology, computers, special effects

Id: 8yNkBic7GfI

Length: 8min 23sec (503 seconds)

Published: Fri Oct 23 2020

Reddit Comments

[u/xBreadButta]

On Disney+ there’s a Doc. On making the mando. They show a lot of this new way to film.

[u/gin_and_toxic]

Better video of the technology: https://youtu.be/Ufp8weYYDE8

[u/sprace0is0hrad]

Maybe it was just a placebo kind of thing, but I think I can see the curvature of the stage when watching The Mandalorian, and the sets seemed to have a circular arrangement too. Not that it's a bad thing I guess.

[u/phil035]

Tom scott did a gameshow series using one of these stages really intoresting tech behind it

[u/kadse_rekt]

The whole idea of "stagecraft" is amazing and definitely a cutting edge technology for big budget movie making. I think the thumbnail and title is misleading. To use this tech there are many restrictions to actually getting it to work in post.

Virtual production sets will be another tool, but NEVER replace green screen.

1. You have to be quite far away from the LED panels, otherwise you get the so called moiré effect.

2. Using LED panels on the floor are a pain in the ass, talking from personal experience. Set building will always be a crucial part of Hollywood!

3. We talk about illuminating panels, replacing a door for example in post will not work without a green screen.

4. It's super expensive and hard to get working. Syncing everything in real time, LED flickering (fps <-> frequency) and so on

It's amazing what ILMxLabs created and I'm really excited what big budget movies will accomplish with it! Don't be fooled, green screen will always have a vital part in visual effects!

[u/platedserved]

This tech and the use of light-up lightsabers in the sequel trilogy to cast real light and reflections on the actors and physical set are the best and most immersive things to happen to Star Wars productions in a long time. Star Wars effects are emerging out of decades of poorly composited cgi backgrounds and looking exciting again.

[u/J0hnR0gers]

I think Lazy town did this first

[u/NeonSteeple]

Weird question: have they said what the pixel pitch of the LED screens are?

[u/Petunio]

Kids, learn Unreal Engine. It’ll take over all things 3D.

...unless you are actually using it to make a 2D game, for which Unreal is godawful at.