Narrator: While filming this scene from Disney's "The Mandalorian," the actors could see their surroundings, but the surroundings
weren't actually there. All of this is just LED screens displaying backgrounds pre-made
in a video game engine. Compare that with this fight scene from "Avengers: Endgame." Where actors jumped
around in a sea of green, imagining how VFX artists
would make this planet look once filming had ended. "The Mandalorian" is one of
the first major productions to choose LED walls over green screens. And the benefits for the actors are just the tip of the iceberg. LED walls make the lighting
better, filming smoother, and in certain cases, cost a lot less than using green screens. But to understand why the
team behind "The Mandalorian" chose these LED screens, we have to understand just how they work. "Mandalorian" showrunner Jon Favreau revolutionized virtual
production while directing "The Lion King" and "The Jungle Book." However, the process for these two remakes still relied heavily on blue screen and post-production work. For "The Mandalorian,"
LED wall technologies seemed like the next logical step given the show's production
budget and time frame. Now, you may be thinking, 'This isn't so new, I've seen
something like this before.' And you're right, kind of. The predecessor to what we
see on "The Mandalorian" is a driving scene like
this one, from "Dr. No." You've got the actor in the car and behind them, a screen with footage of the road they've traveled. But the technology was limited. Say you want to move the camera angle during the scene. That projected footage can't move with the camera. But by using Unreal Engine, tech borrowed from the video game field, that problem is solved. Artists can create a
photorealistic 3D background that moves strictly with the camera's field of view, known as the frustum. So if the camera swings
around and changes angles, the background shifts in
precisely the same way. This allows motion-tracked cameras to execute traditional
cinematography techniques within the virtual set, achieving cinematic movements like the parallax effect, where
an object in the foreground moves at a different
speed than the background, amplifies the illusion of filming
at an actual location. Now that LED screens can
move with the camera's eye, virtual sets can solve a bunch
of green screen problems. The biggest one probably being lighting. Richard Bluff: We've all
seen movies, TV shows, where the lighting on the day didn't necessarily match
the post-production work that was added many, many months later. Narrator: That's Richard Bluff, Industrial Light and
Magic's VFX supervisor for "The Mandalorian." He says lighting is
one of the key benefits of working with virtual sets. The light coming from the LEDs provide realistic colors and reflections on the actors and props, something you simply can't
achieve with green screen. They are also able to completely
avoid the problem of spill. To better understand that, we talked to Kim Libreri at Epic Games, creators of Unreal Engine, which was used to build the virtual sets for "The Mandalorian." Kim Libreri: The problem
with the green screen is that you have to have
a green screen behind you. And what that does is
that instead of projecting beautiful lighting for
what the environment would be behind you, it basically puts a lot
of green light on you. And we call that "spill." If you wrap an actor with
a big 360 LED wall, you can light in a way that you would never be able
to do on a green screen. So you can really make it feel like the characters are
embedded in their environments. Narrator: This green screen spill would've been an even
bigger problem than usual for "The Mandalorian,"
whose main character gets a shiny new suit of armor early on in the season. So realistic lighting from the LEDs makes the show look better. But virtual sets help with
practical concerns too. Take this scene in the
office of The Client, played by Werner Herzog. In a scene like this, surely they just built
a simple set, right? Wrong. All of this is that same LED screen. According to Richard, only the desk, the floor,
and one or two columns were really there. Richard: Everything else
was displayed on the screen. But this allowed us to limit the amount of stage space that we used in shooting season one of "Mandalorian," which has, understandably, a
big impact on time and budget. Narrator: To be fair,
the initial investment in a virtual set like this is huge. However, the creators
claim that money saved from traveling to locations,
building new sets, and costly post-production, can make the investment worth it. Along with saving time and money, the team developed a system that allowed artists to make changes and control the world
on the day of shooting. Settings like exposure, color, animation playback, and fill lighting are available to the filmmakers
at a moment's notice. If they want to move a mountain from one side of the virtual set to the other, they can,
right there on set. Not only is it helpful for the actor, who no longer has to imagine
where that mountain is, but it's a game changer for
the director of photography. Richard: Because in the distance, if there's a mountain range
that is a story point, then the DP's no longer guessing where those elements are, he can frame up to them. Narrator: The virtual set
constructed for season one was 75 feet in diameter, 21 feet high, and also had a roof composed of LEDs. The cast and crew referred to
this space as "The Volume." Richard: And we had doors
that can kinda close in to almost give you full 360-
degree coverage of LED screens, minus the floor. It's driven by a small group of machines running Unreal and our
StageCraft technologies. Narrator: At the end of the
day, the big question is, do virtual sets really mean
the end for green screens? Well, even "The Mandalorian," with it's technological advances, used a version of a green screen, but via the LED wall. Another capability of virtual sets is to choose pinpointed areas of the wall and turn those green. Richard: Now, we now
have the ability to limit the amount of green screen that's visible behind the character. So now it doesn't wrap
around the entire stage, it's just a little piece, but also, they gain the lighting
from that environment. Which gives us a more seamless result in post-production. Narrator: Additionally,
scenes involving explosives are best kept to traditional methods. This is to prevent LEDs
from getting damaged. So will green screens survive
this wave of tech innovation? Richard: Eventually, of course, we hope never to use green screen. But, I still see that there will be a future for it in the short-term, because there's likely always
a need to remove people, to add additional action behind them. But we are getting to the point where the amount of green
screen that's being used is massively reduced and the sky's the limit right now.
Definitely watch the Disney Gallery: The Mandalorian for all the behind the scenes on this show. It's amazing the stuff they did!
Damn, technology really is amazing.
Seeing the technology used for alien landscapes and generic forest backgrounds was neat, but I was shocked when they showed The Client's office was rendered rather than real. It had never even occurred to me that they didn't just build a little office set.
Rewatching the scene knowing the tech, it took a little while to recognize the little imperfections, like the shadows on the central pillar change based on camera angle or a couple of times when the non-focal actors were in slightly sharper detail than something they were supposed to be standing directly in front of or next to. It's really minor stuff that you wouldn't catch watching this normally.
The video states that the upfront costs are significant, but that's not a big deal for Disney, especially if they're able to re-use the same tech for their Disney+ shows.
It's a great time to be an environmental artist working in Unreal 4 soon to be 5.
The key here is it has to be real time in order to take advantage of the parallax effect on the horizons.
You could technically do it in rendered CGI but you'd really be limiting the creative freedom of on the spot changes.
NOT TO MENTION
The screens actively work as light sources which are practically as effective as being in the real environment.
Honestly this stuff is massive and I'm super excited for what it can achieve.
What's cool is you can go back in film history and see examples of precursors to this sort of stuff. The Dawn of Man segment in 2001: A Space Odyssey was shot in a studio but used a cool front-projection technique so that everything was in-camera. The foreground is studio, and the background was photograph plates shot in Namibia for the film.
the scene https://youtu.be/ypEaGQb6dJk b-t-s: https://youtu.be/AgNyCluIRhA?t=237
Once again, video game propelled tech saves the day! I always thought the Hollywood studio warehouses were crazy because of how much dynamic filming they could do in a single building, but this is like that on steroids. Sort of like the 'VR headset' of film-making- in that the device gets smaller, but the illusion gets bigger. The future is scary and wild, but this (graphics-tech/optimization/innovation) is definitely one of the aspects I appreciate most and am looking forward to.
That's genuinely one of the most fascinating videos I've watched in a while. Crazy to think how so many of these huge scenes were filmed on a tiny set.
Does the ceiling screen display as accurately as the side screens? It seems like it gets distorted at hard angles.
Do the screens have physically accurate contrast like an OLED, where black is truly black?
How do you light a scene with direct sunlight, instead of just cloudy ambient lighting like alot of these shots?
Otherwise, this is quite a game changer.
Are we at a cross roads in modern film production?