Here's why Virtual Production Screens are taking over Hollywood.

shout out to Squarespace for sponsoring this video what's up everyone my name is Sarah DC it rhymes with peachy and today we are talking about virtual production so some of you guys might have seen the behind the scenes of Mandalorian when it came out three years ago that was a huge uh big volume virtual production stage and so we're going to talk about how that actually works um but also I'm not in my office right now I'm actually here on a virtual production stage with Sonia NAB where I can do things like this so we are talking all things NAB but more importantly virtual production so you're going to see a lot of different types of these led walls and I'm excited for it so let's get into it so yeah in a b yes I know it was over a month ago apologies this video is a little bit behind I realize that when you go to conferences you just shoot a lot of random stuff Maddie how's the NAB event so far uh it's fun Hey look it's Gerald and done wait who's filming this am I why am I holding your camera I am Floyd Gerald he's going to be my new filmer so let's go check out NAB we're good we'll get some action shots and YouTube does not want random stuff YouTube wants you talking about one subject so it just took me a minute to put this together but one thing that I notice is virtual production was everywhere every single Booth it seemed like it just had something to do with virtual production it was so cool to be on Sony's virtual production stage do you want me to to stand on a box I tried a vuse virtual production setup on the motorcycle okay this is exciting [Music] stupid and turns out it looked stupid I wish I actually sent out the full horizontal video they were shooting with such a fancy camera but of course everything now is to just share vertical and social so that was a little bit of a bummer but because the motorcycle wasn't moving and the camera actually wasn't tracking with the background and the fact that it was a super like digitally background is what led it to you know make a cool Instagram story but not the best actual example of virtual production and we're gonna get into that in this video you know what makes virtual production cool versus maybe some things that just fall flat and speaking of falling flat when I was researching for this video virtual production obviously has a lot of positivity around it because it is a cool futuristic technology that well has been around in Hollywood but it hasn't really been perfected and I started to learn shows like Mandalorian that really put virtual production on the map in 2020 didn't have the easiest time with it it really isn't just oh you just figure out everything in pre-production and then you don't have to do anything to it after I found all of these breadcrumbs across the internet about Mandalorian not even using the backgrounds that they shot whoops okay fix it and post right we got we know how to do that I don't know man is it all a cover-up but some of you guys might be like Sarah what the heck is virtual production in short it's just a bunch of LED panels that stack together and make a bigger LED panel and it could be the size of a very large wall or it can be curved to create a ginormous volume and on that screen you could have a multitude of things you can have a 2d background and then put physical props in front of it so that's what I did to film the intro of this video with the fx6 I shot a blank plate of my office without my desk or chair and then we inserted the desk and chair at NAB and so if you want to take a step further you can have a fancier looking 3D background this plate at NAB was shot of course with a fancy Sony Venice cinema camera as you can see the video is super crisp they shot it with a lot of resolution and because it's a town you have the leading lines of the road and it just looks more realistic when you put that subject in front of the background and then the next step is where we get really fancy and this is where a lot of virtual production have okay so you guys have seen now two examples of me with 2D backgrounds but where it gets super cool is when when the camera moves also the background can move so this is where you have something called the frusto this is when you start introducing live camera tracking so there's a sensor on top of your camera and then there's a ton of cameras up above shooting the environment and that information is then relayed to the background so what you're putting on the LED wall behind you is a complete virtual environment that was built in Unreal Engine so all of that information from the camera you're shooting your subject with to the cameras that are shooting the environment get sent to your LED wall and as you move the camera to the left your frustum changes so this camera tracking allows for this proper Parallax effect think about it I'm looking at my camera right now and say my eyeballs are a sentiment camera when I move to the left I start to see more of the right side of my camera right because I'm moving in a 3D environment and so that's what's happening with live camera tracking so your frustum is actually changing as the camera is moving it is so cool and so that's why it can look kind of fake and not good at all when you stick a subject right in front of just a flat screen if the camera is moving but the environment is staying static behind you well that's not like real life right that's just gonna look fake oh you're shooting in front of a screen Mandalorian uh combined all of it to make it look so cool because they had physical props they built these entire physical environments in front of these huge volumes and so they combine both practical things and the virtual production wall behind them to get a really cool end result so a lot of the work that visual effects artists would do in post-production with you know a green screen has actually moved to pre-production making all of these digital assets you can design an entire world in Unreal Engine from from the ground up or buy 3D assets from different 3D model stores right and then stick a subject in front of it and then explore that world with the camera and your real life subject it's pretty it's pretty crazy when it's done right at Sony's booth at NAB I actually got to chat with a DP and focus puller who have been on these sets what I think is really interesting is for example the plates we shot here we shot them in Montana in the snow in somewhere that might be hard for the production to get to especially if it's just for a couple days and to bring that into this environment where we have control there's a lot of freedom there so there are a ton of positives with virtual production you golden hour right so instead of only have having 30 minutes to shoot that perfect scene right you now have full control to shoot that scene however many times that you need to in order to get it perfect because you control 100 of the lighting and the environment everything you're able to really quickly transition in between different sets and different scenes Reflections on costumes and objects reflect the actual environment so that saves a ton of time in post-production I don't even want to know how long it takes the Marvel people to clean up all of the green spill from the green and blue screens another point was how much money you save from not having to scout all these locations and traveling to the locations however I feel like that's just substituted for the pre-production time visual artists have to spend actually making these environments though so I don't think you're saving time there you're showing up on set knowing exactly what the weather is what the colors are going to be the only variables you have to worry about is actors acting and speaking of actors and while everyone on set they are just so much more immersed into the actual story because they feel like they're in it yeah there's definitely times where you're in the in the moment and in the shot like either playing Focus or operating and you kind of peek your head out and real like you forget where you are for a moment I mean it's pretty funny you can be you're really in the shot and then all of a sudden you look around like oh yeah it's this isn't real so the line is really blurring in between mediums right Unreal Engine is usually used to build video games they cross over into film is actually pretty cool I mean think about it you could potentially Port over the same environments that you made for the movie straight into the video game that then users can then explore for themselves right there's there's many possibilities just beyond just shooting a TV show or movie but there are problems it's not perfect does it actually save time when it seems like a lot of these Productions are still having to spend a ton of time in post-production fixing things because well taking a camera and shooting it at a LED will has some issues uh take your phone camera and just shoot it at your TV you're gonna get some more you're gonna get some some weirdness right and that comes up and what's so interesting as I researched this more I found all of these breadcrumbs across the internet about Mandalorian not even using the backgrounds that they shot with this wall apparently they still did a ton of rotoscoping in post-production if you don't know what rotoscoping is it's basically just masking your subjects out from the background and inserting another background which is not something you would think you would have to do with virtual production because that's kind of the point right so you still get some of the benefits right the actors were right in the action uh the reflections that were happening off of the very reflective armor of the Mandalorian were still accurate to the actual and environment you just couldn't have a costume like that if you were shooting green screen but it's just kind of interesting that Mandalorian led the church with his virtual production thing and still had to spend all the time in post-production fixing a lot of it when that really is the biggest selling point I don't know man is it all a cover-up is is it big virtual production and it really just started from someone's comment at NAB saying oh yeah you know Mandalorian and they had a rotoscope out almost all of the backgrounds and put in high-res versions of the backgrounds and that led me to Googling and then you know I would see a tweet here or a YouTube comment there but I could not find any official news articles or interviews talking about it just redacted Reddit comments and little threads here and there it's interesting it's very interesting so my hypothesis is yeah the first season of Mandalorian they tried out new tech and it didn't work out the way they thought it was going to but all of their newer Star Wars Productions are still using the same technology so my guess is ilm who did all of the virtual production from Mandalorian was like whoops okay fix it and post right we got we know how to do that and they see this technology as something worth investing in and learning along the way and I guess it might just be very very messy up front because it takes a lot of money that Mandalorian screen was like a 30 million dollar wall how do uh visual effects artists feel well they have a huge part of it like we can't do this without them a lot of the time they have to help build these worlds they have to clean up the edges I think it's a hand in glove thing with visual effects I don't think they're antagonistic at all and so VFX artists don't worry you still have a job the tools might be changing but all I've learned in my research for this video is wow it still takes a village Disney has another Star Wars project coming up that is using this Tech and you know it looks super cool I mean sometimes I feel like everything is just getting a little too crisp and so you know sometimes that makes it look a little fake but nonetheless you know looks looks better than this so going back to the issues what specifically are those issues that led to all of that rotoscoping number one more so this happens when you have clashing patterns in between a screen and a camera Moray is the main issue why Mandalorian had to redo a lot of the backgrounds this is why a lot of virtual Productions have to have a very large space to shoot because you don't want the camera to get too close to the screen so the distance in between between the screen and Camera can affect more but also something called the pixel pitch distance the smaller the pixel pitch means that the pixels are closer together the closer the pixels the better the image is and the less Moray that you're gonna get so the closer to the wall you can actually shoot which then means well maybe you don't need as big of a wall because you don't have to shoot so far out this is actually a part of the big bet that Sony is making so um it is masaki I'm in charge of Sony's vulture Production Studio it's called Sony Innovation Studios as a stage wise I'd say we are not the biggest station in the world but resolution wise most likely one of the highest resolution we have essentially we want to change the perception that some people think that vulture production you cannot shoot the Box you know the background and that may be combination of the asset quality and or you know the display quality we want to change the perception even if virtual production you can you know focus on the background because you have high resolution panel and that's something we have been you know trying our best to solve they're making walls with much smaller pixel pitches than in the industry standard the industry standard is above two millimeters and while their new walls have a pixel pitch of only 1.5 millimeters another issue is color and something that you have to correct and fix and post a lot of the times the color of the LED wall is not going to match perfectly with the color that the camera is shooting and then another issue if you're shooting with real backgrounds is actually having cameras that shoot high enough resolution for the plates so think about it if you have these huge LED walls well you then have to have cameras that shoot super high resolution to fill up those walls of course that's not something that you have to worry about if you're building a virtual environment an Unreal Engine but if you're going for a real environment you need a lot of resolution to make it look real and then last but not least cost which I don't see this problem being solved anytime soon but this virtual Production Studios are charging day rates of anywhere from like 40 000 to a hundred thousand dollars a day because not only do you need the wall all but you need the talent that can utilize it in the past three years a ton of companies have popped up so maybe competition will bring the costs down maybe I think the company's entering the arena are kind of playing with All or Nothing mentality they're like no we need to develop our own proprietary technology that makes us the best so we are one of only three or four Production Studios that then gets acquired by a Disney or a Sony Etc and I imagine these will be kind of like their own Technologies within specific Studios and this really isn't going to democratize filmmaking at least for the next 10 years because in order to make this look good you need the money the money and the talent and the expertise bringing it back to Sony I think this is why Sony is really paying attention to this space one they have the money to invest two they see these issues and then three they're actually positioned in a very unique place to solve these issues Sony both makes LED wall and Cinema cameras and well would you have it they make huge budget movies and music productions that can then benefit from this technology so you know move over Disney there's another big man on campus and this is a reason why Sony cine was so focused on this at NAB not just because they have their new LED walls with the minimal uh pixel pitch but they also released something new called their virtual Production Tool Set so this includes a virtual Venice Camera a display plug-in and a color calibration tool that works directly within Unreal Engine so the display plugin allows virtual art departments to have access to the venice's color pipeline remember Venice that's like the fancy cinema camera right so they have access to the Venice color pipeline during pre-production to create all of those assets that you need before you show up to shoot this plugin also creates an alert system both in pre-production and while you're shooting when it detects Mooring because oftentimes you're not going to see that in real life and then with their color calibration tool the colors on the wall and the colors in camera will also match up perfectly so you don't have to fix that in post so all of those tools seem like a great step to get virtual production one step closer to what everyone thought it was when all of those Mandalorian behind the scenes videos were coming out you know truly capturing everything in camera I don't think it's gonna trickle down to us normals for a while but I did see a little bit of that at NAB it was actually pretty cool before I send you off with that clip Squarespace thank you so much for sponsoring this video Squarespace is the all-in-one platform if you are a artist who wants to show off their portfolio to the world to get more clients if you run an e-commerce store if you have a physical location where you need to book appointments they have all of the tools that you could possibly need Squarespace is always adding so many features and guys the end of last year was a huge update they introduced the fluid engine and it changes everything you can use one of their beautiful templates to to get started on your website or you can go with a fluid engine which is essentially a drag and drop website builder where you can put photos different blocks email newslettered blurbs anything wherever you want on the website so building a website on Squarespace is now 100 customizable and you can adjust things on how it'll look based on if someone's viewing your website on a desktop a phone a tablet the sky is the limit now and Squarespace has always had amazing third-party extensions that help take your website to the next level if you're selling physical Goods digital Goods anything under the sun I just opened up a printful store that integrates into my Squarespace website for Merch it's just there's so many things and now I'm finding so many more fun things like there's this plugin that changes the cursor to a peach Emoji I don't know if you guys can tell but I'm very excited for the Sarah dichi.com 2.0 all on Squarespace so whether you need to do scheduling for a space rental or maybe you're a creative and you want to show off your portfolio to clients or you want to start kind of a patreon-like community but on your website sell physical Goods sell digital products you can do it all with Squarespace it is time to build your beautiful Squarespace website if you want to check it out go to squarespace.com for a free trial or go to squarespace.com Sarah DG that's me for 10 off of your first purchase of a domain or website okay so if you're sitting at home and you're like well how does this affect me I I watch the content I consume the content it doesn't matter if it's green screen or virtual production who cares well this concept of shooting something in front of a screen isn't just for the big guys I've actually shot b-roll of technology in front of my Samsung qled 8K TV and you know it turned out pretty good now that TV being 65 inches doesn't make it the best for that you know usually you want a little bit more space but this is really not a foreign concept and I love seeing what potato jet did with his new studio he essentially just got I think like three TVs put some curtains over it you know using physical props in front of a in front of a screen and then he can change his background in his videos to whatever he wants and it literally looks like it's a real background and you're just looking through a window it is so cool there were a lot of really cool virtual production demos at NAB but this one was definitely one of the coolest and I feel like something that you guys could probably use like right now we have the aksun CMO Pro here which has an SDI and HDMI in book so we're taking the video signal from the camera out into this device this converts it into a stream that then goes to an iPhone or an iPad or iOS device and what we're feeding here is an iPad Pro running a Beta chromalink app and then we have these tracking points here that are tracking the colors in the shot this one's tracking up here this one's down here and they're controlling two pro like lights up here the color of the lighting is following the color of what we're shooting and it's kind of a poor man's virtual production it's just a TV that's all it is I think a lot of people are doing this now because you can do car shots you can do all sorts of really clever shots just with a large TV and when you have the ability then to track the color of the lights so if it was a carshot process shot as the car goes under the trees and things like that the light would change and that's one of the things that really makes virtual production much more convincing is is when the foreground lighting matches the background light oh okay these videos always always turn out to be longer than I think did you enjoy it I know it was the kind of out of left feel but I found it interesting so even if only 12 people are watching this video hello Hi how are you again Sarah peachy like that like button hit the Subscribe button down below for new videos every single week and until next time everyone stay peachy okay bye [Music]

Channel: Sara Dietschy

Views: 44,371

Keywords: virtual production explainer, virtual production NAB 2023, virtual production sony cine, sony virtual production announcement 2023, sara dietschy virtual production, sara dietschy tech event recap

Id: pyyaNU7kBNQ

Length: 20min 59sec (1259 seconds)

Published: Thu Jun 15 2023