Indie Virtual Production | Real Time Key in UE4

hello and welcome to this virtual production vlog my name is Matt workman and we are shooting in our virtual spaceship today [Music] now this is a live rendered background let me show you here oh not the best camera move but I don't have an operator with me right now but it's a live rendered 3d set in Unreal Engine and we're also doing the compositing which I think looks pretty good it took me a couple days all on Unreal Engine and this is like pretty much what we're looking at right now quality wise as far as my virtual production studio so in this vlog we're going to talk about the changes I made to my camera system we'll talk about what I've learned about live keying and lighting and we'll end by showing some a handheld operating shots so we can see little bit more dynamic movement look at the tracking in the sink and also the Kim all at once and most importantly what does this look like how does you know in the virtual production look at the current state that I'm in I think there's still room for improvement but I think the results are pretty awesome looking forward to sharing them with you pipeline is is that we shoot through the Blackmagic g2 and we output the footage through SDI back to Unreal Engine Unreal Engine does the 3d rendering of course and it does the live composite which we'll talk about in a little bit and then we output the live composite back out through the deck link through HD SDI or SDI and then we put it into here so I can see the live composite now through this monitor and I actually record from it as well now there is a little delay I haven't measured it it does feel like 812 frame something like that it's not great so you'll see in later demos if I move there's a delay with this but in most cases I'm okay I can I can compensate and it's really obviously helpful to see the final composite right there and for me because I am camera Department you know DP former camera assistant I want to keep as much camera Department as possible you can record SDI onto your computer onto all sorts of stuff but I want to record SDI right here like I'm just rolling the camera keeps everything very camera Department and it keeps things easier in my head they're not gonna have to go roll in the computer and go over there all I have to worry about in this virtual production setup is recording onto this monitor and then when I'm done I take out the SD card and I have the footage it's almost like Unreal Engine isn't there right so I like to keep all the pewter stuff kind of separate but keep the film stuff all together and this makes me really happy to be able to record see it do waveforms put it in like frame guides hit stop give the media to the DI T or the downloader who is me in my case but it keeps it all very camera department so I think this is like a really good way to go about it okay so this is standard shoulder style operating and the monitor really needs to be like this far out so this extension I'm putting it like all the way at the end at the top handle extension and it just gets the monitor where I can see it like this if I want to go into like hip shooting like this which most people will probably use like an Ergo cine or something like that I actually just grab it by the side handle here which is like nice and strong now I'm holding this and I'm pushing the this like into my arm here it's like a gun stock pretty much and I just go like this I usually flip the monitor and I can still pull focus with my thumb here I don't know if you can see that but it's not ideal but you know we're India here we don't have a follow focus I don't have a focus puller with me but I can get it done right so if I'm shooting hip style I'll go like this and I can show you in this frame but you can imagine if I'm doing low angles I'll just hold it like this a one-hand like a skateboarder and I'm pretty much set up and comfortable to be able to switch between these as I need to with kind of minimal setup or need for assistance with cinema rigs when you're switching between different modes it's like easier egg no easy rig and you know it takes a couple more people usually to kind of get it set up this I'm pretty much good to go so that's the handheld setup let me put this back on the tripod here so that's the demo of the rig a lot of it was for like sooner ergonomics so that I can operate this I'm gonna do a lot of hand held some tripod I'm probably gonna get a jib or slider next we have our primary record monitor his right here we can see the final image we have room for the tracker and we have room to grow as we add Wireless follow focuses other computer boxes as needed for tracking and we have our battery once we need to power everything so this pretty much gets you up and running for indie virtual production but we took the g2 and re rigged it so that it can scale into I would say like upper indie production I think once you end up on like a big virtual production shoot you probably switched to an Alexa or a camera of that caliber bigger support you know 19 ml everything's gonna get even bigger but this is still 15 ml and they'll pretty slim and Indian this is our virtual set it's live it's basically a video game it's pretty awesome you don't have to render this so to speak frame by frame you just look all around you real-time stuff is awesome so this is our live video and I'm gonna give you a quick overview on how this works if you want the really detailed version I will link Andy Blanton's tutorials in the description below this is a quick run-through so to do this we're going to be using the composure plugin you go up to editor plugins turn on composure boom so this composure plug-in is basically like After Effects so this is a new composition and we're gonna add a new layer right and I'm gonna add a media plate in this case so this is your first layer here on top and we're just going to go and select my live video feed this is live video coming in through the black magic card and now we're going to key it so I'm going to delete these kind of like default filters here and we're going to be using the new color difference key R here so I'm gonna write color diff there it is yours might be called something different but there we go I'm gonna just change a couple settings here both the pre pass click on this thing and we're going to just kind of drag and average these colors together click and drag except I'm going to do a little bit of a threshold here to kind of crush that down and you can view just the Alpha which is helpful and I'm gonna change the Alpha offset here like this to completely make this white I could probably go even further you have to tweak this depending on what you're shooting and there some other settings as well this will get you pretty far so if I turn these back on like this we've now cut out oops cut out our character right really nice key there's a lot of tweaking you can do to do it better but uh that's a good start the next thing you want to do is add another filter basically these are like filters the way I think about it so we're gonna add another filter here and to pick your filter you go here and I'm gonna right d spill average green-screen one that one so that kind of kills all the green spill on them on the foreground does uh does a lot really helpful and because of the the way that the color comes in from the Blackmagic camera it's a little bit flat right so I'm going to kind of grade it so I'm just gonna add another filter or material pass and the one I want in here the way you find that is to go to let me find this thing the composure content materials and color grading yes this one right here drag it in or you could type it if you remember the name and now for my case I'm just gonna go to gamma and just kind of bring the gamut down like that okay so essentially we graded it we distilled it here and we keyed it from there and you can do this much better if you spend a little bit of time and tweak it but look at the results we get so far not bad so we just want to add one more layer here and it's going to be a CG layer which is the CG camera that goes in the background and we have to give this a material that I am NOT going to go over how you make said material and these n DS tutorials do that as well this one should be fine hold on I just need to add it here and drop this into there but boom boom we're going to tell it to use media plate one for the top and CG elements for the back and there is our composite really quick super fast composure tutorial right there getting the tracking and all that stuff sync together that is a whole different can of worms there but that's composure you can get some nice live composites going just wanted to show that process really quickly and the other thing that we are doing now that's new is that if you click the comp itself you can actually do an output right so if I use this I already have it set up if you set this all up you can output through the deck link and this is how I get a preview of my comp back at the camera right here so if I turn this on I think I can hit preview okay it's not running you have to be running turn it on in cetera maybe if I turn this on and then do it this is the image that will go back to the SDI monitor so here I am shooting a little bit of handheld with the system testing out the tracking testing out the keying everything looks pretty good I'm pretty happy with this thank you again to my lovely wife Diana Levine who is our stand-in because I cannot shoot handheld and film myself that's not something I'm able to do yet we're able to see that the composite looks really nice I think there's some tweaking that could be done but overall I pretty much feel like she's in this scene and the track and even though we're not getting like perfect timecode and perfect syncing from the vibe I think that for a lot of cases this is going to be fine [Music] [Music] whoa okay so that wraps it up for this vlog I'm really happy with the live composite the live track and considering that it's a five tracker you know off-the-shelf solution not like super expensive I think really cool things can be done given just even static shots so just very small camera movements can be done with live rendered backgrounds from Unreal Engine mixed with a composite being able to see everything at once it changes everything it's it's kind of inspired me to want to make a short film I was watching I always get the name I'll love death and robots I watched it all at once I don't remember the name noise I was watching that series and I was so inspired by it even though most of its all CG or I think it's like all CG I forget if has any real people and I couldn't tell after a while I didn't really matter that series has really inspired me and I want to make a short film that kind of shows off the tech here right um I'm not a director I'm not a writer but I want to put something together there's probably gonna be robots I kind of like this spaceship and there's but I gonna be people like just medium shots and I have a very big green it's got a very big green screen but I think really compelling visuals and stories could definitely be told using this indie virtual production setup and I I'm inspired to kind of work on that so again that wraps it up for this thanks for hanging out if you have more questions and you want to like really get into this hit us up on the virtual production Facebook group that'll be linked it down below I'm trying to keep like a big list of gear also in the descriptions it's kind of changing constantly I switch things in and out but I'm just trying to keep people updated in case you're like what was that piece of gear it should be linked down below so thanks for watching and I'll see you guys on the next one

Channel: Cinematography Database

Views: 59,228

Rating: 4.9627399 out of 5

Keywords: cine tracer, cinetracer, unreal engine, ue4, previz, previs, virtual production, virtual cinematography, blackmagic design ursa mini pro 4.6k g2, blackmagic video assist hdr 12gsdi, composure, live key, virtual reality, virtual camera, vive pro, vive tracker

Id: j8YdN7zjvsA

Length: 11min 55sec (715 seconds)

Published: Wed Mar 11 2020