I HATE GREENSCREEN - Living Room Virtual Production with Unreal Engine

hi there welcome back to filmmaking tips i'm josh and i've been doing virtual production in my living room so thanks for joining me now i'm going to give you a little tour around my living room and what i've been doing with virtual production that doesn't just mean filming on this green screen here but it also means using unreal engine and tracking the camera position so that i can use 3d assets on the green screen so they have parallax and so that they move more naturally and ultimately trying to make things a bit more realistic generally speaking i hate green screen and that's why i want to try and make make it better i've done some tests previously using uh back projection which worked very well but as usual there's always a caveat when you're doing things that aren't real back projection you need a lot of distance between yourself the lights and the screen otherwise you get a lot of light spill on it making sure well basically means your black levels are muddy and grayish instead of pure black which is troublesome but when you use a green screen you can actually use the live assets so the color will be accurate but of course it brings a whole bunch of other different problems like lighting the green screen so i've got two if you can see here uh there i've got these two leds newer leds uh the 660 model and they're basically on a c-stand which is here so you can see that and also a flag as well mounted to the same c-stand i'm gonna pop through here might be easier to see and that means i am supposed to sit here i'm gonna mount this camera here so the big deal is i'm trying to make a short film called cabbie and that's a film entirely set inside a cab and at the best of times filming inside the car is tricky which eats into schedule times each eats in the budget and all sorts so i'm going to attempt green screen but i'm going to do it with unreal engine and i'm going to do it with 3d assets if it works that's if i can make it look good so today i've already finished my shooting for the day but i've got my green screen set up fairly evenly lit given the space i've got the two led lights are about 60 inches away from the screen themselves and they're giving a fairly good spread but i don't have an extra um 60 inches to spare here so if i move myself too far away from the screen the screen will become too small and i won't have enough room to move around so i'm either going to need a bigger screen or a larger space or both this reflector is bouncing off some 3d assets from unreal engine that's the side view of the car window so what that's doing is as the car moves you can see the light changing on this side of my face like that which is dynamic lighting that's being streamed directly from unreal at the moment i'll show you around a few other things let's come let's get you off here okay right this is the whole thing about filming in your living room it's just really small so you can see this is kind of my scene here which is um a street scene that i've built using some free assets from the unreal marketplace so that's running there and then it's running from back in my computer hdmi cable to the projector right here here's the tracker which i've just finished using now actually so that's the vive tracker right there that would normally go on this camera that i'm holding right now that is being picked up by unreal engine as well and the vive base station is over there there it is ready to pick up the tracker signal so at the moment i'm trying this with a tripod but generally speaking the idea would be to have the vive tracker on this camera so that when i move around left right and the green screens behind me it should all move and i'm not really ready to test that just yet what i want to make sure is i can actually get the green screen keyed well and the background looking realistic that's pretty much an overview of what i've got going on in here we can now dive into after effects with the different layers i've captured and you can have a look at what the result of this test was see in a moment okay so here we are in after effects as you can see this is the footage that i shot previously and i've already put a mask around the edge i did this and i'll show you i'll just quickly toggle that mask on and off here none and you can see i've masked off this flag which is you're blocking some of the light so i'm just going to run you through some of the plugins that i used to create the effect you're about to see the first one being a lumetri color now i've used this uh not very intensive uh effect it's just a custom lut downloaded from the panasonic website this lut is the nicest lut it's actually what it's called it doesn't mean it's the nicest it's just called that so i've just applied that in order to bring out the green a bit more so it's easier to key i haven't had as much luck keying uh log footage i think it's always better to throw on lut first then these are just the inbuilt after effects keying plugins so key light is the first one before i turn it on you'll see i'm using an intermediate result rather than a final result there's a very strong reason for that the final result looks like garbage because it basically over grades over corrects your footage and it can end up looking quite noisy and messy i'll turn this effect on right now and as you can see that's me masked out i'll show you what's going on in here my recommendations so far with having messed around with this plugin for quite a while now try your best to keep these by their default values i've had to actually push these a little bit more than i would normally like but moving the screen gain and balance is probably not the best method for getting a clean key but if you need to it's possible to push them a little bit if you use the screen matte overlay you can see that the white areas the uh it's the solid layer the black being the transparent you can see there's a nice bit of opacity around the hair which is what you want you do not want these really hard lines around the edge and the status is another way of uh seeing that you just making sure that you've definitely gotten rid of all of the uh green area i'm gonna nip back to intermediate results um i've been messing with the clip black and clip white mainly that's literally all i use i don't mess with any of these other things because actually they have more of an impact or the impact entirely when you're using final result and they don't apply when you're using intermediate so an interesting thing that happens when you use key light in after effects if you type in key into your effects panel you have key light options this one here is well that's key light there for keying there's also this key light plus key cleaner plus advanced spill suppressor which is a good combo to use um to start with but i've actually found a slightly better method the key light plug-in yes use it advanced spill suppressor yes use it key cleaner though has given me some pretty messy uh results it can over soften edges it doesn't it it it does reduce the sort of chatter you can get a lot of like bubbling and noise around the edge of a key sometimes and that did a fair job but i always found this there's just this one thing that i couldn't solve and i was getting a sort of uh edge like a just color edge around the uh around the footage which is sort of like slightly darker than my skin and not quite the same color as the background so it always looked like i had this tiny one or two pixel outline um and until i found this other solution i was stuck with key light in place the next thing to use is a refined soft matte that is what i'm using in place of the key cleaner it's really quite amazing what this thing does um i wonder if i can show you i've got this uh temporary back background here so without refined soft matte um the easiest way to see what it does is looking around the hair so if i just turn it on keep your eyes on that hair see what it did there brought back quite a bit of of extra detail a little bit extra sort of transparency some hairs that weren't there before same on this side actually turn it off and on again there now this hard edge around the ear and this this is obviously still a little transparent but if you refine soft edge it really brings back quite a bit of detail and i've found here's what it here's its main superpower calculate edge details with it checked on you're going to be so so so careful that your edge radius is not too high i've put it at four pixels probably as far as i would push this if you go to 10 you can start getting bubbling and noise and what they're calling like chatter that's why they have a chatter reduction section but even with that turned all the way up if you have too much of an edge radius you're gonna get still a bit of chatter because it starts to not quite be able to tell the difference between your green background and the edge of your subject so i keep that really low and i i've turned this all the way up its other superpower is the decontaminate edge colors which if i turn it off it does a fair job and this is actually this setting that helped me reduce the edge this the the border i was getting around my subject which isn't mega apparent here but with it on it's even pulling back more detail here uh in in this side of my head i'll turn it off and then again so you can see that you can see just there's a few hairs just coming back which i hope you can tell uh on youtube if you can't don't worry about it it does it um anyway finally and be careful as well not to put the spill suppressor before soft mat the order of your plug-ins matters that is the keying uh all in place um you will see a clip of this end of the video so stick around for that um but i'm gonna now show you what's happening now with the background that i've done via unreal engine uh and trying to integrate all of these elements this is the camera from unreal engine quite a basic sort of city scene car driving scene and i plap me on top there we go now obviously i don't quite match the background at this point but having the two layers you can see that i i fit perfectly where i should be in the frame i composed these in unreal engine as a guide for where i was going to be placed so it's not just luck i was put in that position specifically because it looked like the right place to be when i shot it so there was a monitor uh from my computer which you saw and that's how i know what's uh what's being composited on top of another so a couple of other magic effects that i'm using super comp is a red giant effect not a built-in plug-in for after effects but uh to be honest is the one plug-in that i can find that actually does a few things i i actually don't have the expertise in after effects to do um and i'll show you what these are by just turning toggling them on and off so i'll turn the supercomp layer on so you can have a look up here and see what it's doing in its entirety that is blending me quite nicely with the color of the backdrop and it's doing that in a few ways so one of them is this color correction where there's a literal button that says match background it knows the background because i've dragged it in down here and i say match background with it out without it with it without it with it it was that simple i've done nothing to any of this uh none of these other settings have been touched and then you've got an edge blend which is a slight edge radius probably don't even need much of that to be honest but here's the main one that i love and that's light wrap it is taking the lights from the background and wrapping me in them to give it a sense especially around here around the ear it's a little bit sort of light blooming just sort of try and bring the two layers together like that really good amazing stuff uh and on the background i've even got a little bit of diffusion because i am by default i didn't have the um any kind of blooming as you would expect to see in camera if you were filming lights city lights so it just gives a little bit of extra kind of light bloom and of course what it's doing with the light blue on the top layer is being affected by that via these effects as well it's really smart i love it finally we're getting into the very very end a bit of a color grade on top it's a bit dark but to be honest with you i think i just need to improve my lighting overall on this scene and i will show you the final result [Music] now [Music] so thanks for joining me on this brief dive into after effects and using unreal backgrounds and trying to match the two refining the lighting is where the effort needs to go next thanks for joining me and i'll see you on the next one

Channel: Joshua M Kerr

Views: 8,640

Rating: 4.8562093 out of 5

Keywords:

Id: Gi5iX3CoEeI

Length: 14min 51sec (891 seconds)

Published: Sun Apr 11 2021