This INSANE Software Just Changed Filmmaking

what if you could change the lighting on your actors in post to match any environment now you can we are re-lighting video in Unreal Engine let me show you how you can do it with switch light all you need to do is go to the create tab select relight image select an hdri which is going to be used to relight the image then simply drag and drop the image you actually want related and boom done but the most important thing though for us who like Unreal Engine is you can download separately the normal map and Albedo map so I'm going to film a short sequence on the green screen here and then re-light my footage in Unreal Engine so stick around to the end to see the result so we're going to shoot with completely flat lighting like this I've got two LED panels one on either side of the camera to completely get rid of any shadows and that's going to help hopefully with our relighting result and we'll find out what kind of results we actually get really soon I'm about halfway through shooting and it's just occurred to me that the nature of filmmaking is entirely changing again with the two lights and the green screen I'm just in this void I don't need sets or props I have scans and this has just become the equivalent of performance capture except for a video where now asset Gathering performances we're at the mercy of switchlight's algorithm I don't know whether it prefers faces whether it can handle a full body hands or objects so we're going to find out just how it handles this footage I've keyed and masked it in After Effects and exported a set of image sequences one's an RGB with Alpha then just an alpha mask on its own plus a kind of makeshift roughness map where I made the footage black and white and added some sharpening I want to make it clear I'm not sponsored by switch light here I just really like what they do now we're not going to use the relight video option I wish we could but we can't currently get the normal and Albedo out from relight video so we're stuck using relight image so I've navigated to one of my RGB sequences and found an image I want to relight and then click open every time I do this I'm surprised by the result switchlight is doing a really good job on the roughness here as well to get the shine on the forehead and the side of the face here and here's the magic on your hdri we can just spin that round and then click apply all bam completely different lighting and it's done that in in seconds really imagine what this is going to be like when it's actually released or in a year's time from now so on the right hand side you can also see independently the normal that is a solid looking normal but all of the detail in my jumper there and an Albedo which is of course with all of the Shadows taken out so that's a nice flat image to download them you would go to the right hand side panel uncheck relit composite and just check normal and Albedo and then download all now enough gushing from me the sequence that I filmed on the green screen for the end of the video is about 30 seconds long so I'm gonna have to now process every single one of those frames through switch light in batches of 10. but let's just skip ahead to the results so here I've got folders for each of my shots and inside there the normal and albedos from switch light so this is the Albedo you can see in the jumper there's temporal inconsistency there's exposure changes there's the normal map there again some temporal inconsistency by no means the worst I've seen and in fact I think it's incredible really now there are a few ways of dealing with this kind of temporal noise I use DaVinci Resolve because it has a flicker filter that deals with this really well here we are in Unreal Engine I've got all four of my sequences the Albedo normal roughness and mask playing back simultaneously in a level sequence and on this plane we're getting a re-lit result in normal lip mode it looks a bit weird a bit of lighting adjustment would probably sort this out but I just go straight for path tracer because yeah it like immediately looks really nice that's quite epic actually let's add a light and I just want to show you how the plane reacts to like a point light say so I've brought a light in from the left let's move it to the back a bit maybe there we go and you can really just play around with it I mean this is a point light with a Source radius of zero so if things start to look a bit weird like they do here just adds like 50. soften it right out maybe less even 20 [Music] . who needs meta human when you have human this is a real person being re-lit in the engine God damn and the great thing about this is you're adding lights they're affecting the scene and the character you can see that that's already lighting up the ground behind me so it doesn't look unnatural absolutely incredible so I've been speaking with the people at switch life to find out what's next on their roadmap and there's some very cool things apparently within the next couple of months they are actually looking at doing roughness marks which is pretty much the only piece of the puzzle that I think we're missing when it comes to Unreal under certain circumstances you can really tell that the skin clothing and eyes don't reflect the way that they do in real life I think their AI masking is really good and they're going to be adding an alpha matte export option as well as the roughness so that's really cool they do have a video relighting feature of course at the moment but it's a little limited but it will be expanded to include normal maps and Albedo map exports so that's going to speed this whole workflow up so much and if all of that wasn't cool enough they've said that they want to cater primarily for industry professionals like filmmakers and visual effects artists so they're developing the software so it's not just web-based but also a desktop Standalone application plus pro plugins for nuke and after effects and you've waited long enough so here's the short sequence that I made using switch light foreign [Music] [Music] now I think the ability to film a subject and put them in Unreal Engine and have them react to lights it's game changing and it's going to change filmmaking forever no matter what it feels like a bit of a superpower but actually when everyone's super no one's super and that's a good thing because filmmaking shouldn't be a superpower it's magical but it should be accessible to everybody when everyone has these amazing abilities and can focus more on telling their stories than the tech that's how a good filmmaker will be defined I'd like to give a quick shout out to my patrons thank you guys so much for making videos like this possible and if you're not currently a patron and you want to support the channel well just head over to my patreon there's a link in the description I've just added all of the image sequences and switch light results from this video for all patrons to download and test out

Channel: Joshua M Kerr

Views: 138,396

Keywords: virtual production, virtual production unreal engine, virtual production unreal engine 5, virtual production unreal engine tutorial, virtual production unreal engine 5 green screen, virtual production unreal engine green screen, virtual production unreal engine 5 tutorial, virtual production unreal 5, relight video ai, unreal engine 5, unreal engine 5.1, unreal engine 5.2, unreal engine 5 tutorial, unreal engine tutorial, filmmaking, switch light ai, switch light

Id: CYN2alBR3P8

Length: 7min 15sec (435 seconds)

Published: Tue Jul 18 2023