Re-lighting Real Footage | Nuke Compositing [Advanced]

hey guys just here with another tutorial this is a technique that a lot of people seem not to know about so I figured I'd put a video up and just quickly go over what this topic is about so if we look at our original footage here this is what we're starting with so this is 300 frames long just a drone shot rotating around this hill and so I'm not gonna play the whole thing because it's not cached but that's kind of what we have is kind of rotation so what's really cool about this specific technique I'm gonna teach you guys is with the depth generator node just a quick example of what we can actually do with that depth learner pass this is just a quick comp by no means would be completely final but this is sort of like a relight using kind of depth pass normals pass and position pass so I'll just quickly go over those individually so we can see how to get this sort of result so what we have here is a 3d camera tracked scene so I already have solved the the 3d camera so if I hit tab here you can see if I double click the point cloud you'll see that we have this already solved so if you have this setup already solved and this technique really works best with shots with a lot of parallax so you see our cameras rotate rotating and traveling quite a bit in this scene which is what's going to allow us to do this effectively so if I double click the depth generator node we have a couple settings in here when we're using it so to get those Maps that we need it's really simple and pretty much if you hover over any of these with your mouse it it describes pretty in detail what each slider does you know for example depth detail noise strength if you hover over easily okay matching pixels in between frames increase to force the matches for example we're fine details are missed so you can play around with these settings when you're going to use this but you don't have to really mess around with it that much all you really really need to do is hook in your camera and hook in your source and you analyze the sequence and after you've done that what you want to do I'll create a fresh one just to show you guys so we're plug in my camera and plug in my source footage and so what you want to do is you want to have this set to depth one over Z which is the default and you also want to export these to other maps here so we want position and normally these two channels aren't actually here so what you normally have to do is click this box go down to new and you'll type in position and then you'll hit this button RGBA and it will automatically fill these boxes with red green blue alpha in the new channel position and we can do the same thing for this normal point so you say surface normal go to new channel say normals you press this button so it's going to create a channel named normals or channel set called normals with these channels stored in it so if that's confusing to you off your complete beginner this might be a little bit complicated you know I have classes going over channels and 3d system and CG compositing so that's all available in the description below if you're interested but so this is what we have and we hit ok and so what we're gonna get now is if you were to analyze the sequence I'm not going to do it because it takes a couple minutes to process if you analyze a sequence and you'll notice that you actually get your channels here now so we have a depth channel a normal Channel and a position pass channel or chance at rather so if I look at position normals or depth we have all those utility passes and if you guys aren't familiar with utility passes again that's covered in my nuke 303 CG compositing course on how to use those channels so I'll just give you guys a couple examples how to actually use them here in with what we're working with so let's take our pre comp footage so what I did was I took that depth generator and I wrote it out so I took it and I saved out a file called depth underscore you know the three hashtags so it'll render out an image sequence and then I said to you 32-bit float this is important it sits under the datatype you want to set the 32-bit float and you want to turn off the compression and these two settings are important because these these special utility passes they're storing 3d data sort of in these pictures so you know distance across this geometry is what's being stored so we have to make sure there's no compression otherwise you'll lose some of that data but once you have this rendered out let me show you guys what it's actually useful for so I'm gonna start with depth because that's the most easy to explain so let me just rear-ended that that's supposed to be depth okay so we have our footage and we have our depth pre-comp you see this one's called depth and if now if I shuffle out the depth into the RGB and a just so we can see it here in the RGB na we see that we have this kind of foggy image here and that's representing the distance from the camera of our hill so it's actually created this this depth Maps just from our live footage which is really awesome so what I can do is I inverted this picture and then I graded it down so I if you look here in the grade it's set to 0.99 so I took this black point slider and I started sliding it all the way up until I start to see something and you see that the image starts to kind of pop in and out so then I just take my up and down arrow keys and just adjust that until we start to get like kind of a faded gradient result and that's what I want so what I'm doing is actually kind of creating like fog across our real scene so now that I have this alpha that what I've created if I look at the RGB and a channel that's what we've created and by the way when I was grading this I made sure to switch this to RGB and a I plug this into a grade and we say it's being masked by the alpha RGB alpha so the Alpha that we just created over here and now when I'm lifting we're actually adding some it looks like fog to our scene so this is how you could create fog and a scene that has absolutely no fog and normally you have to do roto shapes or something like that but if you can get away with this there's a lot of parallax in your scene you can actually use this depth map to create a more realistic fog and what's really awesome about this as well is you know you can play with your level of fog really easily if I just go to the black point and I click after the two numbers and I start using my up and down arrow keys you see I can slide that fog along the surface so you could do all kinds of effects with this you can make it look like you're flying through a cloud you know you could you know easily adjust your fog levels in this way so this is just another example a softer fog so we just compare the two offices that was a little bit more harsh this one I played a bit with the gamma and the black point so that's how you can play around with feed depth okay so the next one we have here is the position so I have the depth normals in position those are the three we rendered out so I have our basic footage here and I have shuffle out the position pass into the RGB na so that takes the position Channel set it puts it into our normal red green blue alpha image so our normal kind of image data stream that's kind of how you can think of it so this tool is a custom tool you can download this script if you're interested this was made by Adrienne her in 2016 I think it's available on Yuka pedia you can also find it there but if you want to just get it here in the script you can download it as well for free so what this tool does is I explained in my nuke 303 class CG compositing class but you know you can take this little picker here and you know we have this really bizarre colorful image and what this image actually is representing is the 3d data in our scene so if I hit the alpha while I'm taking this color picker so I'm sampling by holding ctrl or command on Mac is it yeah I think it's command using a Windows keyboard at the same time here but yeah so holding command on Mac or ctrl on Windows you see this numbers changing while I'm sampling it but if I hit the Alpha Channel you'll see actually what's happening here is we're getting a 3d bubble based on the position that we've rendered out so I can I can move this thing around and you know create an alpha based on this position and you see we have a lot of detail we have all these little trees and all the little bumps that we got from the parallax and so that's why I said it works a little bit better when your shot has a lot of parallax it captures a lot of detail so we can use this alpha let's say if I stick it on the top of this hill we have that alpha saved here and now if I go to a color grade you can see we can grade the top of that hill based on that alpha and what's awesome about this is it's gonna stick to our surface so if I hit play that alpha is actually gonna stick to our scene and we don't have to do anything like placing cards or anything like that we just have a color correction that's sticking and we have a really nice alpha especially on the edges with the trees so that's really a pretty cool example of you know you can get creative with this and come up with different ideas the last one we have is the normals and so normals are used for relighting scenes or CG or whatever you're doing basically normals just stores the direction of a of a face of geometry we don't have any geometry here it's just a 2d image but if we just take a look at what the normals path looks like it looks really crazy but what we need to do is we need to shuffle that into our main image so we have our footage here and we have our depth kind of pre-comp that i saved out over here so we'll take the depth pass and what we need to do is say we create a shuffle node and we say I'll do it I'll just restart it which is so it's easier visually so I plug it in and plug in a and what I want to do is I want to copy what's in this picture into this data stream and the reason we have to do that is because that's how this node is gonna recognize it so they need to be in the same stream you can't have it the normals here and the color picture here so what we do is so we say from a so we see in the shuffle WCB and B we want to copy the data in from a in and put that into B so we want to switch this to a and say normals and then we want to copy that into the H into the B string for normals so I'm going to switch this little box to normals and then I'm just gonna take this and drag it so you see that these lines are copying so we see from a into B so this is B a into B and we're copying the normals so normals read normals green almost blue so we've copied that channel so now if I view this and I look at the channel normals will see that that data is there and if I look beforehand you see it's not there so that's what we've done we just copied this layer sort of into this the stream alright so now that we've done that if we go to the real light node and if you use this node you know how it works but basically you need to plug in a material so you can plug in the Neukom aterial like a basic material because one you can also type in a fog that's another one this is more of a metallic looking material but you need your camera your material and your color which is your background and also lights so I'm using a direct light which is like a Sun and if we look at that and double-click in the real light node and you'll see that I've set the normal vectors to normals so it's picking up that channel that we created and now we can actually create a light based on that scene so we're getting an alpha that looks like this and if I just quickly show you what that looks like through grade so I've created that alpha and I'm plugging that into a grade and grading a little bit of orange and you can see now we're at you starting to get some kind of light hitting on the tips of these hills so it actually looks like now we're getting like a sunset so this is a way you could create like a golden hour of fact kind of cheat it and that's pretty cool and also what's awesome about this is you can rotate this light so if I double click that light and I rotate it we can see that that actually can be manipulated you know if we want light on this side of the hill we can do that because we can rotate the slide around in 3d space and get different results so you see now we have the light on this side so you could use multiple you can create multiple alphas with this and you know totally rely to seen you know using this technique so those are the main three main techniques I'll just show you guys again what I started the video with here combining all those techniques and this is just again a really quick comp this is not you know final quality it would need a lot of finessing to make it hard percent realistic but if I just break break it down here and go through this is just sort of the real light that we have using those techniques so we have the original and we have kind of a foggy sort of Golden Hour effect starting to happen here again you would have longer shadows and more highlights on the rocks and maybe more reflections and pinging highlights and all kinds of stuff if you really really want to make it more realistic but again this is just to demonstrate the class so I'm not gonna go through it entirely because I've already explained two techniques but maybe the one you guys are curious about is the fog so just to show you guys how I did it it's just the depth pass doing the black and white white balance grade or sorry white points and black point grading and what I did was actually took a gray constant and graded up a little bit sunlight into it using a radial and I just masked that by the depth so we get this kind of sunlight filling the volume of the smoke that we've created and then I just basically put that over the top and that's how we get that kind of sort of effect there and then again just a little bit relighting on the tops and some final color grading at the end and just a crop to give it a bit of a wide you know sort of look so you can make this more realistic as well you could break it up with the atmosphere and all kinds of stuff if you if this was like a final production shot but hopefully you guys got something useful out of this video and again if you liked the video hit like it really helps with the YouTube algorithm and helping the channel grow and it will allow me to produce some more content like this and if you really liked it hit the subscribe button as well and you can hit the little bell to get notified every time I put a new video out and yeah so thanks for watching

Channel: Compositing Academy

Views: 14,527

Rating: 4.9949107 out of 5

Keywords: advanced, re-lighting, relighting, normals, nuke, compositing, relight, position, pass, cg, live action, footage, day to night

Id: VYjmvB6d9NA

Length: 16min 26sec (986 seconds)

Published: Wed Jun 03 2020

Reddit Comments

[u/enumerationKnob]

Clever effect, but your video could’ve been about 12 minutes shorter.

[u/Christdnl]

Looks cool! I'll check it out later!