Finally! A VFX Workflow With Less Guesswork

have you ever worked on a visual effects shot where you were just kind of guessing to get the lighting to match your footage and then in compositing you're endlessly tweaking stuff to get things looking right okay shift the game a little bit less blue a little bit more gamble put his gamma gamma the heel with the game lift it up too much blue [Applause] this is how I used to work but not anymore with my new workflow I can confidently match the lighting between my footage and CGI elements this works in different lighting conditions too so I'll break down this entire workflow which uses a few simple tools when filming along with the Asus color system to achieve these great results let's jump in so you can follow along you can download this workflow graphic along with the footage photo and blender assets for free at the link in the description now the success of this workflow begins with a careful filming process for my main camera I use a Panasonic gh5 with a v-log color space which gives me very flat footage and I make sure to set my white balance to a constant value so that it's not changing throughout the shot I film my footage and then right afterwards without changing any lighting camera settings or general camera position I'll put this color Checker in the shot and take some footage of it this is the x-rite color Checker classic you don't absolutely need a color Checker you can use a white balance card or nothing at all but a color Checker will provide the best reference next I'll keep the color Checker in place and set up my 360 camera close to where my CG elements will be in the scene with this 360 camera and most 360 cameras nowadays I can shoot multiple photos at different exposures so in the end do we have our main footage footage of our color Checker and our 360 photos including our color Checker to start let's create our hdri first we need to merge all our 360 photos into one image some 360 cameras come with free programs to do this but I use Photoshop I'll merge all the photos together using the merge to HDR Pro function I'll select my individual photos and then make sure to uncheck the alignment option since we know the 360 camera was stationary and then let's let Photoshop merge these images together I'll uncheck this box hit OK and here we have our hdri at this point we're not going to worry about adjusting the hdri we Simply Save it out as a Radiance file next we want to convert both the hdri and our footage into the same color space to do this we are going to use Aces if you haven't watched my Asus tutorial definitely give it a watch before continuing with this video because I'm going to use Concepts that I explained in that tutorial now every image or video file has a certain color space with a specific color gamut and a gamma mapping my gh5 footage has a color gamut called V gamut and a non-linear gamma mapping called v-log my 360 camera takes individual photos which have the color gamut srgb and a non-linear gamma mapping after I merge all the 360 photos the resulting hdri has the same color gamut srgb but my gamma mapping is now linear to match our hdri to our footage we want both elements to have the same color space this is where Asus comes in we will apply a unique Aces input display transform or IDT to our footage and hdri this will convert both to have the same Asus color space so to begin this process I'm going to jump into the free version of nuke because nuke is really good when it comes to color management so here nuke will go to the project settings then in this color tab we can tell nuke to use Aces then I'll press R in this area to read in the gh5 footage if I then click on the read node I can see this input transform option this is where we will select the aces IDT to convert our footage into the Asus CG color space in this case we'll select an IDT called input Panasonic v-log V gamut now how do you know what IDT to use for your camera first search Google 4 correct Aces IDT for your camera name with color profile name that you use and just know that there are only Aces idts made for the most popular log and raw color profiles and here's another neat check that you can do set up your camera and record footage of a scene at the correct exposure plus two stops Overexposed and minus two stops underexposed in Nuke I have the footage and I've selected the Asus IDT I want to test here I've split out the correctly exposed Overexposed and underexposed footage if I add an exposure effect to our correct exposure and I change the adjustment to stops I can add two stops zero plus two stops should look very similar to the original plus two stops from our camera and they do similarly I can apply this exposure effect here and I can subtract four stops two minus four stops should look very similar to the original minus two stops from our camera and it's pretty close passing this test does not guarantee that you have the right IDT selected for your camera but you know if you fail this test you don't have the right IDT selected okay back to Nuke next I'll read in the hdri and then for this input transform I will look for an Aces IDT called the linear srgb next we need to make the exposure and white balance of our hdri match our footage here's how we'll do this we have our footage and our hdri both with the color Checker showing we're going to sample the color of this gray patch in the footage and then also in the hdri we can see the RGB values of these two gray colors which are not matching so I will multiply the hdri by three values that will result in a color that will now match the other color and with this small multiplication applied to the entire hdri it will now match the footage so in Nuke let's look at our footage and we'll zoom in on this color Checker we can then hold command or control and shift to sample the pixel values of this gray patch I can see the RGB values down here to record these values I'll create a node called a sticky note and write down the values then let's view the hdri zoom in on the color Checker and let's Now sample the same gray patch here you can see these RGB values are quite different from the ones we've recorded so let's add a multiply node and we'll click here to give us separate RGB controls now starting with the red value I'll adjust this till our hdri is reading the same as the red value in our footage then I can do the same to match the blue and green values and if you don't have a color Checker you can try matching by sampling a different part of the scene like the concrete here but now that things are matching in theory if I sample any part of the scene say part of the sky the color value should be exactly the same in our footage and hgri at this step it's important that you don't add any major color grading to your footage or hdri certain color grading adjustments like gamma will mess up the linear nature of our footage taking away the benefits of this linear workflow at this point just stick to making adjustments using multiply operations such as gain which keeps things linear and of course make sure to apply the same adjustments to both the footage and hdri but that's it for our work in Nuke we've calculated these values to match our hdri and footage and we'll use these values as we jump into blender I have blender configured for Aces check the description for a tutorial on how to do this I have a camera in my scene and I've set the background to my camera footage this color space option here is our Aces IDT and so I set it to the same Panasonic IDT we selected earlier I've also aligned my camera to the proper perspective for the scene next we'll go to the Shader editor and select the world option here I've opened that hdri file we created in Photoshop and again the color space option here is the same thing as our Asus IDT so we select linear srgb now we want to apply the same color correction here that we applied to our hdri in Nuke to do this we'll add a color mix node set to multiply with our hdri plugged in make sure the factor is turned up to 1 then for this bottom color we will copy the r g and B values that we calculated in Nuke [Music] I found this really cool Dodge Charger model for free on blend swap first I'll open this model in its own blend file I'm going to go through all the materials and look for any images is being used we want to make sure we're using the proper Aces IDT to convert all our materials properly into Aces for any images which are part of the diffuse color or Albedo of a material we want to use the IDT roll matte paint or utility srgb texture they do the same thing for any images which Drive the roughness metalness bump displacement Etc we will use the IDT named roll data or utility raw they also do the same thing finally for any parts of the materials that use an RGB color input we need to convert this value into the aces CG gamut unfortunately blender does not yet do this for us so we'll have to do it manually I've used this website to calculate the new values which I'll link below back in our main blend file I'll append The Collection with our updated car model now let's go into rendered view to see how things are looking we'll notice the lighting from our hdri is a bit off we don't have the brightness and shadows that we can see in our reference footage when I capture the different exposures from my 360 camera this was the darkest exposure the camera was able to capture and you can see the sun looks like a large blob the sun should look like a very small point in our darkest exposure just like this unfortunately most 360 cameras cannot be equipped with the necessary filters to capture such low exposures so here's a quick workaround first in Photoshop I'm going to cover up the bad sun in our hdri to do this I'll add an exposure adjustment so I can see the sun more easily then let's add an empty layer over our background let's select the Color Picker and make sure we set the sample mode to current and Below no adjustments then let's go into the gradient tool and we'll go into the color settings for the left value we'll select an area just outside the sun flare and for the right we'll select an area a bit further from the Sun then make sure to have selected the radial gradient type and we'll drag out a gradient from the center of our sun then let's turn the layer off and select the ellipse tool select path for the mode and let's draw a mask around the Sun and hit mask turn the layer back on and we can select our new mask and increase the feather amount in the properties window now we have effectively taken out the bad Sun let's turn off the exposure adjustment and save out a New hdri Radiance file back in blender we'll swap in this new hdri without the sun in it now we're going to create our own Sun object we'll dial in this strength value later but for now let's eyeball it then we'll rotate the sun to align the Shadows with how they appear in our reference footage you can increase the sun angle value if you want the Shadows to be softer but my reference Shadows are sharp so I'll keep this value low now how do we know how bright the sun should be what I can do is download a special exr graphic of the color Checker and bring this into blender using the image as planes tool in the material settings our IDT is set to ACG which is correct since this exr is in Aces CG I will adjust the roughness of my material to 0.8 then I'm going to position this color Checker in roughly the same position and angle as the real color Checker we film let's look through the main camera in the render View and we'll create a small render region just for our color Checker oh and let's turn off our floor plane for now let's hit render and in the render result window if we right click and hold we can sample the pixel values of this gray patch here here our values are reading around 0.23 and remember in our footage our gray patch is reading around 0.28 so from here we just want to increase the strength of the sun until this gray patch reads around 0.28 now let's turn our floor plane back on and our lighting is looking a lot closer to our reference for our floor material I'm going to Simply project our footage onto this plane to do this I'll add a project UV modifier select the default UV map and select the camera for our projection object then for the aspect X and Y will set 16 and 9 since the footage we're projecting has a 16x9 aspect ratio we'll also add a subdivision surface modifier set to simple and place this for our UV project modifier this just makes sure we have enough detailed geometry to accurately project onto next here is the material setup this node is the footage of my clean plate again set to have that Panasonic IDT then I've added an RGB curves node to match the colors a bit more and I've raised the roughness slightly on this Shader node this technique of projecting the footage onto the floor plane won't always work this well in some cases you might have to create your own Shader to recreate the floor material now let's set this plane as a shadow Catcher And in the render passes tab let's enable the shadow catcher render pass through the car windows we're seeing the hdri which doesn't look realistic so I'll duplicate this floor plane place it in the background and uncheck the shadow catcher property let's duplicate this projection material and change this to an emission Shader and I will delete the RGB curves then we only want this plane to be visible through the windows so in the object properties tab we'll uncheck all the visibility options except for transmission now we're ready to hit render don't worry if you don't see the shadow being rendered out on the final render the shadow information will be hiding in this Shadow pass that we enabled finally we'll save out this render as an open exr multi-layer file now the beauty of all this setup is that compositing is going to be super simple in a new blend file let's go into the compositing tab and check use nodes we'll then read in our background footage change some of the frame settings here and again we'll set the aces IDT to the Panasonic one then we'll also read in our car render which should have an Aces IDT set to Asus CG first let's work with our shadow pass which we can view by Control Plus shift and clicking this node multiple times this is a relatively new pass in blender and it's really cool because this pass not only captures the shadow Darkness but also the color of the Shadows to composite this pass properly we'll add a color mix node set to multiply and we'll multiply the background footage by this pass [Music] next we'll simply add an alpha over node and place our background in the top input and our car in the bottom now let's compare what we have here so far with our reference I think for our first tweak our shadows should be darker to do this we'll add a gamma node to the shadow pass and increase the value slightly then I want to adjust the color of our car since it looks a bit too magenta so we'll add a color balance node and adjust the gain away from magenta then I think our shadows could be a bit more blue so let's add another color balance node and here we don't want to adjust the gain we want to stick with only using the gamma adjustments just like before to render out our final comp make sure your composite is hooked up to the composite node I will set an output type of open exr and name the file output then hit render animation now for the final color grading in DaVinci Resolve I will go into the project settings and in the color management tab I will set the color science to Aces and set our output transform to srgb I'll import the exr of my final render then we can right click the exr go to Aces input transform and select Asus CG this is the aces IDT next in the color tab I can add some nice color grading finally I'll go into the export tab select h.264 and I can render out my final shot so there you have it the full workflow now your homework is to go download those free assets and try adding that car to the shot in the shade now for this one you don't have to replace the sun in the hdri because it's not visible and when you have your results send them to me on Instagram or Twitter and I will send you my Asus Kickstart kit for free alright thanks for watching and subscribing and I'll see you in the next one [Music]

Channel: InLightVFX

Views: 121,894

Keywords:

Id: G2SacOKhJto

Length: 17min 12sec (1032 seconds)

Published: Wed Feb 01 2023