The Math of Color Grading in Nuke

hello and welcome to the math of color grading in this video we're going to talk about some of the operators that we have available to us in nuke to grade the image or change its brightness and we're going to talk about the math behind these grading operators and sort of the fundamental building blocks of how they work i always find that it's easier to understand what an operator is doing when you can visualize what's happening so to visualize this i have this simple setup where i have a linear ramp from negative point one to three and i am plotting that through this gizmo called plot slice which basically takes a single scan line at the bottom of the image and plots the pixel values as x y coordinates so here we can see this linear ramp which is increasing from left to right represented visually as a graph a linear line that increases at a constant rate and any operation that we put between the ramp and the plot we can see visually what's happening to the values of the pixels in the image so before we jump in i want to talk briefly about linear what is linear and why does it matter linear basically means that something is increasing at a constant rate if we look at the rate of change of this ramp we can see that for every x increment y is increasing by the same amount this forms a straight line and means that there's a linear relationship between x and y in vfx the images that we work with are most commonly encoded in something called scene linear an image is scene linear if there is a directly proportional relationship between the light intensity and the pixel intensity so the light intensity in the scene and the pixel values in the image have a proportional relationship and when i say proportional i'm using the mathematical term here two quantities that are varying are proportional when their ratio yields a constant value so light intensity divided by pixel intensity should yield the same constant value anywhere over the range of brightness in the image and linear images are critically important to achieving realism when we're manipulating and generating imagery in a linear domain the math works the same as a camera and the light in the real world motion blur will streak correctly bokeh will form on defocused highlights in a photorealistic way and computer-generated renders will integrate with photographic plates properly so maintaining linearity while we work is super important all right so with that out of the way we'll look at a couple of elementary arithmetical operations first the most basic is addition addition is basically a plus or minus b so if we use this add operator on our ramp here we can see that it is adding or subtracting a constant value to every pixel in the image and the second operation we'll talk about is a multiply and if we multiply our ramp by a constant value you can see that as you would expect we are increasing or decreasing values above or below zero [Music] so with only these two simple operations we can form something called a linear equation so here i am in desmos which is an awesome online utility for graphing functions and here you can see we are graphing a function y equals mx plus b and this is a linear function in slope-intercept form so m is a constant which defines the slope or how fast the value changes and b defines the offset the y-offset or where the line intersects the y-axis any combination of these two constants results in a linear function the line stays straight and it turns out we can do a lot of very useful things with only these two operations for example if we add another addition operator and put it before the multiply and say we set this to negative one and the second addition to positive one if we adjust the multiply in between we can see that the point at which the multiply is pivoting is no longer zero now it's one because we've basically offset the image so that one is at zero done the multiply and then reverse to that offset so many useful things are possible with only these two simple operations now we'll move on to talking about the grade node the grade node has a lot of parameters here but at its core it is built up of only these two simple operations addition and multiplication if we mouse over the help here we can see that the math that is being used is actually shown to us and i've created a desmos plot of that exact math with all of the parameters here so we see the black point white point lift gain multiply offset and gamma and you can see that there's two constant terms being calculated here a and b and these two constant parameters are combined together in this function which as you will notice is in slope intercept form so a is the slope or the rate of change of the line and b is the offset or the point at which the line intercepts the y axis so if we adjust these parameters we can see what the behavior is the black point is pivoting around one the white point is the same as a multiply but inverted lift is the opposite of black point so if we lift up this is happening whereas with black point when we raise the black point this is happening gain and multiply are identical and offset is an add and we can see that the behavior in the grade node is the same but why are there duplicates of the same functionality well each of these parameters has a purpose the black point and lift are a pair intended to be used to match a source zero value to a target zero value and the same with white point and gain white is meant to match a source value to a target value and then multiply offset and gamma are creative controls allowing you to adjust the image after this matching has been done you'll notice that any combination of these values still results in a straight line this is a linear function which preserves linearity gamma on the other hand does not preserve linearity it is curving our graph so gamma is actually a different type of function called a power function if we graph y equals x raised to the power of p if p equals 1 we see our familiar linear line where the input is equal to the output if we raise the value of p we start to get this curve which is pivoting at one so the value of one doesn't change and values between zero and one are decreasing values above one are increasing and we know this because if we plot a linear line where the input equals the output we can see that values between 0 and 1 are decreasing below this line and values above 1 are increasing above this line and the opposite is true if the value of power is below 1 and above 0. so in the grade node if we enable this variation on this function this function is basically saying take our slope intercept linear equation ax plus b and raise it to the power of 1 divided by the gamma parameter so now if we change the gamma function we can see that the behavior is the same as we saw earlier except that it's inverted so if we gamma up we're decreasing contrast and if we gamma down we are increasing contrast and if we switch back to our grade node we can see that the behavior is the same it's important to be careful with the gamma function when you're working with scene referred imagery because as you well know there can be values between 0 and 1 and also highlights and brighter values that are above one in some cases significantly above so if we gamma down to increase the contrast of our image we are darkening values between 0 and 1 but we are actually increasing the brightness significantly of our highlights and you'll notice that this continues to curve as it goes upward the further our x value increases the more the rate of change of our output the y value increases well it turns out that the grade node is not exactly a power function above one if we gamma down to say a value of 0.5 and we change the range of our plot to a higher number say let's use a max value of 10 we can see that above one the line is actually a linear function it's a straight line if we switch to the gamma node with a gamma value of 0.5 we can see that this line continues to curve as it goes upward so the grade node's math is not entirely as advertised by its tooltip the gamma function is actually replaced with a linear function above 1. to do this you can basically calculate the slope or the derivative at a value of 1 which in this case actually simplifies to simply the value of the power itself and if we simply make a cut in the function at 1.0 and replace it with this function which is basically multiplying our slope-intercept form by the gamma and offsetting it to be positioned.1 we get the same behavior as is in the grade node and you'll see the difference here [Music] so this essentially makes the gamma behave a bit better with scene linear imagery and we can emulate that same math with an expression node shown here and we can see that the result between the grade and the expression is the same whereas compared to the gamma it's different now we'll move on to talking a bit about exposure exposure is a term that we are probably familiar with from dailies where vfx supervisors like to refer to brightness changes in objects in photographic terms to say brighten or darken something by a stop or half of a stop exposure is basically the exact same thing as a multiply except that instead of being parameterized on a linear scale it's parameterized on a log scale and basically a one-stop change is a doubling or halving of the light in the image so if we expose up by one stop we are essentially doubling the values in the image which would mean multiplying it by a value of two and we can see this is in fact matching if we expose that by two stops we would be doubling it again to a value of two times two which is four and we can see this matches and the same goes for reducing the brightness of the image if we reduce it by one stop we would be taking our value of the image and dividing it by 2 or halving it multiplying it by a value of 0.5 and we can see that matches as well and if we reduce by two stops we would be having it again to 0.25 and we can see this matches as well it turns out we can represent this relationship by this simple expression which is the value of the input image multiplied by 2 raised to the power of exposure so here i have this expression node and it's basically linked to the exposure node and if we look at the result here we can see that the result between these two is matching perfectly cool so now that we've talked about adjusting the brightness of an image in f-stops with an exposure adjustment we've sort of laid some of the groundwork for logarithmic functions log functions are super useful in image processing and logarithmic behavior is actually very common in natural systems as well and there's many examples like population growth is the inverse of a logarithmic function called an exponential function and we talked earlier about f-stops which is a logarithmic scale to measure light intensity a film negative has a logarithmic response to light and our eyes perceive light in a similar way and log scales are used very commonly to measure things that have a very large range of values for example the richter scale is used to measure earthquakes the decibel scale to measure loudness the ph scale to measure acidity entropy many other examples in the logarithmic scale every unit represents a doubling or a halving of the value so it's very useful in representing data that has a very large range of values in a concise way but in image processing log encodings are very useful as well because it allows us to remap a linear image which has a huge range of values into a more low contrast representation that has values only between 0 and 1. and with the login coding it's very easy to just use the inverse transform an exponential function to map the log image back to a linear image again so there's many types of log curves that are used it's common for camera manufacturers to design their own specific log curve for their camera in order to represent a specific range of values and there are also log spaces which are used for working for example aces the academy color encoding system has several log spaces which can be used for grading so here we can see a log to lin node set in lin to log so we're mapping a linear input image our ramp that goes from negative 0.1 to 20 in this case and we are mapping that into a log encoding and we can see that the image gets a bit more low contrast and the range of values is now only 0 to 1.04 and if we plot that we can see uh sort of this curve which is a log curve and if we look at a detail view of just the zero to one range we can see that the curve looks a bit like this so the log to lin node represents a cineon log curve cineon is the login coding which most closely represented the behavior of film negatives so it's been used for many years in linearizing film scans and we can see if we switch to a color space node set to linear to cineon it matches exactly so you might be thinking oh this curve looks pretty similar to the gamma function that we saw earlier and you're not wrong it is a very similar curve but the math is a bit different here i have the same function that we saw earlier for the gamma and if we adjust the power we can see that the pivot is at 1 and 0. so 1 doesn't change and 0 doesn't change if we look at a log function instead say log base 2 of x it looks like this and if we zoom out a bit we can see that it's kind of a different curve we can see that it has a limit at x equals zero and is a smooth curve that sort of continues up and we can add constant factors in order to change the behavior of the curve if we look at the plot of log base 2 of x minus a divided by b plus a and we adjust a we can see that it's sort of an x y offset of the graph and if we adjust parameter b we can see that it's sort of the slope of the graph if we graph an exponential function we can see that it's the inverse behavior of the log function see how this is mirrored over the y equals x line so what things are useful to know about a log function well normally when a log function is being designed there's a value usually around middle gray which is 0.18 and there will be a maximum value that the log curve can represent in a 0 to 1 range and a minimum value that the curve can represent in a zero to one range and usually these are defined in stops above or below middle gray so if we take the inverse of the cineon log curve and we make a ramp from zero to one [Music] we can see that this is the graph of that curve of the data from a 0 to 1 range and this is the inverse remember so it's an exponential function rather than a log function and if we're curious to see what is the maximum or minimum value that a log curve could represent we can make a constant node and if we look at the output of a white constant through our log to linear conversion we can see that the maximum value a cineon curve can represent in a zero to one range is 13.52 similarly if we set this to black and we look at the output uh the minimum value it can represent is negative 0.00565 so i have here a tool i built for sort of designing your own log curve called log2 shaper this allows you to specify the offset and the number of stops above and below that point so interestingly in the log curve if we take a value of 0.18 and we run it through the inverse direction so the linear to logarithmic mapping we can see that the output value is 0.5 so it's remapping middle gray to halfway between the zero to one range and we can also calculate what the maximum seen linear value would be based on this theoretically if we use a calculator and we take 0.18 times 2 to the power of 6.5 this should be the maximum scene linear value that is represented in this log space 6.5 stops above middle gray so this value is 16.29 and we can verify that if we set our constant to 1 set this to the inverse direction and we can see that indeed this value is 16.29 similarly if we set this to 0 we can see that the output value is 0.00199 and if we do 0.18 times 2 to the negative 6.5 power we get a value of 0.0019 cool so here i have the math that's used in that tool as a desmos plot and it's a little crazy but don't be too overwhelmed it's not that complicated basically we have three parameters a middle gray or the constant value the min exposure and the max exposure or the number of stops above middle gray and below middle gray and the function looks like this so if we adjust the middle gray point you can see what's happening basically it's the position and if we adjust the stops below middle gray this is what's happening and above and we can also graph the inverse function which is a exponential function and it looks like this so basically the opposite of the other function and in log functions sometimes it's useful to have a linear extension so that you can handle values below zero because as you can see with a pure log function there is an asymptote at x equals zero in the inverse direction the log function and an asymptote at y equals 0 in the forward direction the exponential function so to do that you can calculate the derivative of the log function uh the derivative if you don't know is the slope of a function or the rate of change so as you can see where the slope is very high the rate of change is very high and where the slope decreases the rate of change decreases so if you calculate the derivative value at a specific position on the function you can get the slope of the linear extension so this is the slope at the position c so here's a log2 shaper function with a linear extension at the cut position so if we change where this is we can see it's basically adding a linear extension at the position of the cut and this allows us to continue to represent values below zero and similarly if we plot the inverse we can see this is what it looks like if we change the cut we can see we can choose where the linear extension happens from this is the difference between aces cc and aces cct the two logarithmic color spaces in the aces system asus cc is a pure log function so it has no linear extension and asus cct has a toe which is a linear extension in the toe here's a little comparison i put together of some of the different log functions in the forward direction so mapping from linear to log and this is a alexa v3 log c which is the most commonly used log curve used in the alexa camera system this is red log 3g10 notice how alexa has a linear extension and redlock3g10 just goes below zero but continues the curve the cineon curve and here's aces cct and aces cc notice that one has a linear extension and one does not so before we move on i'd like to share one interesting property of log functions and to show this i'll use the alexa log c log curve so if we take our ramp that goes up to three and we use this color space node to go from linear to log and then back from log to linear and we chuck a grade node in between we can take a look and see what actually happens to the values as we adjust a grade node with our familiar simple multiply and addition operators inside of the log space so if we multiply up you can see that it's actually kind of linear but the behavior in the shadow areas is slightly non-linear so shadows tend to stay darker but highlights get brighter and if we gamma you can see the opposite is happening in the shadows it tends to stay brighter and the highlights tend to stay darker and with the offset it's more of a multiply so the whole image is more linearly getting brighter or darker and the behavior in the shadows sort of depends on the toe that's used in the log space sort of the linear extension and where it's placed and how it behaves but this is pretty interesting because we can add a bit of contrast in the shadows boost a bit the highlights and adjust the overall exposure of the image all with these three controls and this is completely different than grading in linear because we have a lot more control over the different regions in the image and this is actually how di tends to grade is in a log space and they'll use lift gamma and gain operators to adjust the exposure and contrast of an image but not in linear in log pretty interesting all right so now we're going to move on to something a lot more simple um which is an invert an invert if we look at this ramp from negative 1 to 1.5 is pretty simple it's basically the complement of the input which is equal to one minus value so if we enable the invert node here we can see that the direction of the linear function just changes and i have a plot of that here so if we plot y equals x which is our identity function and then we plot y equals 1 minus x we can see that at 0.5 they are the same and everywhere else they're exactly inverted interestingly this is equal to the complement which in set theory is basically the difference between a and b pretty simple and very useful for inverting mats all right so now we'll move on to something called a toe and a toe is basically an operator that affects only the part of the image near zero so if we look at our plot and we check out what the toe operator is doing we can see that if we lift the value of the toe we start to get this effect where everything below the threshold value is increased and note that this is very curved this is not a linear function so this function does not preserve linearity and you might recall this is somewhat similar to our black point adjustment so if we raise our lift here we can see that it's achieving something similar except that it is preserving linearity however it is affecting values in the image above the lift that we are doing whereas the toe only affects values below that threshold as a quick side note if you are matching your darks in an image it is strongly recommended to not use a toe operator because this is actually not physically what happens in the real world in photographic systems and in the physics of how light scatters in the atmosphere so if you're using a toe to match your blacks don't do it use something that's linear the lift and black point in the grade work very well there's also this toe reversible gizmo which i wrote which allows you to set a pivot and do some lift value similarly to the toe operator if we lift we can see that everything below the pivot is being affected and everything above is not however we do not want a kink in the operator like this where there's a hard transition from lifted to not lifted so we can use this soften function as you can see this is using a power function there are some limitations however when soften is enabled values below zero are clamped so we can use this pre-offset to fix that so i made a newer version of this tool called compress tow which allows you to similarly set the pivot point and adjust the lift and this also allows you to adjust the power which is sort of the slope of the curve it's useful for specific things but again if you're matching blacks use a grade next up we have the color correct operator we have separate categories for shadows midtones and highlights and in each we have controls for saturation which we haven't seen before and won't talk about because this video is not about hue we have contrast gamma gain and offset so gamma gain and offset we know what they do if we adjust gamma we'll see that it's a standard power function and gain is a multiply and offset is an add and we also have contrast which is basically just a gamma that has a pivot at 0.18 instead of 1.0 and i can prove that to you if we hack together something real quick if we multiply up by 1 divided by the pivot point we could do this with a grade as well if we set our white point to 0.18 [Music] and we make a copy of it and instead set our gain to 0.18 now you see why these parameters are useful so this will round trip it does nothing and if we apply our gamma in between and set it to a value of 2 or in this case 0.5 because we want to increase contrast by a factor of 2. we can see that it in fact matches the color correct with a contrast of 2 except for the values below 0 which are not accounted for here so that's pretty interesting but what about these shadows mid-tones and highlights what does that do so basically what the color correct is doing is it's partitioning the input image into different regions and adjusting a grade with those regions independently to affect the range we can use this ranges tab and if we enable test we can see where the divisions are happening so if we increase this say we want to add a value to our shadows can see that values within the range that we specify are being changed say we want to gain down our highlights a word of warning it's very easy to get very strange and in continuous results with this node so be cautious but it is useful oftentimes in order to specify the range i'll just use a keyer and mask a grade node with that so i have very specific control over what i'm affecting but some people prefer the color correct moving on a color lookup is super useful basically it allows you to create a curve sort of a mapping of an input range to an output range and this directly controls the color so if we create a point here in the middle on our master channel which affects all channels we can see that when we adjust this curve down the same change is being shown in our ramp pretty cool so this is very useful for very specific nonlinear adjustments we've also got a few interesting and less commonly used nodes so we'll look at the roll-off contrast really quickly and if we take a look at this you'll notice that it's clamping values above 1 and below 0 by default so it's not really intended for grading it's actually designed to create an s-curve if we adjust up the contrast we can see that the black point and white point are being increased or moved closer together which is basically increasing the perceptual contrast in the image this is not very pleasing however as it's a very harsh discontinuous line at zero and one so you can adjust up the soft clip and this creates this sort of nice s-curve and this s-curve is interesting there's a quote that i'd like to share with you there's this ves white paper called cinematic color from your monitor to the big screen and this is a excellent excellent document which i would encourage you all to read it's about color management in feature film visual effects pipelines and it talks about a whole bunch of interesting things and there's this specific quote which i'd like to share with you friends don't let friends view scene linear imagery without an s-shaped view transform so what does that mean it sort of hints at a larger subject which is how to view scene linear imagery correctly and we're not going to go into detail about this but i would like to share with you how a lut is made to view scene linear so to do that create a linear to logarithmic mapping and then you apply an s-shaped transform like this and i believe this is actually what the roll-off contrast node was designed to do so for this i think i'm actually going to bring in a real image this is an image shot with the sony f65 from cinematography.net and i'm going to switch my viewlet in nukes viewer to raw so that there's no color transform being applied in scene linear this image looks very blown out but if we gain down we can see values out there and if we gain up there is values in the shadows as well and if we look at this with a logarithmic shaper function we can see that there is much less contrast we've sort of remapped the values into a narrower range here's where the s-shaped view transform comes in if we look at the log image and then look at this roll-off contrast suddenly we're starting to get something that looks a bit more like a pleasing image and if we adjust the parameters here we can sort of customize what it looks like where the center is how much contrast we want you can even go crazy and override these values and adjust the behavior of the shoulder which sort of affects how bright the highlights are the black point percentage and the toe which sort of lifts the shadow areas of the image of course we're not handling hue here at all because we're not talking about that in this video but it's sort of one important component of a view transform the tonal relapping it's a useful node if you use it for the right thing and last but not least we're going to talk about the histogram node and this node as you might notice clips values above one and below zero so be cautious of that and you're probably familiar with the behavior of this from applications like photoshop if you adjust the input range you're basically increasing the contrast similar to adjusting the black point in the grade node up the middle value is a gamma function and this value is the white point similarly with the output range this is a lift and this is a multiply it is a very big limitation however that this operator clamps values above one and below zero so it may not be useful in most cases if you're working with scene linear imagery cool so that concludes this video on the math of color grading in nuke at least for the value operators part of it i'm planning on doing a video on hue as well um sort of diving a bit more in depth into the more complex side of this which is adjusting the hue and dealing with color but hopefully this was interesting and hopefully you learned some things see you next time

Channel: Compositing

Views: 2,492

Rating: 5 out of 5

Keywords: Node based compositing, Nuke, Foundry Nuke, Foundry, Color Grading, Grade Node, Math, Multiply, ColorCorrect, ColorLookup, DI, Linear, Linear Function, Scene-Linear, Scene-Referred, Stops, Exposure, Gamma, Toe, Histogram, RolloffContrast, Logarithmic Function, Log, Log Function, Camera Log, Cineon, Invert

Id: umpA40uheIs

Length: 36min 5sec (2165 seconds)

Published: Tue Sep 22 2020