Distorting Reality with The Matrix Node | Nuke Tutorial

so this is the matrix node i will just kind of show you show you how to make one as well and we'll speak in a moment about about what it does so by default it asks you how big do you want the matrix node to be um we will make it the default size three by three and um and then we'll see what is it that what does it actually do and how does it actually work so basically what the matrix node does is cross correlation um quite often people confuse it with convolution because it's a very very similar idea but convolution just has one extra step for now you can think of the both of them to be the same thing but actually what the matrix node does is it's cross-correlation so partly sometimes you hear people referring to this as convolution and and they're not too wrong it is very very very very very very very similar to convolution so let's let's look at the matrix node and how does it actually work what does it do and then later we'll see some kind of fun fun applications as well because what you will find actually that matrix node is is nothing nothing nothing that magical but but it's still useful so anyways initially when you make a matrix node it does nothing in fact i'm tempted to quickly drop my screen resolution to to be a little bit smaller just so that i can have a you can see slightly bigger how's that cool slightly bigger numbers for you guys there we are so um so initially the matrix node seems to do nothing so what do these numbers here represent what these numbers are is the current pixel and how it connects to the pixels around it so in compositing a lot of the operations that we do are are known as pixel operations so for example when you do color correction you could for example multiply a pixel's value by 2 and then it will be twice as bright so but you do it for every single pixel individual pixel so you do things for the pixel and for the color what you don't necessarily do you know there's no dependency whether this pixel has anything to do with pixels around it whereas with the matrix node we actually do those dependencies so [Music] we link pixels with the neighbors so and this is what it does so what this gives you here is a weighting of how much should this pixel's value be dependent on its surrounding pixels values and and if you simply type in one in the middle in here what it gives you is the original image so if you type zero i'm using no pixels from anywhere if i'm using one for this current pixel let's say this pixel in here i'm just using its value and none of the values around it so one example that we could do is if i if i do another version of the same same one just next to it and let's do that so let's see if this works nice connect this in here and for this one what we'll do is something else is instead of using this pixel's value i will use the value of the pixel from the left so if i do this one instead so instead of using this middle pixel's value i use the value of the pixel from the left then what will happen is that the whole image will move to the right and you think like wait wait what what happened here well basically what's actually happening is that i'm taking the pixel from the left and putting it to the middle and by doing that i'm moving everything to the right so moving everything to the right by one pixel so this is a very inconvenient way for example to make a one one pixel transform which you could you know there's definitely better ways of doing that right so but you could do that and and effectively this is what is happening so if i then take this one pixel and you know maybe i want the original pixel as well i want to kind of have both of them adding together so this is then what the matrix node kind of actually does so if i do this if i do contour sequential v and in addition to the middle pixel and in addition to the pixel on the left i want to have the middle pixel the pixel to the left so i want to move the pixel to the left over to the middle and add it to the current pixel so if i do this i'm taking the pixel from the left and moving it to the right i'm adding those pixels together so now i get something that's twice as bright and slightly moved to the left so great so far but this is the very very very basic in here that what we are doing for these pixels is is taking their value and starting to implement the the value of its neighbors into the final value as well so at the moment i'm not using any of the other values nearby they contribute contribute nothing but from the pixel next to it i'm adding adding a whole pixel value from that so i'm just placing them together and then if i want to if i don't want my image to get brighter but i would like to get like an average value of this pixel at the one to its left what i can do is i can normalize this image so basically what normalization means is we take all of these values add them together which at the moment gives me two and then i divide my final pixel value evaluate that so normalize now i get the same value as before but if i compare this original image and this one afterwards what you might find is that the new one is just a little bit softer so here we had a shop line it's just now a little bit softer so what we have achieved is filtering so we're filtering we are taking an average of two pixel values so i'm taking average of this pixel value whatever pixel this is and the pixel value to its left and as a result because i'm comparing every pixel with itself and the one to the left the whole image seems to slide a little bit to the right because it's from the left right i said right like a pun anyways so we're taking an image from the left and moving it to the right sweep um let's look at it in a kind of a little bit more microscopic example if you're going to zoom out on this you will see maybe uh just checking it how visibly it is on zoom as well as like it's it's not really that visible in jump but your image will be just a tiny bit softer okay so zoom is doing its best to to compress the softness um and to display what is happening but effectively what i'm doing is i'm taking a pixel from the left and adding it to the pixel on the right so let's look at some more microscopic examples with some worked out examples in here as well so here i have a tool that allows you to visualize what is actually happening inside of this node so i'm going to change my layout as well to to look a little bit more like this i'm just going to hit home to kind of fit as much of it as i can i'm just squeezing a little bit more in here there we are so what we do when we make these filters so i have a matrix node in here which kind of controls it and that's you know um that's that's what is happening at the moment so we're doing a a add and the normalize so if i add one pixel from the from the left and i added to this one and the normalize then what is actually happening is that i have created this filter and wherever i kind of go in my image i take all of these incoming values i add them together with an appropriate waiting and then because i'm normalizing i'm going to divide with the final final value so we can work through this one pixel at the time so let's take this one here as an example no not that one let's take let's take this one here as an example it's a good example so from this pixel down here i'm taking 0 so i'm doing 0 times 0. okay and from the next one this pixel's value is is originally zero nine nine something like that so what i'm doing is i'm still multiplying it with zero so this is how it overlaps so that's the filter and that's how it overlaps from the next one again taking nothing from this one i'm taking all of this value it just happens to be zero from this one i am taking all of its value which is 0.978 and from the others again taking nothing so but the problem is that down here that's not the real problem is that i've added together this all of these values in here to do the normalization step so down here this is where i add all these together so the weighting of bottom left bottom bottom right left this pixel so these pixels here i'm actually taking contributions even though here the original value was 0 and this is like 0.97 my final output which is the middle of this big square in here so if i just show you this so that's my final output this square in here it's half because i'm dividing it by two then if i was to kind of include more pixels in this so you can see i'm blurring a little bit and again if i go show you examples this is just moving the whole image to the right by one pixel right so again numerically this is what's happening lots of multiplying with zeros and at one point we multiply with one so effectively what we're doing is we don't take the contribution of this pixel value we use the contribution of this pixel value so you can see this zero ends up over here and if i do let's say this pixel over here it used to be 0.92 but we're taking zero of this contribution we're taking all of this contribution so that's the contribution to this pixel in here so you can see you know i'm taking the pixel from the left moving it to the right taking from the left moving to the right and i'm doing it in this kind of uh unintended convoluted way by taking every pixel checking the contribution and adding it to the final image so this is filtering and that's that's how it works so you can see here and because i'm doing it for every pixel it moves the whole image so this pixel gets its value replaced by the pixel under on the on the left this pixel gets it value replaced by the pixel on the left we're doing it for all the pictures now you're noticing here as well is that actually something funny happens with the pixels on the edge so for example here um this one is fine so this pixel i'm going to replace with the pixel on the left which is 0.175 and this pixel over here 0.236 so it used to be 0 175 now it's going to be 236 that's fine too but what about the one to the left of that there is nothing more on the left you know this this one actually breaks that's a tool i built for this so because there is nothing to the left of this nuke's default behavior is just duplicate whatever is your h pixel so again this is something that you might have seen before if you use nuke which is that when you do transforms and merge things so let's transform an image if i want to move things to the right and i don't use a black outside basically what we end up doing is stretching the last pixel right so this is what's happening is i don't have anything else to take so i'll just use the use whatever was the last pixel value so the matrix node does something similar cool so edges do matter so what happens at the edge can be a bit of an issue so coming back to this so this is filtering um i'm moving this one pixel to the right in this very inconvenient way but what would happen if i include more pixels in here so for example if i do this pixel and the pixel on the right and the pixel above and the pixel below basically again what we are doing is we're kind of adding all these contributions together so i'm not i'm not taking anything from the pixels in the corners but uh but from the pixels on this cross i'm taking the full value of that and by adding these together and then dividing it i'm averaging together the different values and you can see from the result of this is this this average is closer to gray which if you look at it kind of makes sense because if i have a dark pixel surrounded by bright pixels the it will become less stark and likewise if i have a bright pixel surrounded by dark pixels it will become less sprite so filtering and averaging takes out extremes it makes signals smoother and in other words when we talk about images we often use the word blurring for this so if you just average values together the average uh seems to be closer to the mean value and reduces the variance so so we had quite a bit of variance in here very dark very bright values but by averaging values with bitter neighbors these then to then get removed which is cool so so basically what we're doing is we're just averaging these values together so we're adding all of those values together all five of them and then dividing it by that now one way you could do this as well if you know that you have going to use five of these you could just do 0.2 of each of these as well just illustrate that you can and you would actually get the same result that we had before that's because all these five is zero point two is to add up to one so when i was normalizing earlier and i was dividing it by five if i divide every value in here by five then then that's the same thing anyways so this is this is how the matrix node works and that's that's all it does so this was a tool that i built to visualize it but actually this this is this is exactly what it does so if i'm using these values in here 0.2 and whatnot what you should get if you zoom into this um let's zoom into this uh this hand in here what we should see is uh before and after it's a filtering effect effect it's becomes smoother okay that's cool now the other thing to remember is that we are doing this for every single pixel so what we are doing with this tool in here is we are kind of starting starting with that pixel at the bottom left which has its own issues but effectively go through every single pixel every single pixel and you take it surrounding eight pixels and the pixel itself so it's all together nine pixels and you contribute them together with some kind of a weighting you could actually just make nine copies of your whole image move those whole images by one pixel and add them together to get the same effect just to illustrate that you totally can do that i have done that so let's have a look over here so here is our original image and here i have nine different copies of this image so once that move it by one pixel this way one pixel that way one pixel that way one pixel in that direction and that that's the original pixel not moving anywhere at all and so on and so on so i'm moving all of these different values and contributions in here and then if i just multiply each one of them with the weighting that i want such as like one from each of each of the sides and then i divide with the sum of all of these values then again i get the same result so i divide with that value so basically i'm dividing with the sum from the from from all these values but effectively this is what's happening because you're you're moving every pixel around this pixel you could think of it also in this way is that maybe i'm moving the whole image that many times so same thing okay because i'm doing it for every pixel anyway cool so this is filtering and it is a really really sort of um it's one way of making making more blurry blurry looking images that's that's cool um is it any better than doing blurs using the blur node inside of nuke no no it isn't because the blur node is pretty good at making these filters for you but the one thing that the blur node does differently is that the player node also allows you to choose how big is the area that i'm going to change um that i'm going to use so for example here we are only looking at the nine pixels immediately around this this this image in here but what if i wanted to blur further apart so let's take another example of this let's take this footage and let's look at it and let's do a little matrix and this time i'm going to use a different dimension for this so i'm going to make this matrix very wide i'm going to make it 9 pixels wide and just one pixel high because um i only want to blur horizontally so you can see it gives me a lot of these values that i could add together so i could do this so i could do one one one one one one one so what i'm getting now is an image that's nine times as bright as it was used to be whereas if i do normalize then you can see um i get the original value i've divided by nine and if i zoom into this let's see if we have some place where we have some some features where we can really see that we're only blurring in one direction uh just checking for a for a good example maybe maybe this stopped in here so before this dot was a little bit soft but if i blur it it's blurring it left to right because again what's actually happening is it's taking all nine values nine that's two nine three five six seven eight nine i can do nine there nine values and it's within this region it's averaging together all these pixels so in the middle this pixel changes kind of the least but even this pixel that used to be quite dark is going to be averaged together with these pixels that are a little bit brighter so even the middle pixel here is going to get a little bit brighter because we're adding this contribution from the from the pixels around it left to right but when it doesn't affect anything with regards to what's happening top and below likewise some of these pixels in here like this picture in here it used to be very bright but by doing this averaging over these values this pixel is now going to get darker because we're on on the left hand side these bricks are roughly the same brightness but these are much darker so the average value is again going to go down so you can see this fix is going to get darker so at the moment what i'm kind of really explaining to you is how blurring works so this is how the blur node works inside of nuke in fact i think if we do a blur in nuke and we just do it in the width and i'm going to use a box blur and let's use a width of 9 we should get exactly the same effect so that's our matrix so that's a blur and that's our matrix that actually makes this makes this blur so you can see they are identical so exactly the same result so this box in here basically refers to the fact that this filter that we are kind of parsing along this image if you consider its brightness it's it's effectively its shape is a little bit like a box so if we just kind of consider the brightness of uh of our pixels in here so just taking one line and imagining let's imagine if i draw a line through here and let's just make it a little bit less opaque and let's just take this stroke and make it kind of make it less opaque and i can do it over here i think this this one here okay so we're just going going through through that line in there so just this one line so what you might imagine is that if i was to plot uh the brightness of these pixels so we have some bright pixels then a dark pixel some bright pixels again a dark pixel and some bright pixels again in fact um dark bright dark attack bright and these ones are actually slightly brighter than that so this is if you were to just look at the values of those pixels on the on that line um this kind of representation by the way is is what you can get if you look at if you look at the waveform monitor inside of nuke so let's look at the waveform monitor which is somewhere here so wave monitors give us this kind of view of pixel values from the side at the moment because it's it's very stepped because it's individual pixels if i zoom out it will be a little bit clearer and if i just look at the luminance for now so these are the different different brightnesses of those pixels um plotted here but effectively this is this is the brightness of those those values in here so the filter that we are using at the moment it looks at this pixel and its surrounding pixels by an equal amount so it's taking exactly the same amount from from each of these so what we get is again with this we can plot something with this is that it's just looking like that so it's looking at this pixel and it's looking at these pixels around it by exactly the same amount so it's kind of like a box filter whereas we have other options which is that maybe i'm more interested in the values in the middle so maybe i would like something like an exponential filter which we don't have inside of nuke but where every filter closer to the middle wouldn't matter more so we could do something like this one two four eight sixteen eight four two one so now this filter will be blurring but more you know keeping more of the regional value but the further it goes the less of those values it uses it's a nice way of kind of making some glow effects um we don't have that built into nuke but we have a few others we have um we have a triangle which effectively um would be something like this which is that the pixel values themselves that you are using in your filter they will just kind of form a little triangle so we use more of the middle values less from the edges we also have a gaussian which is the most common one which gives you this kind of very nice soft fall-off as well so these are the blur operations that you can use and the effect of them is is just how much do they do they actually include the pixels from further further away so box kind of includes all them by exactly the same amount but then what you actually get with boxes is that you don't always get the smoothest image so why is that let's have a quick look so if i do a box filter and let's say in my image i have a straight line that goes down so i just have a straight line in my image let's look at it here similar to this actually that's a good example already but um i just have this one pixel and let's do full hardness and let's try to draw here a relatively straight line okay so i now want to make this line softer i want to blur it left to right if i use a box filter um in here let's do one one one one one one oops one one one one one you notice the result isn't actually soft because what's actually happening at this this stage is in here is that you're still using these these nine nine pixel values and if i just turn this off any of these pixels in here because they're you know these are roughly similar values these are awfully similar values but for any of these pixels in here they just happen to be getting the same amount of of dark values and bright values so all these pixels the middle value ends up being as the kind of same amount of dark and bright from both sides so when you're doing a box filter even though you're softening your image or filtering it it might not necessarily look that much softer that's my suffering happening here there so there it is that's because we are using the same amount in all these values so and this is then telling us something about the shape of this filter which is that if you have a input where you have very bright highlights very dark shadows in a area where the background is very different they will start to recreate the shape of the filter this shape of the filter was a box so if i only actually had one little dot in here and let me just try to draw a dot let's go to the next frame so let's just try my best to draw a dot and i'm just going to clean out some of the edges of the stock to make it more like a dot it's surprisingly hard in nuke to to draw just a single dot there there's a single dot so if i blur this i will get exactly the shape of my filter which is this line and this is nice because now you can actually look at what the other filter shapes look like so if i look at this box or a triangle you can begin to see that's what the triangle will look like uh so you can imagine this is like the top of a triangle where if it's pointing towards it it's brighter and this is the quadratic and the gaussian so that's kind of how they so gaussian is kind of it's using more from the middle and less from the edges quadratic has this kind of soft fall off everywhere um so quadratic actually gives you blurrier images or like softer images than gaussian but gaussian has this using a bit more from the middle but less and less from the from the the further away you go and triangle is actually like a like like a top of a pyramid so um now what do we do with all this knowledge basically um you could do something like this just for fun um if i have a picture like this and i make and this works in two dimensions as well at the moment we just made one one dimensional filter so let's make a two dimensional filter and i'm just going to draw the same dot in here so we have a little dot so we can see there's a dot in there let's make the matrix that is much bigger let's do 11 by 11. it's nice to do them as as odd numbers when you're explaining these things because the middle value will always be the same so i have a much bigger filter now wow now if i take the you know the middle value in here again i get i get the original image i'm just going to immediately normalize this as well so what if because i said that if you have this in like one bright pixel and you try to try to um if you're blurring an image which also has this one bright pixel it begins to give you the filter shape um so what we could do is what if i just try to draw a smiley face in here so if i just draw a few of these dots in here and maybe a few in here and then try to do like a smiley face in here so something like that okay what happens is that the smiley face is upside down and wrong way around so just to make this different from each other as well is that i mean it's it's hard to visualize what's happening in the script because all this all these numbers are quite quite similar looking um but uh you can just about see you know this is the three in here that's these three and this is the smile these are the ones that's the smile that's the value in the middle of the nose and this is this is the four over here which is this this four over there so so what's actually happening is that this is our new shape that we are moving across our image but if you keep moving it across the image and taking the middle value and because what we were what i was showing earlier as well is that if you're taking an image from the left the pixel from the left you're actually moving it to the right in a similar way if you're taking a value from the below you're actually moving it above so what has happened here is that there are this pixel up here um let's do something like this so i think that's roughly where we're at so so this big slop here it was looking at this value of this pixel and because it had a one somewhere here below you know when it was looking at like this that's why when we were kind of at the smile stage in here this value ends up up here but it ends upside down because we're moving the values from below above and be moving them from left to right which is why it flips that way as well and this is why this is called cross correlation so cross correlation is this process that we've been doing which is we slide a image across another image and we look for the correlation actually that's the technical term for this in a neighborhood but what happens is that it if you're if your filter and and the original image are the same around your result ends up upside down flipped around so convolution solves this problem by first mirroring your filter and flipping it over and then doing the cross correlation so that the the result actually ends up upside down again now this is actually something that we are looking at here now is looking here's a grid of numbers there's a two degree of numbers 2d grid of values and when we were looking at this picture in here we are also discussing two degrees of values so we're discussing like you know what are these values now what were they before what are they after so you know each of these pixels here has a value as well this is also a two degree of values what i would maybe actually like is is to have this 2d grid of values as a picture that i can draw rather than typing these values in here so that i could use this as a filter instead and that actually exists in nuke and this is called convolution so when we're looking at the matrix node we're doing cross correlation because you don't get this at that numeric level but if you want to actually get to this level where you're now making these filters that actually need to have some kind of a visible shape it'd be much nicer to draw this so what we do is we use the convolve node to convolve together two images so if i look at this if i take the same dot and a background in here and we draw a little picture so i'm just going to make a constant and i'm going to try to make a tiny tiny constant so let's do a crop of um i'm gonna make it quite small uh let's do 25 by 25 so let's reformat this so it's a really really small picture here okay and then let's try painting on this and let's do something like this so let's give it some hardness as well and i'm just going to do my initials another gauzy but in a kind of cool sign looking way so a and okay how cool is that right um now yes amazing so or you could do anything else any other picture that you want let's you know just just just because um we haven't drawn a cut yet let's draw a cut so here's a cut neat so with this picture here what we actually have is again it's it's a grid of numbers um you know these ones are zero uh this one here is um this one is a one and that one's like you know 0.6 or something like that but really it's a grid of numbers if you were to slide this picture across this other picture can we use this as a filter can we blur our other image using this picture and the answer is yes yes we totally can and the node we use for that is the convolved node so let's do this this is our filter this is our image and it is blurrier but actually if you have any highlights or kind of very visible parts where one part of the image is very different from the rest then in those areas i've lost this top that i drew earlier look at the summer here don't remember anymore i'll just draw another dot over here let's make a dot so i'll just make a dot here and let's look at it after convolving you can begin to see this outline of a of a cut in there just just make it make it more contrasty for you guys so it comes comes through zoom as well so you can see i've blurred it with the shape and because in convolution the it gets mirrored and flipped before you do the cross correlation then then it ends the right way around so convolution is correlation where the filter ends up the right way around but it's the same idea so so you can see any highlight that i have in the image actually takes a little bit of this shape of this cut is this useful not necessarily but this is a technique that we could actually use um when uh can you map a data from a file into the node screens values um yes in some ways this is kind of kind of kind of what we're doing um there is another way of doing this with a lot of expressions um and a bit of bit of python you could um you could write the script which uses the sampler sampler node inside of inside of uh tickle as well so you can actually sample a particular pixel's value but this does this is much easier so um another place where you might have seen this kind of filtering is in the z defocus node so zd focus node for example um works um works the same way so the seti focus note just gives you extra controls whereby at the moment how much filtering we get in here depends on the size of this filter that goes in it was 25 by 25 if you were doing a zd focus instead of convolve you can actually choose the size of your filter so if i do z defocus so z to focus the idea with that is that you use a depth pass to blur different parts of images if they're closer or further but if you don't have a depth pass you can just still blur your whole image it's just that now it gives you extra controls so for example here if i use a a filter as an input so we use a filter and i think i'll find here nice so we are already getting getting this effect of this this blurring using these these shapes and if i do if i do size i can actually control the size as well so it's kind of even even more convenient so you can make some crazy crazy filters like that okay so you can you can begin to see this this this shape of the cat in here as well again there's like some crazy crazy big cuts in there right um you can use it for subliminal advertising so you know um put escape logo into into every every every student project that you do here for example just when things go out of focus that just kind of starts subliminally saying escape escape escape so that's cool so but you know so obviously you now realize that we actually have the tools already to do this and these are good and they use kind of gpu as well so they're nice and fast so why did we kind of learn this matrix node to start with well so to understand what's actually happening underneath now let's try something else so so far we've been kind of adding values together and this is kind of doing some averaging and stuff like that but what happens if i give negative values into this because we already saw we can put fractional values to this you can do 0.2 what happens if we do a negative value let's find out so i'm just going to set these to be zero so i'm not going to normalize and i'm just going to do this as one and this one as one but this i'm gonna do as minus one interesting um so turns out it's perfectly legal you're not you're not breaking any laws so what this thing is actually doing now is you have created an edge detector and you've created an edge detector that also gives the direction of the edge so if all the values are the same so if i look at this value in here so all the values used to be one one one so if i take minus one from one side plus one from the other side i get zero what about uh this pixel above it's zero on this side zero on this side again i take one from the other still zero so you can see this is still zero what about this one in here so here what we're doing is i'm taking multiplying that side with one and that side with minus one so there's nothing here to to start with but this one has minus like it has like 89 or something like that so i get minus 89. so what this is telling me is that there's some kind of an edge and it also tells me that this edge is one where left side is brighter right side is darker because i'm taking that you know in that direction i always take from left to right in the indigo and case so i also know that there's an edge and which way this age is facing let's look at this on a real picture as well getting the same kind of results so here i'm getting a positive value which is telling me that the pixel on the left was darker the pixel on the right was brighter so if i compare this pixel and this pixel this one is brighter this one is darker so if i take the negative value for this and the positive value for this and i can't add them together i'm subtracting from the right value i'm subtract subtracting a darker value i will have something left so this gives me a positive edge over there it gives me a visible edge so you can use this to do age detections and so adding negative values gives you edge detectors and now because at you it gives you an ace detector with the direction uh you can also check the directions in the other direct side so basically if this one was one and this one was minus one so you can see now that's the bright edge on that side so we can find where that edges but also if you don't care about which age you're looking at what if i take it from both sides and i subtract them from the middle value so then you could keep sort of subtracting this and let's see the middle now has to be four you're taking the values from all around so you can start building these edge detectors using this so we can subtract values in addition to adding them because subtracting is just adding with a negative value um much harder to translate that into a picture for the convolved node so and i don't think whether it i don't know if it will even allow you to do that but um but here we kind of we've done an edge detection and you can see i've included four in here because i've taken four different edges i've subtracted one from one amount from that side one amount from that's the one from that side it's just so that it would pivot still around zero but if i wanted to put the original pixel value back to this i could do that i could add five so now what it's doing it is doing an edge detection so it's looking where there is a sharp difference between a bright edge and the dark edge and it's adding this difference to your current pixel value it is creating a sharpening effect so the pixels that used to have similar pixels around them nothing happens to them it's like uh i'm taking away blue and then i'm putting black blue still the same it's only if there is an actual difference in the original image so for example here is a bright pixels on one side dark pixels on the other side and the middle is kind of being something in between by doing this subtraction making the darkest even darker and the brighter is even brighter so making this kind of sharpening effect really intense sharpening effect for the moment but you know just to illustrate the point that you can so filters allow you to do more than just um just uh softening this check there's another another question in here cool good questions in here is this process used during motion tracking um some of this um actually yes so this shifting pixels left to right is is part of the process that you can use to [Music] to tell if you're trying to match one image on top of another image and by doing this abstraction basically it also tells you the gradient exactly tells you in which direction things are different so you might need to move them further in that direction anyways what it shows is that we can actually have quite a few different types of filters giving this giving us different blurring effects different blurs sort of uh darken effects sharpening effects edge detections let's call them that so one cool thing thing we could do with this is um is the following and just find another script in here um just to kind of put it into into an example so for example here we have this um this spaceship flying into the shot so i'm just going to concentrate on that actual part in there where we there we have the spaceship let's see how that works so here we have a little spaceship car and um there's another question about color matrix which is a different topic um it's a very different topic i will i will answer that in a moment um so uh let's let's let's look at this one more time so anyways so let's move this down here so just to show you another kind of fun example of why you might want to do h detect so here i have a picture of some some noise so here's noise and i'll soften it i've blurred it a little bit just to make it less you know high frequency and that here what we've done is uh i've subtracted the pixel from the left from the one from the right so what you could do is instead of using a transform and a from you could actually use a matrix as well so [Music] why we might want to do this is for this reason so for example here is all the if i just shuffle out the red channel in here and i'll do a matrix and i just want to do the pixels on the left and the pixels on the right what this gives me is that left and right difference in my image now i could actually maybe i might want to use a little bit more difference from the top and bottom as well and just a little bit more from the from the side so it's it's so we're kind of doing a little bit of of the differences in diagonals as well so differences in diagonals and difference horizontally but effectively it's giving me left and right difference here so on one hand side one hand i have um positive values on the other i have negative values so what this effectively tells me is like it's it's facing that way or it's facing that way or it's facing towards me so i'm kind of thinking of this this noise now as if it's like a bump map and and this tells me in in which direction things on this pump map are looking and i could do similar things with my up and down view as well matrix g are still straight and let's do that so let's do my minus one again there's there are reasons why you do care about whether you do them top to bottom or bottom top this is noise i don't care so let's say we do this for for the green green values in here as well so top to bottom and left to right and then i kind of add them together so i get this top to bottom left to right view in here and i could use this to move pixels in my image up and down randomly by this amount um yes bump maps if you actually want to see another another related tutorial um i will give you a link a link in the in the in the end but uh what this now allows us to do is that i've done some kind of um glow effects in here so here is some glow effects just using some some noise and glow nodes and stuff like that but it would be nicer if this had a bit of a heat haze feeling as well so by selecting just areas where i want this to happen which is just over there just behind those thrusters we can convert those red and green values to forward you and forward the vectors that the eye distort node can use to push pixels around so the red ones will move things left and right the green ones will move things up and down so what this will give you is this kind of a nice warp effect as if there is um as if there is like a sort of haze or something like that happening there so you can see a bit of a bit of a glow effect happening there which is cool so just because you know that's that's nice when it does that so let's do a little little ping pong bounce in here see if it if it plays back cool so just another kind of example that by using these matrix nodes you can do other things you you don't have to just blur things because we already have good learners inside nuke anyway um and if you want to blur things with a kind of custom filter then we have the convolved node or the z-focus which is also really good under gpu accelerated but you can use um matrix node to do edge detection type of things and edge detections also we have a node for that but um this is this is quite annoying to look at it's gonna view it as a repeat uh playing forward let's give it give it a bit more time but but it is what you use this for so basically here we're using it to kind of create some distortion effect um and um we're making using it to make a map where that ids.node could use afterwards so there's another question here about the color matrix the color matrix is a different node and basically what the color matrix does um is something completely different let me just show it in another another script in here the color matrix gives you and unfortunately the column matrix inside of nuke is is three by three instead of four by four but um the color matrix does something else the color matrix takes every single pixel in your image this red green and blue and it represents this red green and blue just because you asked this question so i'm happy to show you so it represents this red bean and red green and blue as um as if the red green and blue actually represent positions in space so let me just make really really small points for you so this is all the all of the pixels in this image can i make them really small smaller still uh maybe just even a bit smaller than that so if all the pixels in my image are colors and those colors in my image represent a position in space then the color matrix allows you to do transformations on those um values for the red green and blue so for example if you only did the diagonal you could make the red twice as bright so now it is twice as red so in here we're effectively multiplying red green and blue but basically what's happening in here is a matrix and vector multiplication which is a big topic for another day the reason why you would use this is um if you start combining these other values in here in addition to just two in the middle which basically gives the scale of the red green blue you could for example combine the upper quadrant in here to figure out an angular connection between those colors so you could um represent some of these things as rotations so if you have a normal map you can rotate your normals by rotating these colors by linking them to a something like an axis node at the inner axis node you could you could look at the first first first few few values in here so for example you can see if i rotate things in x or in z this is the values that i could put into into my into my column matrix so color matrix is actually doing a transformation in um let's see if this works does it work it does not work i think you have to do them one by one yeah yeah so it allows you to for example rotate your colors in in z which isn't necessarily massively meaningful if you rotate your colors in z because like what does that actually mean as you can see my image is getting a little bit sort of doing a huge hue rotation in here but if you actually look at this as the as the position to points what you should see is some rotation happening here for this for these values in here so i'm rotating those color color values and now this color values represent position so i'm rotating them in the space but actually yeah you're doing like a hue rotation anyways so column matrix node totally different topic uh for another time uh useful for when you're dealing with normals and figuring out which way things are pointing so um yeah useful useful to know too cool um that's that answers that question um cool so groovy um yes so we saw displacement maps earlier and um and just the one last one that i kind of wanted to show you uh let me just show you this um so there's some other tutorials that deal with filtering um i have one about uh digital painting and spatial frequencies so they look at the filtering aspect in a kind of more global fashion this was looking at in this very very granular fashion that what happens for this pixel when you filter it spatial frequencies explains what happens to a whole image when you're doing filtering so this kind of brightness differences that i was talking about earlier what happens if you actually make images smoother or or less smooth so what actually happens when you filter images just from the blurring point of view nothing to do with edge detections in here but another one that you might want to look at as well is um is if you go to search in the escape studios one i think it's called lighting uh so my relighting hacks um we are actually doing things in here as well which is um this is this is all based on other tutorials from other people as well but we are using the same matrix nodes to again create great this left and right difference and up and down difference for an image based on its brightness and once you have the left and right and up and down difference you can also from those you can figure out is it is something pointing towards you or not towards you because if there is no difference then it's pointing towards you so you can figure out the blue channel out of the red and green as well to get an actual red green blue uh normal pass uh the way that you're expected to used to used to seeing it and then this normal pass you can actually use to relight 2d elements such as this blood splattering here just by adding adding a bit of noise to it or this smoke plume by by kind of making it look as if there's some kind of specularities and things so so other other resources to look at um from us as well and obviously the best thing is to just come and study with us because then then you can learn these things from you know us directly and ask us questions more often um but yes that's everything i wanted to say you

Channel: Escape Studios

Views: 1,051

Keywords: Escape Studios, Escape, studios, VFX, Visual Effects, CGI, Maya, Autodesk, Nuke, Foundry, I want to learn, tutorial, modelling, rendering, dynamics, texturing, animation, renderman, compositing, ncloth, motion graphics, design

Id: xnWa5wTsNKA

Length: 64min 3sec (3843 seconds)

Published: Mon Nov 08 2021