How to Make a Toon Shader in UE5

but soon cheater like the ones you can see in games like Zelda breath of the wild or Genji impact allows us to make our 3D scene look like it was taken out from a comic book or a 2d animated show and the way it works is easy to understand since all we need to do is take the lighting of our 3D scene and instead of having smooth transitions between the parts that are lit and the ones that are in Shadow as you can see right here we Define a threshold for the amount of light an object in the scene receives and make it so areas that exceed this threshold are considered as fully lit and areas of the scene that do not receive this minimum amount of light are considered as in Shadow okay so here I'm in honored Engine 5 and the first thing I'm going to do is create a material in which we are going to implement the logic of our tune Shader so I'm going to press Ctrl space on a keyboard to show the content browser right click select material and then I'm going to rename it to PP underscore tune Shader for post process tune Shader I'm going to open it and here by default a material allows us to modify the look of an object here for example if I press 3 on my keyboard and left click in the graph then connect this new node to the base color output and change its color let's say a pink one for example this will change the color of the sphere in the viewport however what we want to do here is instead of modifying the look of an object we want to modify the look of what is shown on screen in order to do this we are going to change the type of material by going in the details panel right here and then I'm going to change the material domain which is available right here material domain I'm going to change it to post process so here as you can see the screen turns black not pink because now if you want to modify the look of the screen we need to plug our color in the emissive color instead of Base color so here I'm going to click right here and drag the wire in the emissive color and there you go we have the look of our screen that was changed to a pink color okay so now a pink color is great but to apply our tune Shader first we need to get the look of the scene as is and plug it in the emissive color so I'm going to delete this constantly node I'm going to right click type scene texture select the scene texture node right here then I'm going to select right here in the scene texture ID I'm going to click and select post process input 0 this one post process input 0 represents the look of the scene without any effect applied on it so here if I'm going to plug the color in the emissive color right here you can see that we created a pass through past process effect which means that this change is nothing to what the screen is supposed to show so here just so you know how it's possible to modify the look of the scene we are going to desaturate it to give it a black and white effect by dragging the syntaxure node here I'm going to drag a wire from the syntaxure node and tap this saturation I'm going to select the desaturation node this one if I plug the emissive color right here you can see that everything turns gray so now that we have a simple black and white effect and before applying it to the scene we're just going to change another parameter by going in the details panel right here I'm going to search for blendable location I think it's at the bottom of the details panel here you have the blendable location you are going to select this and instead of taking the after tone mapping we are going to check the B4 tone mapping option so I won't go into details here but just so you know if we don't change this now later this will make the screen when we'll Implement our tune Shader okay that's great so now that we have our base Shader we are going to go back to the viewport so that we can apply it to our scene so first we are going to create a material instance that will allow us to modify the parameters of the Shader and see the result in the viewport without having to recompile the whole material so I'm going to press Ctrl space on my keyboard I'm going to right click on the pp tune Shader was processed on Shader and select create a material instance I'm going to rename it to PPI for post process instance underscore tune Shader then to apply this material instance to the scene I'm going to go right here click on it select volumes and then select post process volume I'm going to add this post process volume to the scene this will be right here okay so I have my post process volume and then in the details panel on the right I'm going to put myself in the middle of the screen here in the right you have the details panel you are going to search for post process material this is right here rendering features post process material I'm going to unfold this press on the small plus sign then click on choose right here asset reference and then here Android is going to ask me to select an assist and I say add an asset so I'm going to press PPI on my keyboard and select the PPI tuned Shader so here you might think that it doesn't work but by default a post process volume applies the effect assigned to it only if you go within its bounce so here if I go in the bounce of the post process volume right here you will see that everything becomes gray so this means that the material was applied correctly so to apply this effect to the entire scene we need to check a parameter in the details panel which is called infinite extent so the infinite extent it's right here Below in the post process volume settings you have the infinite extend Unbound I'm going to check it and voila now that we've finished the setup of our post process material we can start modifying it to apply our tune shading effect okay so now that we know how to sample what is shown on screen we are going to extract the light buffer from it so that we can access the amount of light each pixel of the screen receives and later performs some thresholding on it to create our tune shading effect so here to be able to extract the light buffer first I'm going to get the base color of the scene without any lighting applied to it I'm going to select the scene texture node press Ctrl D on my keyboard and here instead of selecting the post process input option I'm going to select the base color as stored in G buffer option if I plug it in the emissive color output and then press apply if I go back to the viewport you will see that we have only the color information from the scene without any lighting applied to it so here I'm going to get the desaturation node right here press Ctrl D on my keyboard and then plug this in to desaturate the base color of the scene now what's cool about this is that if we divide the desaturated version of the scene by this we are going to remove the color information from the render and only keep the lighting information so here I'm going to drag a wire type divide and plug this in so I'm going to divide the post process input 0 desaturated by the base color desaturating so finally before plugging plug in this to the output of the Shader first I'm going to add a saturate node saturate node to make sure that the values stay between 0 and 1. then since what we've created outputs an RGB color and since the RG and B components of that color all have the same exact value we are going to use a component mask component mask node to extract only one of these components so here on the left side of the screen you have the components that are outputted by the component mask and here I'm just going to select the red component okay so now let's comment this bit of code let's say that we can extract the light buffer I'm going to plug this in and there you go if I go back to the viewport you will see that now we have the lighting information of our scene pixels that are white represent the pixels that are lit and pixels that are gray or black represent pixels that are in Shadow okay so now that we have our light buffer we are going to perform some thresholding on it so that areas of the scene that receive an amount of light that is greater than the threshold are considered as lit and areas that receive an amount of light that is smaller than the threshold are considered as shadowed so I'm going to go back to my Shader and contrary to what you might think it's quite easy to achieve since all you need to do is add a step node to the graph like this I'm going to right click here type step I'm going to select the step node right here then I'm going to plug this in the X input this in the emissive color output of the material then on the Y input of the step node I'm going to right click select promote to parameter and rename this parameter to late Shadow threshold I'm going to set its default value to 0.5 and here by doing this we tell Unreal Engine to return a value of 1 so a white color for pixels that receive an amount of Light Above This threshold and the value of 0 for pixels that receive an amount of flight equal to this threshold or below it so 0 equals a black color so that's great at all but if I go in the viewport you will see that the transition from black to white is a bit too hard so now we are going to try to smoothen it a bit so what we would like to do here is still use the light Shadow threshold but along with that we are going to create another parameter called light Shadow transition size I'm going to select the light Shadow threshold press Ctrl D on my keyboard to duplicate it then I'm going to rename it to late Shadow transition size then I'm going to set its default value to 0.2 and here this parameter will help us Define as its name suggests the size of the transition to be more precise about what we are trying to achieve here if I set the Threshold at 0.5 and the transition size at 0.2 this means that we want the transition to go from 0.4 to 0.6 and what we expect our Shader to do is return 0 for pixels that receive an amount of light below 0.4 one for pixels that receive an amount of flight above 0.6 and the smooth gradient going from 0 to 1 for pixels that are in the transition zone in other words for pixels that receive an amount of flight between 0.4 and 0.6 and to achieve such a thing we need to use a smooth step node instead of a step node so here I'm going to right click type smooth step this and this node is exactly what we need since the way this node works is as full follows first it gets a value as an input as well as a minimum and a maximum value and then it returns 0 for values that are below the minimum one for values that are above the maximum and a smooth gradient for everything in between now I'm going to code something and then I'll explain it to you how it works so here I'm going to delete the step node then I'm going to get the light Shadow transition size multiply it by 0.5 then I'm going to get the light Shadow threshold drag away from it get a subtract node plug the multiplayer in the B input then drag another wire type add then plug the multiply node in this and then I'm going to get the smooth step node plug the result from the light buffer here in the max I'm going to plug the add the subtract I'm going to plug the mean and then I'm going to plug this in the emissive output of the material then I'm going to add a small command I'm going to tape late Shadow transition okay yeah so here what we've done is exactly what we wanted we take the transition size this one divided by two and remove it from the threshold to get a value of 0.4 that we plug in the mean input then we get the transition size divided by 2 and add it to the threshold to get a value of 0.6 that we plug in the max input so now if I go in the viewport you will see that we have a smooth transition from black to white of course here if you want you can still go back to a hard transition by setting the light Shadow transition size to 0. so if I save and go back to the viewport you will see that the transition is heard again okay so now you can see the code that we've created so far as the new lighting of our scene however I think that you'll agree with me when I say that the black color for areas that are in Shadow and white color for areas that are lit is a bit too harsh and not interesting at all so to make this lighting more interesting we are going to implement something that will allow us to tint the light and simulate for example a bluish lighting for when it's daytime and a slightly orange lighting for when it's the end of the day so what's good about the implementation that we've done is that since the value of the light amount is always between 0 and 1 all we have to do to achieve such a thing is to use a LARP node so here I'm going to right click type lurp and select linear interpolate this one then I'm going to plug this in the alpha input this in the emissive color and here I'm going to press V on my keyboard and click in the graph to add a vector parameter that I'm going to rename to shadow tint then I'm going to select this press Ctrl D on my keyboard to duplicate it and I'm going to rename it to light I'm going to plug it in the B input the shadow C in the a input then the shadow tint I'm going to select this change its color to let's say something like a gray color put an alpha of one here and the light tinted I'm going to double click on it and select let's say your white color Alpha as one press OK and here by doing this we sell Unreal Engine that if the value from the smooth step is equal to zero in other words when a pixel is in a shadowed area instead of setting its color to Black we change it to the color defined by the shadow tint parameter on the other hand if a pixel is in a lead area instead of returning a white color our Shader will now return the color Define By the Light tint parameter and the last but not least for all the pixels that have a value different from 0 and from one the alert node will blend the shadow tint and light tint based on the amount of light they receive so for example uh the color of the pixel that has a value of 0.3 will tend to be more like the shadow tint than the late tint and the color of a pixel that has a value of 0.5 will be a perfect blend between the shadow tint and the light tint so now things are starting to get really interesting and if you go in the viewport and open the material instance press Ctrl space open the PPI tune Shader here you can change the parameters the light tint and the shadow tint you can set them to interesting Shadow and color teens to get a cool result however based on the what you need to achieve you might think that you have a small problem with the Shader okay so here it's a bit subtle but if you try to move the camera around maybe you'll find some weird artifacts appearing like inconsistent Shadows like you can see right here so I've investigated this issue and found out that it was due to Lumen the global illumination solution of Android Engine 5 since Lumen calculates Reflections and Global Illumination in screen space I think that this causes inconsistency in the lighting and to fix this all you have to do is deactivate Lumen by going in the project settings so edit project settings right here then you go in engine this and search for rendering and here if you go down you will see that you have Global illumination and Reflections set to Lumen you can select this and set it to none select the reflection method and set it to none instead of lumen and if you save it and go back to the viewport you will see that there's no more artifacts going on in the scene so of course doing this will remove the global Illumination in your project so you need to make sure that you don't need it before deactivating Lumen and just so you know later in this video I'll show you how to apply the effect only on some actors of the scene so if you plan to apply this Shader only on your characters like in the game gen impact for example then you can turn on lumens since it doesn't seem to create these artifacts on skeletal mesh like the characters for example so as for me I don't mind these inconsistencies in the lighting so I leave Lumen turned on okay so now that the new lighting of our scene is set up we need to apply it to the scenes so applying a lighting to a seed is as simple as blending its color with the base color of objects by multiplying them together so here we are going to get the scene texture node we used earlier to get the base color of the scene without any lighting applied to it so I'm going to get this one the scene texture base color and to press Ctrl D on my keyboard to duplicate it bring it right here then I'm going to multiply this by what we've created so far so here I'm going to drag a wire from here multiply plug it right here and this right there so here Android engine is giving us an error right here because the colors from the light and Shadow were tint parameters are composed of three components red green and blue whereas the one from the scene texture node is composed of those three components along with an alpha component that represents the opacity of the color so to solve this error all we need to do is remove the alpha component from the color of the syntexture node with the component mask node that we've used earlier so here I'm going to drag the wire from this type component component mask I'm going to plug this in the B input and here I'm going to check the r g and B component so that we have three components in the color from the scene texture node so here if I save it and go in the viewport you will see that the Shader seem to work seems to work correctly however there's a problem with the way it works here if I place for example a point light I'm going to go right here select lights and place a point light in the scene this way for example I have my point light right here the lighting information is taken into account but if I change the light's color to something like a pink color you will see right here that the lighting color doesn't change okay so to solve this issue we are going to use a little trick that will allow us to add to this result the original color that the scene had before we applied our tone Shader on it so here I'm going to put myself on the right side of the screen I'm going to go back to my tune Shader and here what I'm going to do is that I'm going to duplicate the scene texture node right here Ctrl D I'm going to select the post process input 0 to get the color of the scene without any effect applied on it then I'm going to multiply to add a multiply node I'm going to right click on the B input select promote to parameter I'm going to rename this new parameter to base render blend amount I'm going to set its default value to 0.5 and then I'm going to add this to this okay by doing this I'm going to add the color from the scene before the tune Shader is applied so here the base renderer blender Mount allows us to Define how much we want the original color of the scene to be added on top of the Shader if you don't want any color from it you can set the base render blend amount to zero and if you want all the information from the base render you can set it its value to one so here if I save and go back to the viewport you can see that we have our pink lighting that appears on the rendering okay so now that we have our base Tunes Shader now we are going to make it so the effect is not applied to things that are too far away from the camera like the sky for example so first of all we are going to create a parameter in the tune Shader that will help us Define the maximum distance at which the tune shading is applied so here I'm going to right click type scalar parameter I'm going to rename it to effect max distance and I'm going to set its default value to a really big value like let's say for example 500 000 it's over 500 000. okay then we are going to get the sin depth node scene depth node that will allow us to get the depth of each pixel of the screen and we are going to compare its value with the parameters by using a step node I'm going to plug this in depth in the y in the X sorry and the effect max distance in the Y input so here for each pixel that is further than the effect max distance the step node will return 1 and for pixels that are closer than the effect max distance the step node will return 0. now all we need to do is tell Unreal Engine that we want to apply the Shader on pixels for which the step node returns a value of 0 and not apply the Shader on those for which the step node returns a value of 1 so to do to do something like this I'm going to add alert node I'm going to right click type linear interpolate and just so you know I can press L on my keyboard and left click in the graph to add alert node so here I'm going to plug the step in the alpha then I'm going to plug the result of my tune Shader in the a input and here in the B input I'm going to get the scene texture post process input 0 I'm going to duplicate it to get the rendering from the scene without any effect applied on it then I'm going to drag a component mask select r g and B and plug this in the B input okay by doing this we say that if a pixel is closer than the max distance we get the tune Shader and if the pixel is farther than max distance we get the rendering of the scene without any effect applied on it so here if I save it and go in the viewport you will see that the Skybox is not affected at all so to better show you the result I'm going to open the PPI iTunes Shader and let's say that I add a bluish color to the lighting or yellowish color this for example and a pink color for the rest you will see that the sky is not affected by the effect okay so now we are going to add the ability to apply our tone Shader only on some of the actors of the scene this can be useful for example if just like in the game Gene impact you want to apply the effect only on the characters and leave the environment as is without any tune shading applied on it to be able to do something like this we are going to use the custom depth buffer and for those of you who don't know what the custom depth buffer is you can see it as an additional depth buffer in which you can choose What actors of the scene you want to sample and to better understand how it works if I go here in the lid select buffer visualization and show the custom depth you will see that the screen becomes completely black since nothing was added to the custom depth buffer if you want to add an actor to the custom Dev buffer you just have to select it in the viewport and here in the details panel you're going to search for custom and check the render custom Dev pass option as soon as you do it you will see that the selected actor becomes gray which means that it has successfully been added to the custom depth buffer okay so now to make it work I'm going to check if an actor is in the custom develop buffer or not in my tune Shader so here I'm going to go back to lead mode then in my tune Shader I'm going to grab the code for the Skybox and put it over here then here I'm going to right click type scene texture select the scene texture node and here in the syntaxure ID I'm going to select the custom depth option then I'm going to drag a wire type component mask and select the red Channel only so by doing this I'm sampling the custom depth and when I sample the custom depth buffer it will return the depth of a pixel if it's present in the custom depth buffer and if this pixel is not in the custom depth buffer it will return a depth that is really big like a hundred thousand for example so now that we have this information we can compare what comes out of these nodes with a really big value let's say I'm going to right click type step I'm going to get the step node here I'm going to press 1 on my keyboard and click I'm going to plug it right here okay so just so you know this is a constant right click type constant and select constant this is just a value you can select this constant and put it to a really big value let's say for example 500 000 for example here and by doing this if the custom Dev buffer return returns a really big value in other words if the pixel that is sampled is not in the custom depth buffer the step node will return one here to make it work with the code I'm going to create later I'm going to inverse this result with a 1 minus node here I'm going to drag wire type 1 minus and so by doing this if the pixel is in the custom depth buffer it will return one and if it's not it will return zero so now just like we did when we when we isolated the sky from the effect we are going to use a lerp node so I'm going to go to the sky code like here like this I'm going to select this part of the Skybox isolation code I'm going to press Ctrl D on my keyboard and here I'm going to get this I'm going to plug the multiply node right here in the B post process input 0 I'm going to plug it in the a input the sin texture custom depth I'm going to plug it in the alpha and this I'm going to plug it in the other lurp node okay so now what I've done here is that if this code returns 1 which means that the object is in the custom depth buffer then we are going to apply our tune Shader if not we are going to show the color of the scene without any effect applied on it so here if I go in the viewport you will see that the effect is applied only on this sector okay if I select this one and select the static mesh then custom depth you can see that this is applied on the object however if I go underneath the platform you will see that we can see that the actors are shown even when we place them behind other objects so now I'm going to go back to my Shader and to solve this issue we are going to add some nodes to the graph and then I'm going to explain to you how they work so here I'm going to get this part I'm going to right click type scene depth then from this in depth I'm going to get the component mask duplicate it plug it right here then I'm going to get the scene depth drag a subtract node and plug the custom depth in the B input and then I'm going to get a step node right here I'm going to plug the subtract in the X input remove the connection to the Y input then I'm going to multiply this by this and then I'm going to plug this in the node so what this code does is it checks if the depth of a pixel is equal to what is stored in the custom depth if it is then this means that not only is the pixel in the custom depth buffer but it's also the closest one to the camera and it's not obstructed by another object so we make sure that the object is in the custom depth buffer and that it is in front of the camera so now before going in the viewport to check if it works we are just going to add the node that will allow us to activate or deactivate this feature so here I'm going to drag a wire type switch parameter and then I'm going to select the static switch parameter here I'm going to rename it to custom depth buffer then I'm going to double click on this wire to create a reroute node plug it in the false output and then plug this right here okay so if the custom buffer switch parameter is equal to true this will activate the custom depth buffer feature and if it's false this will deactivate the custom depth buffer feature so now I just have to apply it save it and if I go to the viewport you will see that our tune Shader works correctly so that's great enough if you want to upgrade this tune Shader like this by adding outlines to the actors of the scene you can have a look at this video and I'll see you over there to make our trip power greater than this threshold where it's this

Channel: Evans Bohl

Views: 98,136

Keywords: Unreal Engine, Unreal Engine 5, Toon Shader, Cel Shader, Toon Shading, Cel Shading, UE5

Id: mzydOmgN7mc

Length: 31min 15sec (1875 seconds)

Published: Tue Oct 25 2022