How to use the stencil buffer in Unity

hey everyone today i'm going to give an overview of how to use the stencil buffer in unity the stencil buffer is a 2d buffer of integers with the same dimensions as the viewport for each pixel in the viewport that your geometry covers you can set a value in the stencil buffer between 0 and 255. you can also read from the stencil buffer and modify your rendering based on what's currently there there are many other tutorials that cover specific visual effects that can be accomplished using the stencil buffer but they rarely explain the somewhat confusing syntax that's what i would like to do today to interact with the stencil buffer you must use special commands you can add to your shaders we'll start by adding a stencil block within either a sub-shader block or an individual pass block there are several optional commands we can add here but the two that are always required are ref and comp ref is basically a place to store a single integer that other commands will read from this makes it a bit confusing depending on the other commands ref may be the value that you write to the stencil buffer or it might be the value that you compare against the current value of the stencil buffer or it could be both basically think of ref as a variable declaration even though it looks like a command comp defines how you want to compare the value from the rough command with the current value of the stencil buffer you can for example test if the value is less than greater than or equal to the value in the stencil buffer you can also say that you want the comp test to always pass or never pass if the comp test passes the fragment function will run and the pixel will render normally if the comp test fails the pixel will be skipped in addition to this behavior you can also specify other things to happen when the comp test passes or fails this is accomplished using the pass and fail commands as an example let's try writing 1 to the stencil buffer for every pixel our geometry covers we start with a ref value of 1. for the comp command we specify always this means that the comp test will pass no matter what is currently in the stencil buffer the object will appear normally and in addition it will also run the pass command the pass command is set to replace this means it will replace whatever is currently in the stencil buffer with the ref value 1. in this example the ref variable was only used by the pass command now let's make a shader that checks what's currently in the stencil buffer and only renders where the value is already set to one again the ref value is one this time we want the comp test to pass only where the stencil buffer is equal to one because we're only reading from the stencil buffer and not writing anything we don't specify a pass or fail command these two examples cover a very common use case for the stencil buffer the first object writes to the stencil buffer then the second object only shows up within the shape cut out by the first object to make this work the first object needs to be set to render before the second object often you'll want an object to write to the stencil buffer but not be visible itself this can be accomplished by adding color mask 0 to the sub shader this stops the shader from writing to any of the color channels you will still need to define a fragment function but its output will not be used in addition to the pass and fail commands there is also the z fail command it allows you to specify an operation to run if the comp test passes but the regular depth test fails in addition to replace there are several other stencil operations you can specify for the pass fail and z fail commands from here i would recommend watching some other videos on how to create specific effects with the stencil buffer understanding the syntax should make them easier to follow if you found this video useful and maybe more importantly if you think other people would find this video useful consider giving it a like it makes a big difference in youtube's algorithm thanks for watching

Channel: Scott Steffes

Views: 48,006

Keywords:

Id: -NB2TR8IjE8

Length: 4min 9sec (249 seconds)

Published: Sun Jul 05 2020