Impossible Geometry with Stencil Shaders in Unity URP

impossible geometry can be found in many games you might remember this part of antichamber where there's these cubes and each face contains different objects which seemingly inhabit the same physical space ooh mind-blowing there's a kind of shader feature which is perfect for this trick so today that's what we're going to learn about we'll be using universal render pipelines features to make this as flexible as possible all my videos are funded and made possible by my patreon supporters thanks very much for your continued support stencils are a shader feature which let us set up rules to control whether to render certain objects or parts of objects when rendering objects we can tell unity to remember which parts of the screen contain that object even if we decide to make the object completely invisible later while rendering an entirely different object we can ask unity hey was this bit rendered by that other object earlier yeah i guess i'll just discard the current pixel then or we could say i only want to render the current object if i rendered a stencil here previously maybe you can see how this is going to be helpful for making a possible geometry rooms like the ones in antechamber first let's write a mask shader to define where it's okay for unity to draw one set of impossible objects in the example i just gave this is the first object i rendered stencils work by reading writing and comparing individual values attached to each pixel on screen by default all pixels have a stencil reference of zero so we'll start by overwriting the value with this shader stencil ids can run from 0 to 255 so i'll set up a property called stencil id so we can create lots of materials with different ids we need to write the new stencil value before any other shader needs to read it which we usually do by running the sensor mask shader before any opaque geometry queue however we're going to use urp's features to force other objects to render after this one so we'll stick with the standard geometry queue inside the pass we need to make the object invisible because we don't actually want to see this mask object at all we can do that by saying blend01 which means for the color of this pixel take zero percent of the color output by the shader and 100 of the color already rendered at this pixel and then add z right off to prevent this shader writing to the depth buffer which would screw up the rendering of other objects then we'll write the stencil we add a stencil block complete the curly braces and put our stencil logic inside first is the reference value we could say ref 1 which would use a stencil value of 1 or f2 which would use the sensor value of 2 and so on or we can write ref stencil id because we already set up a property for this it always pays to plan ahead next we will add a stencil test which compares the rev value we just defined with whatever stencil value is already set on this pixel depending on where the comparison passes or fails we can change the stencil file for this pixel in different ways we're not actually interested in comparison for the mask shader and we always want to override the stencil value so we can say comp always which means the stencil test always passes the opposite would be comp never which always fails finally we tell unity what to do when the test passes or fails an important detail here is that we'll have to pass a depth test too if both the stencil and depth test paths then we want to replace whatever stencil value is attached to the pixel but the new one we defined on the shader which we do by saying past replace there are other kinds of behavior which we will see later if either test fails then we can say fail keep which means unity will keep whatever value was already in the stencil buffer for this pixel that's all we need to write in the shader file so let's put back over to the unity editor it's time for a little setup i'm going to put two objects in the same physical space and use stencils to pick between them with the help of urp renderer features for this we'll need to add two layers which i'll call stencil layer 1 and stencil layer 2 and assign each object to a different layer i'll create two materials using the stencil matte shader we just wrote with a sensor value of one and two respectively then create two quads in front of the two objects and assign a material to each quad for now you shouldn't see any changes to the two objects but the quads should turn invisible now it's a render of each of time renderer features the eurps tool for overriding certain aspects of rendering which we can use here to play with the stencil values being set by the two quads i'm doing it this way because the traditional method for dealing with stencils is to write custom materials for all the objects beside just the mask quads but i want to be able to stick any old shader on these objects without needing to add stencil functionality to them and just let your p deal with the stencil comparisons first find your forward renderer in a default urp project is in the settings folder it should look like this the first step is to head to filtering and untick stencil layer 1 and so layer 2 on the opaque and transparent layer masks your objects should disappear from the scene view when you do that then head down to the bottom and click add renderer feature and then choose render objects i'll name this stencil one opaque we'll leave the event as after rendering opaques which forces unity to render after the default geometry queue that's why we didn't need to change the cue in the shader we wrote under filters we want the layer mask to just use stencil layer 1. the opaque bits of the object should pop back into existence in overwrites tick stencil and this is where the fun begins we want to use value 1 and now we can decide how to render any object in stencil layer 1 which happens to have a sensor reference of 1 which should only be the pixels behind the mask squad if we change the comparison function to equal then we're asking unity does this pixel have a stencil value equal to one and only if this is true the object gets drawn we don't need to change what the stencil buffer value is so both the pass and fail operations can use keep then we will add a second render object's feature called stencil 2 opaque this one is the same as the first except it will use stencil layer 2 and only sensor layer 2 for its layer mask instead and the stencil value will be 2. now we can pan the camera around and see both objects appear in the same space but only when viewed from the correct side if you're planning on using transparent objects as your impossible geometry then go ahead and add two additional render objects features with the same settings as the first two except now the event needs to be after rendering transparence and the cue needs to be transparent and if you want more than two impossible objects then you can add more layers more sensor masks using different stencil ids and more render objects features in the same way if you're looking for other ways to use stencils i briefly used them in my portals video to mask out the edges of the portal thanks for watching [Music]

Channel: Daniel Ilett

Views: 46,037

Keywords: unity shaders, unity shader, shaders, game design, stencil shader, shader, impossible geometry shader, antichamber shader, antichamber impossible geometry, shader code, unity stencil shader, gamedev, unity tutorial, stencil buffer, urp, unity urp, unity urp tutorial, universal render pipeline, unity urp renderer feature, urp shader, unity shader tutorial, daniel ilett shader, unity tutorials, daniel ilett shaders, daniel ilett effect, unity material

Id: EzM8LGzMjmc

Length: 7min 2sec (422 seconds)

Published: Wed Jan 05 2022