Shader Fundamentals - Image Based Lighting

[Music] hello and welcome to another episode of making stuff look good today we'll look at the technique that goes by a few different names but it's best known as image based lighting or IBL IBL is exactly what the name implies lighting based on data stored in an image before we break into some code let's have a super quick high-level overview of physically-based shading lighting in games is typically modeled as the sum of four parts direct diffuse light direct specular light indirect diffuse light and indirect specular light first let's nail down the difference between diffuse and specular lighting diffuse reflectance models how a matte surface scatters light for example in earlier videos we've used n dot L or the lambertian reflectance term as a model for diffuse diffuse light can show the detail of a 3d object but it can't really convey material definition other than matte and rough surfaces no diffuse light will trick our eye into thinking an object is glossy for this we need specular reflectance where a matte surface scatters light in all directions a very glossy surface will mostly bounce light in a single direction the more that reflection direction aligns with the direction to our camera the brighter the apparent reflection is the relationship to view direction is what makes highly specular materials appear glossy as the camera moves about the light appears to dance along the surface of the object just like polished metals and plastics do in real life now let's look at the difference between direct and indirect lighting direct lighting is what we calculate given a specific light source like a directional light or a point light so our lambertian reflectance from before can be more specifically classified as a model for direct diffuse this is how we model a strong light source in a scene a directional light representing the Sun is a typical use case for direct light but a single torch in an otherwise pitch-black cave would also be a good candidate for a direct light source indirect lighting is a model of what happens to our direct light and other less important light sources after they hit the surface of an object when light rays hit a matte object they absorb only part of the light scattering the remainder in all directions those rays then hit other objects which partially absorb the light before bath this continuous bouncing illuminates more than what was hit directly by the light source and we call this indirect diffuse lighting finally in the case of indirect specular we are modeling how the light bouncing in the scene reacts to highly glossy objects depending on how glossy the material is this light can range from a soft shimmer of objects to a perfect reflection of the surrounding environment in an accurate simulation when rendering a pixel we know all the lighting that contributes to that pixel but in reality even a scene with a single key light source and just a few objects to bounce off of quickly becomes highly complex if we want to know all the light that bounced around and ended up illuminating our pixel enter IBL a way of modeling the indirect lighting of a scene will continue on ignoring direct light and focus on light bouncing all around the environment rather than calculate all the ways light bounces what if we capture the way light looks from a single point in all directions and store that into a look of texture the lookup would only be correct for a single point but it happens to be a good enough approximation for other nearby points the texture we store this capturing will be a cube map cube maps are special textures that can be sampled using a 3d texture coordinate which corresponds to a normal direction you can imagine this as having a cube with a different image on each of its six faces pointed inwards if you have a point directly in the center of the cube and specify a direction you'll hit exactly one texel of the cube map so if we wanted to create a cube map that captures lighting information and our scene from a specific point it would be like having six cameras each with a 90 degree field of view such that they all form a cube to keep things simple we won't concern ourselves with capturing data from a real scene and instead we'll use some nice-looking pre-rendered cube maps made specifically for IBL which I'll link to in the description let's start with the indirect diffuse here we have a beginning point for our shader that takes an albedo on a normal map right now it calculates the perturbed world space normal using the map but then it just returns the albedo color as is let's make the object diffusely lit using our cube map we'll start by adding a texture property to the shader where we can assign a cube map it will default to black so that one no cube map is set the object is unlit now in our fragment shader will sample the text cube using our perturbed normal then we'll multiply the albedo color by the sampled indirect diffuse color and return that it's not a bad start but something is off let's remove the maps for a second such that we just see the sampled in direct diffuse value of course we just see a crisp image of the cube map all the various details and bright spots are coming through crystal clear that's not exactly what we'd want for our diffuse lighting we would expect to see these lights spread over the surface in a more ambient and visually appealing way the solution blurring to simulate how all these high-frequency details would affect the surface of the object at a distance we can blur the image in unity cube maps have some import options that will actually blur the image for us I'll select specular convolution for this text cube the other important options here are generate MIT maps and trilinear filtering well toggle those then hit apply unity will prepare several versions of our cube map at decreasing resolutions and store them in the mid levels MIT maps are usually just lower resolution images of the same texture but in this case each step in the MIT chain will be progressively blurred by the time we get to mid level 5 we can't make up the details of the scene at all but we do get a sense of the overall ambient lighting in the scene back in our shader code we'll change the text cube function to a text cube LOD this function allows us to sample a specific MIT level of our cube map as the fourth value in a 40 vector we'll wrap this into a helper function that takes the normal and a mid level so we don't have to fuss with creating a new vector every time we want to do a MIT map text cube sample now we can sample our lighting cube map with a specific map level in this case I'll use a compile-time constant of 5 so my direct diffuse values will always come from the 5th MIT level of our cube map in the editor our sphere now looks like it is ambiently lit by a variety of sources in the environment and if we show the Sky box we see the lighting matches nicely bright white lights come from the entrance to the Tokyo Big Sight Exhibition Building and the soft pink values bouncing off the ground illuminating the underside putting our rocky textures back on the material it looks quite plausibly lit by the environment looking at a few more materials we see the problems that arise when you only model diffuse lighting the three materials start to fall flat and look the same even though in reality hardwood limestone and brick all look quite distinct this is where specular reflectance comes in we might expect a bit of shine in the limestone and the hardwood might even be so glossy that you could see a clear reflection in its surface so let's go back to our shader and add support for indirect specular light first we'll need to add another interpolator for the eye vector which is necessary to make it appear as the light moves on the surface of the object as our camera moves in our vertex shader will calculate the eye vector as the normalized difference of the verts world position and our camera position essentially we'll have a vector that points to the vertex from our camera note that many workflows will use a view direction which is a vector in the opposite direction of the eye Veck this vector is used for many direct specular calculations but in the case of IBL we just end up flipping it again in the fragment shader so we'll stick to the eye vet for now and our fragment shader will need to calculate a reflection vector with which we can sample our cube map HLSL has a reflect function to do just this but let's take a closer look at the math behind it given our I've Ector and a surface normal we want to find this third vector bouncing off the surface of the object through the normal here's the formula for calculating our reflection we can convince ourselves of how this works visually here's our normal vector and the eye vector and the reflection vector we wish to create move the eye vector so it's positioned at the origin then we show the dot product here as the eye vector projecting onto the normal vector the dot product is a scalar value so this part of the formula shows that we multiply it by the normal and two and finally we see this term is a vector subtraction from the eye vector which brings us to this point a voila our reflection vector sampling our cube map using our new vector R and simply returning that we see we get a perfect mirror things start to get a bit too noisy though if we perturb our normals in reality few objects are perfectly reflective like this we instead need a way to show gloss values that are imperfect mirrors but what is imperfect mean in this regard picture the surface of the object at a microscopic level as one which is made up of millions of tiny mirrors for our super glossy object the mirrors might all be perfectly aligned with the surface thus we always see a crisp reflection but if the mirrors instead are slightly misaligned reflection at any given point will actually be a sampling of several neighboring light values mixed together back at a macroscopic level we refer to how aligned the micro facet mirrors are as the gloss value note that some artisan workflows will favor the inverted definition of roughness which is just one - gloss recalling our diffuse lighting from earlier we use the mid levels of the cube map to start progressively blurred images these blurred maps will surface again because each level functions is a less than perfect mirror modeling how aligned the micro facets are so rather than sample constant mid level like we did with diffuse we can instead expose a parameter for how glossy the object is I convert this value to roughness and then multiply by a constant representing the number of mipz we want to use in doing this we create a linear relationship between our gloss slider and which mid level we want to sample from many workflows and PBR shaders will define the relationship as some sort of curve but will stick to a line to keep things simple we can combine the diffuse and specular contributions and use our gloss sliders to create some better material definition for our three examples before we finish up let's also make a slider for reflectivity so that we can control how much we see of the reflection to keep things looking somewhat plausible well have the diffuse color get darker as the object becomes more reflective this is a pretty naive way of modeling energy conservation a concept that is essential to physically based rendering ideally we would provide a specular color or derive one using metalness calculations but we'll save that for later so now with just our cube map albedo and normal maps and our gloss and reflectivity sliders we can create a pretty compelling range of materials we can make materials that read as shiny and wet or dry and matte with a nice range of values in between IBL provides a unique way of illuminating our scenes and it's a common staple of more fleshed out PBR shaders if only for the indirect specular values if you want to improve your IBL solution think about how you might store roughness or gloss data as a tertiary map to create surfaces that have a variable amount of roughness you could also provide a specular input and instead of a fixed reflectivity value derive how reflective the object is based on the intensity of the specular value at each fragment and try bringing in a lambertian light source back into the shader code to create a mix of indirect light from IBL and direct light values before you know it you'll be halfway to a pretty compelling PBR shader the shaders used in this video are available on github and linked in the description below I'll also point you towards free PBR and HDR labs si BL archive where you can find some excellent texture sets and cube maps for experimenting with image based lighting special thanks to my wonderful patrons who will always compel me to keep making these videos you the best and as always thank you all for watching keep on making those videogames [Music]

Channel: Makin' Stuff Look Good

Views: 44,176

Keywords: Unity, Unity3D, Game Development, Programming

Id: xWCZiksqCGA

Length: 11min 42sec (702 seconds)

Published: Mon Oct 16 2017

Reddit Comments

[u/Broxxar]

Hey /r/Unity3D! I'm back with a new video! I see shaders have been getting more and more attention on the subreddit lately which is awesome! There are so many tangents to explore when it comes to shaders and visual effects— so here's another one!

This video get's into the use of Cubemaps as a source for indirect lights. Unity's standard shader partially uses this technique out of box with indirect specular and their Reflection Probes. But breaking into the standard shader to see that logic can be quite daunting. Here, we'll write out own simple shader where we can assign the source reflection cubemap directly and see how it can be used to provide both diffuse and specular light to object in our scene.

Ideally we can extend from here into authoring our own mini version of a PBR shader— which is super exciting :D

Let me know what you think! ^{^}

[u/Dargish]

About 3 or 4 years ago I had to implement PBR shaders in GLSL. I wish this video was around back then! While there are great resources out there for this they are spread wide and are often overly complex. Your video did a fantastic job of taking a complex subject and explaining it in easy to understand terms. I especially liked your visual representation of the reflect function.

[u/cerealghost]

I appreciated the shout out to conservation of energy.