Shader Fundamentals - Normal Mapping

[Music] hello and welcome to another episode of making stuff look good in video games of all the data contained in the mesh normals could be considered one of the most essential pieces second only to positional data in this episode I want to deep dive on normals and how they're used in 3d rendering so let's begin with the question what is a normal picture things in 2d for a moment take any two points and place them on a Cartesian plane anywhere you position them they will always be collinear that is there exists a line which passes through both points there also exists another line that is perpendicular to the line defined by the two points when we talk about a vector being normalized we refer to a vector that is unit length it has a magnitude of 1 there are exactly two normal vectors for our line you can picture them sitting perpendicular to the line pointing outwards in opposite directions we can use these normals for all sorts of things like physics calculations determining how a ball would bounce off the surface given an incident angle or how quickly the players character slides down a slope let's jump up a dimension and look at three space it's still true that any two points will be collinear but an interesting property arises when we introduce a third point we can't guarantee these points are collinear but we can say that they're coplanar anyway you position these points in 3d there exists a plane that all three points lie on just like our 2d line our 3d plane also has two vectors that are perpendicular to its surface we refer to these vectors as being orthogonal to the plane we can find these orthogonal vectors by applying the cross product on two lines which lie on the plane and normalizing the result and we can get those two vectors by taking the direction from one of our three points towards the other two of course we rarely find ourselves calculating normals will typically import a 3d model and it already has normals or possibly the engine we're working with will provide a method for relating normals this is where we will encounter normal smoothing before we were thinking of normals provided per surface or in the context of a 3d mesh per triangle but measures normals are actually provided per vertex meaning that a single triangle can actually have three different normal values at each vertex and interpolate the values in between why might we do this well for one thing it allows some triangles to share vertices because they don't need to define multiple different normal values at a single point and thus have multiple vertices at the same position but the primary motivation is that we can create the illusion of a surface that is smooth instead of one that is faceted let's take a look at a bit of shader code the vertex shader passes our objects based on normal through to the fragment shader as is the fragment shader renormalize is the vector then we remap the XYZ values which range between negative 1 & 1 into the range 0 & 1 to produce a nicer gamut of colors to visualize our normals on the Left we have a sphere showing normals that are per triangle without any smoothing and on the right the sphere is visually smoother because the normals at each vertex are averaged from the neighboring triangles and interpolated across the surface of each triangle at this point hopefully surface normals are beginning to feel a little less mysterious so let's talk a bit about how they're used the most common use case for normals is lighting to understand how we can use normal to light objects we need to talk about the dot product in simple terms the dot product computes how alike two vectors are to one another the dot product of vectors that are identical is one the dot product of vectors that are orthogonal to one another is zero and the dot product of vectors that are opposite one another is negative one give it a direction of incoming light and a service normal we can ask the question how alike are the normal and the lighting direction in this regard you may have to flip how you think about the directional light source it isn't the direction in which light shines but instead a direction towards a light source from every single point in the world we can use the dot product to compare how alike this surface normal is to the lighting direction if the normal and lighting direction are pointing in the same direction then the point is fully illuminated the vectors are orthogonal to one another then the point is fully unlit typically the negative range is ignored because it's not useful to model this negative light with our objects based normals from earlier we can cut out the remapping into a nicer color range and instead compute the dot product between the normal and the lighting direction in unity the vector of the first directional light source on the scene is supplied as a uniform if we define a shader as part of the forward based light mode alternatively you could set up your own global vector and pass in a custom lighting direction returning this dot product gives us super basic lighting there was a problem though we left our normals in object space while the lighting Direction is defined in world space so rotating the object shows incorrectly calculated lighting we can fix this by transforming our object space normals into world space in the vertex shader now we have basic lighting networks even as objects rotate in the scene ok now it's time to get a bit fancier take a look at this setup the light rotating about is the only variable here the normals themselves are not changing visualizing the normals as we did earlier and viewing the scene with an orthographic camera you'll see a 2d image which might look a bit familiar if you've ever seen a normal map you've seen these colorful blueish textures before and now we have a real understanding of what these textures in code there are surface normals of a 3d object captured into a texture if we sample this texture and compare it to the visualized normals they look virtually identical in creating this texture we packed values from negative 1 to 1 into the range 0 to 1 so in the shade of sampling this map will have to unpack these values and get them into their original range using the normal vector unpacked from our texture in our lighting dot product the light hitting our object appears to have all the 3d normal information but it's actually just a single quad note that in whatever engine you're using you'll want to make sure your normal texture isn't sampled using srgb gama corrected sampling is intended only for textures the user will see and other textures that are packing in arbitrary data should use linear sampling instead so we have our very basic normal mapped quad but there's a problem here just like when we had our normals in object space and rotations caused incorrect lighting so too does our normal mapping code except this time our normals aren't in object space they're in what's called tangent space and just like before we'll need to transform our normal into world space if we want to use it with a world space lighting direction to do that we'll need our normal as well as two other vectors called the tangent and by tangent the three of these vectors work together as a set we can use to transform tangent space coordinates into work space these tangent and by tangent vectors are orthogonal to the normal and each other but there are infinitely many such vectors for which that is true and have them interpolate smoothly over the surface the way our normals do we'll need them to be consistent to do this we use the direction of the texture coordinates to inform the direction of our tangent and by tangent vectors calculating these vectors isn't as simple as calculating normals was and again 3d modeling software or your engine of choice are capable of calculating them in unity only the tangent vector is pre calculated in the by tangent is typically calculated in the vertex shader using a cross product of the normal and tangent vectors so once we have our unpacked normal and our three tangent basis vectors how do we bring everything together let's look at the most basic case a normal map like this the red and green values are 0.5 and the blue value 1.0 when we unpack this we get the vector zero zero one we would expect this value to return the unperturbed surface normal so naturally you might think to multiply the Z component of the unpack normal with the surface normal in this way the normal Maps blue channel is encoding how much of the surface normal we want to use similarly the red channel encodes how much of the tangent we want and the green channel the by tangent so given our unpacked normal and our three basis vectors we can calculate a world space normal by multiplying each vector by its associated color from the normal map if we use the normal value for lighting calculations we get a weight dance our lighting is borked and it seems to be flipped on the x axis what's happened here is that when we captured our normal map we did so with Zed facing toward us and Y facing up in unity this actually results in our X direction pointing left meanwhile our tangent values which again encode a direction based on the UV coordinates are pointing right we solve this by inverting the red channel on the normal texture for unity this is actually a correct normal map it should appear as though the map is lit by a green light at the top and a red light at the right hand side use this to quickly check if the directions in your normal maps are correct now we have a simple shader that takes in a texture and change the object with basic lighting informed by that normal map we've also developed an understanding of what these maps mean and how they can be created from 3d geometry there are several other ways of achieving normal map lighting worth mentioning one popular method is to transform the lighting direction into tangent space in the vertex shader to avoid several multiplications in the fragment shader I'll include an example of this along with other shaders used in this episodes github repo and that does it for this episode but it won't be the last to hear of normals I hope to tackle other shader fundamentals soon so future case studies can focus on the really cool stuff and not get too sidetracked with tangential explanations of shader concepts a special thanks goes out to my awesome patrons who can continue to support the channel even though releases have gotten a bit sporadic and as always thank you all for watching keep on making those video games [Music] you [Music]

Channel: Makin' Stuff Look Good

Views: 130,562

Keywords: Unity, Unity3D, Game Development, Programming

Id: 6_-NNKc4lrk

Length: 10min 12sec (612 seconds)

Published: Mon Apr 03 2017

Reddit Comments

[u/Broxxar]

Hey r/Unity3D!

Been a while since my last video, between GDC, work, and other side projects it's a been a bit hectic! But I'm pretty happy with how this one came together. It's about normals and normal maps. Hopefully I did an okay job at explaining them. Admittedly I think I left out some details that would probably come up if you started using this code as is in your own projects... but hey, we'll call those learning opportunities!

[u/Osteelio]

I always get excited when you release a new video even though I only understand about 25% of it. I'm looking forward to more videos on the fundamentals.

Also, I love your simple explanation/visuals for explaining important concepts like Dot Products, etc.

[u/iemfi]

I think you should focus on cool effects or more complicated topics instead of the fundamentals. There are plenty of other sources to learn the fundamentals out there. Your channel is one of the rare sources with more advanced cool stuff.

[u/SauronWasRight]

Great video! Keep 'em coming!

[u/SteroidSandwich]

I just saw this today. I love this guys videos. I have used the ideas and theories he's shown to make some great shaders.

[u/kittysparkles]

I really enjoy your videos. Would you be interested in making a video that covers Unity 5.6 baked lighting?