Fixing the Ugly Shadow Issues in Unreal Engine 5

those of you using unreal engine 5 have probably run into this problem right here where the shadows on the meshes are just really bad we've got these gnarly black shadows splotches all over our models fortunately we have three very easy solutions to fix this problem now before i go any further this is not a bug and it's the way that nanite works with raytrade shadows so the first solution is the quickest and easiest we're going to select our directional light or whatever light you have that is lighting your scene and in the search details panel we're going to search for ray and you'll see here you probably have cathary traced shadows set to enabled this is likely enabled if you migrated a project from early access or ue4 into 5.0 and we're going to set this to either disabled or used project settings and just like that our shadows have now been fixed i can now rotate my sun and you'll see we no longer have those nasty black shadows now some of you might actually want to be using raytrade shadows because as you'll see right here just a bit of comparison raytrace shadows as you can see in this example right here are vastly superior to the virtual shadow maps that lumen uses by default and again taking a look at this example if you need soft shadows you really should be using ray raytrace shadows they are going to be much better much softer virtual shadow maps work great when they are pretty sharp but for any kind of diffused lighting sometimes it starts to fall apart so if you do need to have raytrace shadows in your lights there are still two other solutions to this problem so we're going to go to the console command menu and we're going to use the console command r.raytracing.shadows enable two-sided geometry and we're going to set this to 0 and i'm going to explain why that is soon by default this is set to 1 and by setting it to 0 this solves our issue this way we can preserve our ray trace shadows and get rid of those nasty splotches but there's a caveat here if you have a bunch of meshes in your scene that are one-sided like this plain example right here you'll see this is a one-sided piece of geo right and you'll see it is not casting shadows anymore if i set the enable two-sided geometry variable back to one you'll see it is now casting shadows correctly the way you would expect i'm going to set this back to zero because that's our solution right now you'll see it stops casting shadows but you'll see if i flip it around then it cast shadows as intended and the reason for that is very simple when you're using nanite raytraced shadows do not trace against the actual nanite geo ray tray shadows trace against what we call the nanite fallback mesh previously known as the nanite proxy mesh and i'm going to show you an example of how the fallback mesh works after the third solution so setting the console variable back to 1 which it is by default now we have the shadows cast by our one-sided geometry correctly the last solution to our problem is to select your nanite mesh in question we're going to open up the static mesh editor and what we're going to do is we're going to go ahead and click the show button and we're going to show the nanite fallback mesh and you'll see it is ever so slightly different you'll see it's a little bit more polygonal if i show this is the nanite mesh and this is the fallback mesh and you'll see because it's not quite the same shape that is what is giving us these nasty shadows the fallback mesh is casting the shadows because the shape is not exactly the same so what we're going to do is we're going to set the fallback relative error in the nanite settings of the static mesh editor and we're going to set this to zero and we're going to hit apply changes now this can take a while this also will have a performance impact on your scene just be careful so i'm just going to fast forward here real quick and now you'll see this is our fallback mesh and does the nanite mesh it's virtually the same and as a result our issues have largely been fixed it's not perfect as i back up you'll see we do have a few issues right around here but it's a whole lot better than it was previously and i think it's very mesh dependent some meshes have been totally fine when i use this solution and some meshes have not so go ahead and try that if you need to have two sided shadows and you want to keep ray traced shadows on this is a decent solution that can get you out of a bind so again if i check this mesh here we have these nasty big black splotches on the mesh i'm going to open up the static mesh editor i'm going to hit the fallback relative error down to 0 and hit apply changes once the changes are done save your model you'll see that these black nasty shadows are more or less gone and behaving as expected so like i said it's not the perfect solution but it can help you out sometimes so to recap the first solution is to disable raytraced shadows on your light the second solution is to set this console variable right here to zero i will put the console variable down below and lastly the third solution is to set the fallback relative error to zero and hit apply in the static mesh editor of your nanite mesh so the reason why this is happening is because of what i said earlier raytraced shadows are traced against the nanite fallback mesh not the nanite mesh itself because it is way less performance intensive to trace against fewer polygons than it is to trace against a very dense model from what i can see the fallback mesh is really just an lod of sorts a lower res model of the nanite mesh that actually seems to have its mesh normals inverted as you can see here just for demonstration purposes this would be the inverted mesh the fallback mesh that has this mesh normal inverted and because it's slightly lower res the polygons don't quite face the same way and as a result it's casting shadows on our nanite mesh so to speak so the fallback mesh seems to have flipped or inverted mesh normals which would explain why disabling the two-sided shadows console variable fixes the shadow issue on our nanite meshes so i hope this sheds a bit of light no pun intended on some of the issues he may be having and again the wonky shadows issue is not a bug that is basically by design that's just how nanite works with ray traced shadows so it's important to be aware of so each one of these solutions has its own pros and cons you're going to have to be the one to choose which one works best for you so i hope you found this video helpful if you did do consider subscribing and hitting the bell so you know when more videos like this are available as always thank you so much for watching and i'll see you next time

Channel: William Faucher

Views: 150,867

Keywords: Unreal Engine 4, Unreal Engine, Cinematics, UE4 4.26, UE5, Realtime, realtime rendering, rendering, CGI, 3D, 3D Artist, UE4, Unreal Engine 5, Nanite, Shadows, Bug, Problem, Tutorial, Raytracing

Id: F3XSKXhIAuU

Length: 7min 12sec (432 seconds)

Published: Wed Apr 13 2022