Unreal Engine 5.3 VR with Lumen : Optimizing and Troubleshooting

hi let's talk about Unreal Engine 5 Lumen nanite and VR in this video I'll guide you through my thought process my uh troubleshooting process and my kind of optimization steps of how to get Lumen to work in your in real engine scene with VR because technically uh Lumen is not supported um in VR applications so there's a lot of kind of not hacky things but a lot of optimization and kind of things to consider things that you need to consider uh as as you're going this route okay so let's let's jump right into it first of all let me introduce you to the scene uh we have a neckan capsule Tower the interior of it we have a few trees here and we have um like a weird sculpture that can move and all of these will be used to test out different things and to Showcase different problems oh yeah and we are surrounded it with nanite meshes here as well okay so the first thing that uh I will do is I of course Play Let me just showcase it to you right and once it loads in there we go now we're at uh somewhere around 45 frames per second and if I were to press button one I'll show you how to do that later uh we're dropping down to um scalability settings zero which is low uh quality and now we're at 90 frames per second notice how it caps out at 90 frames per second with 11 milliseconds for the frame draw um this is this is an important thing that will come back to in in just a second it does not go beyond that if I press uh number two uh then oh by the way the file I will make it available for everyone to download um so it's going to be in the video description below um so you can run it uh for for yourselves as well so if I press number two uh we're still at 90 frames per second our game Draw is 11 milliseconds our GPU time is 9 milliseconds that that means that the GPU uh time and the game time since they have a gap between them we we can go even further I mean we're not uh C in out our GPU if I now press uh was it three now we are at high high settings right and yeah this seems this seems still to be pretty smooth but now uh we're at 22 milliseconds and you can see that our frame rates are for some reason capping at 45 frames per second well they're they're going around 45 frames uh per second they never go beyond and this is an important notion of how uh frame repr projection works the moment um my headset currently is running at 90 frames per second by the way right so anything that I see here is still 90 frames per second so the moment the game cannot reach 90 frames per second it Hales it immediately and it then reproject the same frame twice right so it it initially doubles the frames and there's a lot of more Shenanigans going on with the frames that are being reprojected so it's not just kind of repeating the same frame um but that that's the basic idea of it 45 frames for me gets doubled into 90 frames and even if this game uh could be currently running at 60 frames per second um I would still be getting 45 because the reprojection kicks in right if I were to press uh was it three no it's four yeah now it's it's really checking because we're now at Epic settings right okay so now let's talk about where the 90 frames per second comes from right so actually in the Oculus app for my meta Quest 2 and of course different headsets different settings but it's driven by the settings of the headset clearly right so we go um if you're also using meta quest uh you simply go to um your paired device scroll down here and where it says the refresh rate is set uh 72 htz is recommended I have just set it to 90 HZ right so let me actually drop it down because you can see that when I have it at 90 HZ the render resolution is being dropped right so so I'm sacrificing frame rate for the resolution uh which I don't don't like that tradeoff so instead I will go for 72 Herz which is recommended and do the recommended resolution you know jumps up so I'll save and restart okay and now when I play this at 72 uh frames per second let play seems to be okay uh and I used the Tilda key which is below the Escape key on your keyboard click that opens up a command line and I write stat FPS stat FPS hit enter now we're at 35 frames per second with 27 milliseconds um calculation time or or frame time right right uh which gets of course multiplied to or 36 actually frames per second which gets multiplied to 72 for me so again when I'm looking at this this is very smooth you know uh definitely a smooth uh experience so then if I press the Tia key again and I write stat unit now I will see my all of my uh draws right or sorry not draws but the GPU timing GPU time 23 22 milliseconds and my game timing 27 which technically the the way I read it is that I still have 23 to 27 around 4 milliseconds of computing time uh before this starts uh going below the 35 frames per second Mark that kind of um is extrapolate it to 90 uh to 72 frames per second so let me escape out of that and let's see what kind of settings we can change to increase this so all of my Lumen uh settings are controlled via uh VI the post processing volume as as it should B and if I just go to Lumen here I think I just had it as set to default where is it Global elimination Lumen Lumen Global elimination okay so it's almost default the scene view distance is set to 10 m and Max Trace distance is set to 10 m besides that everything else is um defaults so since my scene is a little bit bigger I think I should instead of 10 m here I should use like 50 m I think that that's going to what's the default oh 200 M okay so definitely less than that 50 m and then let's let's just simply increase the final gather quality and the scene lighting quality these two guys right here so for now I will the final gather quality is always more important than the seene lightning quality at least for the perception of the quality so I'll first increase this to two and see my timings what what what kind of timings do I get okay there we go so now we're still at around 26 24 okay so we we lost 1 millisecond of the GPU time by the way if you want to see the actual timings you can you can write stat GPU command and then you get all of the AC ual timings and now you can see our Lumen screen probe gather is actually getting the you know is eating the most uh from from the whole scene this is going to change once we move outside stat GPU we will we will go back to this okay so technically in this room we still have just a little bit of of of freedom to to increase the uh the the the screen uh the scene lighting quality so I'll amp it up to two H we could even go even further but at this point I would kind of stop and I would not increase the um the the quality any further because this seems to be running a okay and instead I I would start investigating what would happen uh if we increase the pixel density so another command that uh you should use if for instance let's say you're uh you're running at one second you know you're running at 6 frames per second that's a nogo so how do you how do you actually make it work well you can always uh write WR R do pixel density 0.5 so your resolution your pixel density resolution is dropped by half and in this case this didn't work at all so let me drop the quality lower but now when when I have the quality set too high with the much lower um pixel density you can see that now our frame rates are for some reason at a 60s range right with our GPU times being closer to 12 milliseconds so if I drop this down even further this is going to look ugly of course but our pixel density 0.2 I mean now now it's just a mess but now we're our GPU times at 8 milliseconds so this is uh very close to running at 90 frames per second right or or 72 in in this case 72 frames per second and if I just kind of walk around here yeah this looks like garbage absolutely because we're only drawing 20% of the pixels right but now with this you can start optimizing and you can start finding the pixel density that kind of works for you so for instance our pixel density of 0.8 is honestly not that bad and it gives us a very a very stable experience even when we are looking at you know the the volage here so that that's definitely u a benefit um I mean right now we are still dropping below uh 30 30 frames per second which is fine uh anything anything that is above 25 frames per second I still perceive that to be quite smooth so so it's okay so technically I could go R pixel density one yeah and now we're definitely checking now now it's right on the edge on the cusp of it not being useful usable anymore with that being said the the preview is is quite uh quite smooth it's just the the hitches it's starting to to Hitch right so that that's the combination of lumen uh quality increase as well as the pixel density you know staying with a high pixel density so maybe I will say the final gather closer to 1.5 seene lighting 1.5 something like that let's not push our luck okay so before I was showing you how you can change the qual the quality of your by the way I don't remember if I mentioned this sorry I don't remember if I mentioned this you can download this scene to check out it's it's going to be in the link in the video description right so so you can download this uh for free okay so I I was showing you how you can do this this okay Switching through different um qualities quality presets right so let me just show you the let me just show you the commands that that are making this happen right so if I go here to my level blueprint so that's the blueprint of the level itself I have uh a few um I have a few events here so uh these are simple Escape quit game begin play set the skill ability to two scalability 2 is basically um High uh High preset right and then if you press one on your keyboard scill ability is set to low uh if you press two skill ability is set to medium three high four epic and just just as a general General note let's let's try doing it this way so if you press um one you can see like the quality is really bad I mean we're at 72 frames per second uh in a very stable way but the quality is real bad there's No Light No No Shadows no nothing all right the jump between one and two is huge because you have ESS suddenly you have light you have shadows you have proper textures and so on the jump between two and three is also huge indirect illumination Reflections all of that right but then the jump between three and four is my Min so that's why we use uh skillability two uh most most of the time because that's the tradeoff for scalability 3 for the Epic scalability setting is just not worth it okay so that that's taken care of and basically that that's that's the setup a very simple setup okay let's talk about other settings so Lumen is pretty straightforward right you increase the quality the drops you find the balance easy as that a little bit more nuanced and tricky are the more experimental or VR settings that you can turn on or off so if I go to settings project settings here and under engine I'll just find rendering and under rendering I'll find uh VR by the way uh these are the settings that I use global elimation Lumen Reflections Lumen 128 Hardware retracing when available surface cach detail tracing virtual Shadow Maps seems to work just fine uh I'm not using retraced Shadows because if I use rrac Shadows um then some geometry just messes up like that and then I need to use a lot of different kind of hacks and then and things to to make it um to make it work like what was it normal bias set to three and blah blah blah and I just can't be bothered with that so I have that turned off when we're just using Lumen uh or or not Lumen but uh virtual Shadow maps that that seems to be okay anyway uh not not about that um about VR VR VR VR there we go our VR settings so these are the settings that ideally I would use especially these top three of course uh we have stereo fation level medium Dynamic fation tick marked and then we have instant stereo ticked there is a problem with this so let me explain what these do so instant stereo uh reuses pixels from one eye onto the next right or from Left Eye reuses the pixels to the right eye um thus reducing the cost of of running uh to two screens you know for for both eyes so technically this should increase your um frame rates dramatically and dynamic foration uh as well as uh stereo uh sorry stereo foration level as well as Dynamic foration uh draws lesser density view so less pixels are drawn in your peripheral vision and more pixels are drawn where you're looking at in this case uh since my headset cannot track my eyes I think Dynamic foration does absolutely nothing so I can have this unticked I think just D based on gpus level range no I'm stupid I'm sorry I forgot this is not that this is basically if your GPU is not used as much then it's going to draw all the pixels and but if it's being hammered then it's going to you know draw less and less pixels in your peripheral vision and this is how aggressive it is I suggest uh using it as medium because that seems to work um the best I mean it's most balanced uh instance Theo let's talk about that this is important so I ideally this works and this works well if I press play now actually let's let's move out let's move out of here and just let let let's start the game outside we're good we're good okay so stat FPS I'll I'll just show the FPS we're at you know again 36 frames per second we're we're good to go seems to be going quite well yeah every everything's pretty stable but then um this structure that I have here it's actually a dynamic structure meaning that if I PE trigger it it starts transforming right it's uh I I I did a tutorial on my main channel uh about the structure it's it's pretty neat we will play with it uh at the end of this video but it's transforming for you it might seem that yeah this seems to be okay this you know my head is shaking I'm trying to keep it as stable as possible but this seems to be okay until I run uh VR do uh spectator screen mode set to two so now you can see what each of my eyes see and then if I trigger this again do you see a difference um I can I can go closer the left eye is fine the right eye there is an insane amount of shimmering and like the especially in the reflections uh you you can see in the on the poles insane amount of shimmering and then this kind of jaggedness that that's going on and that is because it's it's technically reprojecting the pixels from from uh the part of the screen from uh left eye to right eye but it's not doing a great job with uh moving objects right with uh moving geometries so as long as nothing moves instant stereo works just fine but the moment Things Are moving it's it just breaks down so um I had to unfortunately have instance stereo uh turned off uh VR like that I will need to restart so give me a second okay after restarting if I press play of course I need to look around uh we use the VR dot spectator screen mode 2 and I just click yep both eyes show exactly like everything's clear everything works just fine and I can even you know if if I come closer to the columns you can clearly see you know that the columns are not sparkly in on on both both eyes it's pretty fun uh fun structure okay so that's that's done uh with that with that being done let's talk about materials now um with un real engine 5.1 or 5.2 or maybe 5.3 not 5.3 5.1 I think um substrate materials have been reled released uh and it's an experimental feature that can be um enabled through project settings if you just write substrate substrate materials experimental uh you need to tick mark this is experimental be aware that any material set when substrate is enabled won't be rendered correctly if subrate disabler uh is disabled later on right so I hit okay asks me to restart I will restart now okay I have restarted with uh substrate now working so all of the materials are running through the substrate pipeline which is going to be the main pipeline later um down the line for Unreal Engine and let me just press play if I stat GPS FPS not GPS stat FPS you know uh not not not that great a little bit slower uh that's the first thing second thing is looking at uh how do I show you well all of my uh transparent materials are now gone uh that that that that's one thing there we go the glass is gone but that that that's fine the most pressing issue is if I I do VR spectator screen mode 2 see how useful that is you know to to understand what's going on let me show you let's say my my arm both arms that feels to me like a little bit of a problem doesn't it um I can't really have it on when when this is going because it's really messing with my uh right eye but let let me just say this at the time of recording this video 2023 uh September 30th uh substrate materials should not be used with VR absolutely not okay we're back to the normal material settings or material workflow um pipeline so now let's talk about I guess the last thing that I want to mention in terms of optimizing your scene for for Lum and VR and that is the anti-aliasing right so we're not doing it for mobile so we don't care but just in the pro project settings I searched for anti anti-aliasing and the method here is said to be temporal super resolution that gives you the best uh smoothing but in turn you you get this kind of a ghosting effect that some of you might not like so there are many different anti- lasing methods there U that that you can choose from in a real engine and we're just going to go through them the pro problematic thing is oh now I messed it up but the problematic thing is that uh if we just change them here and we look at the preview that does not really give us a good understanding of how this is going to look like when we put on the headset so it's much better to uh change it with the with the headset uh through the command line so that's what we're going to be doing um actually before we we do that let me just explain to you what the numbers in the command line will mean so here an aliasing method if I expand this we have nonone fast temporal multisample multi sampling and temporal super resolution TSR right these in the command line will be regarded too as 0o 1 2 3 and four so we will be able to shift through uh five different options from zero until four right so if I play now and I think we need to find is this a good yeah I think this is a good good good place to to test it right let let's go a bit closer there we go and I I'll keep my head down so that you can see better um if I write in in the command line if I write r dot uh anti or is it not R just anti uh aliasing yeah r. anti-aliasing method like that and I write zero then there's no anti Leasing and actually let me draw FPS uh show FPS H did I Mis mistype oh stat no true stat FPS um and Stat unit just to show you the cost of different Al anti-aliasing methods So currently we're uh our GPU times at are at around 29 28 milliseconds right with no anti- leasing whatsoever if I change anti eling method to one rgpu * are initially the same maybe even lower which is weird that's wild and now we're getting a little bit of anti alist image right still not not the best also um on the left hand side you can see the foliation level kicking in as well you see that that Grain on the left hand side of the screen that that's the fation level that's the decrease in the resolution if I change the method to two now everything is much smoother and our GPU times are actually quite decent uh they're still at around 28 which is I I I honestly don't understand why uh three by the way it's not like we're increasing the quality of anti-aliasing we're just shifting through different versions different ways of how you can anti-alias an image and for this one clearly is the most expensive and this is the one that that we're using right so that's the TSR the temporal super resolution so what I wanted to Showcase here is that every other anti-aliasing method except for TSR for temporal super resolution is going to be um cheap TSR is expensive in this case but damn it looks better so let me switch back to uh I think two was pretty good yeah with two we are always in the green in terms the of the frame rates and there are absolutely no hitches here whatsoever let's play around with a few few things here so if I grab let me grab the ball and let's go into this structure here so it's it's pretty fun in in VR because you can just yep you can just place a ball right there press play and just see the you know did this whole thing being kind of inter interactive just see how it how it behaves and how just in general how the whole structure works also we have um there there are like a bunch of tutorials on my main channel of how to do this but we have ourselves a light control system so you can find a pretty nice light angle for this whole structure to you know to be showcased better who scary so that this is a pretty pretty gnarly thing currently my students are are doing um structures like these you know circular motion into linear motion and trying to experiment with what kind of architecture can be can be achieved from that the battery is low oh a ball for me thank you oops sorry about that but yeah so that that's the way that's the way it goes okay so we are we are done with the Showcase of the settings and the optimization um name of the game is for all of for those those of you who are saying that the game needs to run at uh at least uh 72 frames per second natively I say no it doesn't maybe a game a competitive game for sure it it it does but an experience can definitely run at 36 frames per second with you know it being smoothed out to 72 it's absolutely usable I never had any motion sickness um issues I never had anyone have motion sickness issues um motion sickness starts at around 20 frames per second or it starts when you have a lot of hitching so when your frame holds uh at at certain um places uh then the motion sickness happens but if everything is kind of running smoothly and just gets reprojected it's it's all good it's all fine so I hope some of you found this useful and we'll experiment with your own um of your own own scenes by the way uh just just to kind of repeat the all of the things that I said throughout the video the the the computer that I use uh is running RTX 3080 10 GB vram version so a low lower and 3080 graphics card with 48 GB of RAM which are not being utilized and an i7 uh 10th generation processor so not the most extravagant machine whatsoever right if this was running 4090 we would be at around maybe 72 frames per second natively in this scene probably not but maybe a Boy Can Dream can't he okay I'll see you in the next one later

Channel: Geddan

Views: 25,110

Keywords: archviz, render, engine, felix, vray, lumion, d5, free, real time, render farm, server, online, rendering, obsbot, tiny, crt, mecha, model, 3dprint, build, 3d, print, ue5, ue5.2, ue53, ue5.3, 5.3, vr, virtual reality, xr, meta, quest, lumen, optimise, optimize, troubleshoot, problem, nanite, raytrace, raytracing, virtual, reality

Id: lKFviaw7Zi0

Length: 36min 12sec (2172 seconds)

Published: Sat Sep 30 2023