Virtual Textures Are Now My Favorite Thing

so it's just sinking into the ground just like that it's just magically disappearing hi guys my name is barodante and welcome back to the game so yeah continue working on mostly optimizing my horror game and while I was researching how to optimize things I stumbled upon this thing called virtual textures and I've heard about them before but only about the one side of the question and recently I learned another side so what are virtual textures basically virtual textures the way you use them all the time in Unreal Engine is you right click any texture like normal texture and you convert to a virtual texture right now it is already VT the virtual texture so it can only go back to regular so what it means is Unreal Engine will only stream the resolution of this texture that is necessary for the texture to appear sharp right now considering the distance and the size of the texture on the screen at the moment so you're never like streaming 4K 8K textures from all the objects if they're far away there will be like super small resolution and is doing it in real time for you you never even have to think about it so many call virtual textures like free performance but I've been using that since the beginning didn't help me much so far but another really cool thing another cool side about virtual textures is like these two textures right here they're called runtime virtual textures and of these types of textures let you completely remove the load from your computer when you're using very complex shaders and one thing where you use complex shaders is the landscape because right now it's like a super simple landscape I'm having so far I wasn't doing a lot of like blending of layers or anything but already I'm using so-called texture bombing so you can see right here each texture like it's clearly repeating in like a grid but it's not exactly exactly perfect it has certain variations so if you're a character and you're standing on the ground it will look a lot more natural because the texture is constantly rotated at different angles with each repetition of it and also um it's not actually cut in a grid it's cut in like this and gone shape that's also having like this noise applies to its edges which will be nearly impossible to find and show you guys right now maybe the only way to do it would be to increase the resolution of the virtual texture and this way the seam will be showing up a lot sharper than the textures themselves there it is so this you can see how it's like a very Wiggly noisy kind of line that's the split between the two textures you can see it now but if I lower the resolution of the virtual texture down to the actual resolution of the texture the photo those themselves that are used for the material then it's like almost like it's impossible to see it at all but yeah that's one thing I would love it if it would be possible to completely remove all the repetition by using a tiny texture that keeps repeating everywhere but I guess that's impossible because it is after all the same image you're repeating it doesn't matter how much you rotate and scale it around it's still the same image so it looks kind of repeating but definitely a lot more natural another side about like this runtime virtual texture right now if I quickly turn my screen you may have noticed there was like a lower resolution version of the texture there it is like it popped in a little bit so that's what it's doing like it's streaming little blocks of the texture all over the place instead of having the whole object you know with these sharp textures showing up on them all the time so that's removing a lot of the load from the computer when you're playing the game and you have this giant landscape giant object covered with like probably billions of pixels you know it's a huge amount so yeah it takes a lot of space on the the hard drive because the thing that's happening instead of having just one you know small rectangular texture and then just a command to repeat it forever and then you know your GPU and CPU are doing all the work to make that happen in this case it's sort of grabbing all of the commands inside of the material which is uh right here right so this texture bombing is inside of this function so it's doing this and then it's applying this uh tiling this texture coordinates two of these textures and then putting them together like applying certain things here and there and then it would create a frame with these Textures in this case with virtual texture all of this stuff is done and then it's put not into the material but into the virtual texture and it's literally this one so if we look at my landscape it is this long rectangle that is like brown right so the virtual texture looks exactly the same way and this is another virtual texture that's doing also the same landscape but it's just baking the height information about where the landscape is in the world so those two things are baked and then the same texture this brown texture that we bake into here it's used to actually apply itself in the material so what happens when you're playing the game all of this is happening once and then if it doesn't need to be updated and it doesn't because it's a landscape is just standing still it will be just using this pre-baked texture and just picking it up with those little bits it kind of looks like Google Maps when you move the camera around in there and zoom in and zoom out it's loading in smaller bits like that's also a virtual texture it works the same way so it's all pre-baked and none of these commands none of this texture bombing or anything none of this is being done each frame or something like that it's already like a dead baked texture or this huge enormous size texture but no CPU or GPU has to work on this it's just used like that so it's kind of like baking a gigantic texture for the whole landscape instead of UV tiling even making things a lot faster and it's broken down into small tiles so in here you have settings for the resolution of these tiles and everything how exactly things are broken down so that's really cool and that's something I applied uh recently to try to improve my performance of the game but overall let me quickly show you guys Ctrl shift coma this is my GPU probe so this line is showing which part of the rendering or doing anything in one frame that was rendered right now from the camera how much time each part of the game is taking each frame so the giant piece right here is nanite so I just need to keep working trying to improve nanites to make things faster because this is an enormous chunk of performance here so I don't know either I have to still work with a lot smaller amount of objects and not just throw things around like this so apparently the whole thing about unlocking polygons with nanite doesn't really look like unlocking to me if I really need to use the last object to make it work faster so I'm using fallback objects for nanite and everything if you guys know what that is it improved things a little bit but not much another thing that may have improved a little bit is that I rematched all the objects as much as I could not so much with the trees but with the trees I removed some of the polygons from the inside but I baked everything so there is no overlapping polygons like these leaves right here they're not separate polygons that overlap each other they're actually like voxel rematched so there is no polygons inside of polygons so to speak so it's kind of really hard to do and with the trees is kind of impossible first of all they're Giant and no voxel rematch will handle that as well as Boolean and another thing is that these trees have flat leaves so voxels just won't pick that up at all so yeah I don't know what to do about that but I'll keep experimenting with the assets I think that's the best bet I have so far and of course the fallback meshes are important to accelerate the lighting a little bit but all of that is very tiny Improvement so far I don't know uh so yeah giant nanite a giant like diffuse indirect and ambient occlusion so that's Lumen and in here like base pass again Lumen once again in here yeah obviously the main problem here is just nanite so I'll see what I can do about that but yeah on the side of textures of materials I did my best to you know make the Shader complexity as small as possible and you can see that the stones are kind of Darker shade of green like still pretty good I guess especially considering that stones are not that frequent in here but yeah they're darker green because of another really cool thing about virtual texture specifically the runtime virtual texture of the landscape is that what you can do with that is this so you guys can see the stone is put into the ground and then there's this extra border that's perfectly continuing you see it's the actual stone but it's perfectly continuing the texture of the landscape a little bit with like a noise onto the stone so it would blend in much better like it's in this uh in this mod in this like forest floor is kind of covering it up a little bit so that's the thing I built just finished building it today so it works really well like all of that is procedurally generated so I can't move it around like one by one I mean but if I just grab an asset let's say this one and I put it right here oh I forgot how hard it is to put s's on the landscape manually like procedural generation does all the rotations for you but yeah I put it like this and you see you can't even see where the edge of this object is it's blending in so well with the surface of the landscape so it's like that quick live kind of effect so it's actually really cool and once I understood the way the virtual textures work I knew exactly what to do to make this effect happen this is the whole code for blending things together so what I do is I pick up this virtual texture the the brown one since it's storing the whole landscape like the whole thing so it knows exactly where each pixel of this gigantic texture is in the world without having to tile and align those tiles in any way you can simply use this very texture and blend it on top of the stone so that's what I did and not just the texture like it's called virtual texture but it's storing like almost the whole material it has a bunch of channels in here the only thing that's missing is like metallic I guess from the basic PBR but yeah this is uh the material of the stone here and this is the material from this virtual texture and they're Blended together using this mask and this mask is uh mostly done using this other virtual texture and that's about the vertical positioning of the landscape and we subtract that from the position of the pixel of the Shader of the actual stone and based on that difference then we apply all kinds of stuff including the noises and everything to create like this kind of mask like just this distance plus the noise but yeah it's literally like somewhere over here the landscape is and then it's rised a little bit above it and with a gradient and applied certain noise to it so simple as that you're just blending things together like that so that magical thing about having the whole texture like the end result of tiling the whole thing and processing all the effects you want to do with the textures on the landscape they're baked into the swanji and thing so you can easily stream it on any object that's next to it so it's just sinking into the ground just like that it's just magically disappearing or one thing about it the problem about it is that it is just projecting from the top and that's all it pretty much can do and because of that if it's very vertical it will be stretching the texture a lot so one thing you do is you sort of mask which is something I also did in the in that masking in my material is you mask away very vertical angles of the surfaces so that's why when the stone is like this the distance of the blending is so tiny is just a little bit to kind of hide the actual completely sharp edge of the polygon but overall it won't propagate all over and if it's a lot more flat it actually goes a lot further because it's kind of uh like the dust filtering where you're using just the vertical axis of the normal and that way it's uh kind of like dusted with the texture like right here this is a very flat part and it has a very strong texture presence from the landscape while the thing that's vertical right next to it is completely excluded from that and you can see even like how the normal map of the texture of the stone is kind of going back and forth between the landscape and the stone so yeah this is a pretty cool thing not too expensive and lets you really terraform with just static meshes which is something that will be very useful for me with the whole procedural generation of the environment because the way epic games does it and as I mentioned in the previous video like you can't really use procedural generation like this whole thing that you can quickly update and you know just use things like this like some kind of guides to completely change the environment there it is uh following the new shape it's it will create a different thing with a bunch of rules applied to them you can do whatever you want more on that in the previous video about the forest and you can add a lot more rules and I will for sure right now I'm just making sure things are organized better and mostly I'm working on performance but one thing you can't do with this is uh changing the shape of the landscape like create a trench of sorts or build a mountain smoothly like sculpt the actual landscape you can't do that you have to do it manually but what epic games do with procedurally generated environments is that they place like giant rocks there's like assets in quixel mega scans library that are really high scale rocks like actual bits of mountains and stuff like that and you can just place that procedurally and just blend the materials like this and there you go that's your you know changing of the shape of the landscape just like that no problem all you have to do is just have some kind of naturally just quickly sculpted somewhat flat somewhat uneven base surface of the landscape and then you can do whatever you want with it with actual huge realistic assets like that so that's pretty cool and that's why I think virtual textures will be a big help with a procedurally generated content another reason why I did this was because um well right now the landscape is looking kind of ridiculous absolutely the same texture copy it everywhere and that's it but it won't stay that way if I actually wanna build stuff more complicated like a lot of the times one thing you do is you mask the normal direction of the landscape and on the flat sides there will be some foliage texture like some kind of grass or something and when it's more vertical there would be like a bold rock or just a slope kind of textures and those can be defined automatically with like material layers so that involves this giant masking of the whole thing based on that on those angles and everything and that's already a pretty complex thing especially considering there's many materials that are Blended together into one there can be like 10 or more depending on how big and varying the environment is what kind of textures you want to have blending around and all of that is using a lot of these layers so that's how you get a pretty high Shader complexity like this stuff won't be green anymore it will become very red if you blend a lot of these materials together on the landscape and Landscape is everywhere so it's constantly present on the screen and it's very hard to render it but with the virtual texture no matter how complex things are they're all then baked into this runtime virtual texture and it will say literally the same color as far as I'm expecting it to be at least because there is nothing to be more complex if it's just one texture in the end so that's really cool and that's like at least things won't get worse from what they are right now at least the textures on the landscape are kind of solved at the moment so that's good to know now another really cool thing and kind of a huge achievement on my side is this right here as you guys can see I can now finally package the project which is something I couldn't do before so I looked it up the problem was and the fact like first of all a couple of weeks ago I realized I needed to install actual Visual Studio Microsoft Visual Studio this guy right here to be able to compile the game at all and package it into an actual release right here but also on top of that you have to make sure you install proper parts of Visual Studio the one that's called like game development kind of stuff and it says the word unreal right in it so that's something not to miss is for sure because without it it just won't tell you what's wrong just some kind of unknown error so this already I tried it on Nadia's laptop it works completely like there's no Unreal Engine in there unreal editor so it's working just fine and yeah John doesn't have hair that's because in the final build I removed his hair when I saw in the GPU probe how much time all these hair curves are taking so I'm definitely using hair cards I actually thought I was using her cards I was using the lod3 version of John I am using it right now and I thought it was having that type of like a simple flat cards hair but no it was just very chunky but still the curvy things so yeah right now it will still stutter terribly but overall as I'm running or let's say I won't be even running around okay let's just start the level and yeah the background music that's something I that this is something I composed myself a while ago I don't even remember where it's from maybe I did it on the iPad maybe on on the desktop I don't remember but right now I'll just stand still and we see we have 33 FPS this is full QHD resolution no upscaling so 33 and if we go 50 upscaling that will be 59 I'm seeing 60 sometimes it's actually a bit above 60 usually when I'm not recording I guess so 33 and 60 33 and 59 and let's launch just the previous version that I packaged also no hair but right now well okay it's kind of kind of the same actually with a lot of my testing is actually much better like usually 10 FPS difference I don't know how things are like of course still this is all very low difference like in here we're like 54 55 FPS instead of 5960 uh but yeah and here is like 30 32 and yeah a lot of these stutters like dipping performance where things just won't work at all for like half a second uh that's something that became a lot less occurring in the latest version but yeah it's still all happening and I don't know definitely nanite is like one thing I need to keep uh researching so yeah right now this is full qhz resolution there will be stutters but it's generally running smoother another thing I wanted to show you guys is um I have a bunch of hotkeys in here so first of all um right now we switched to temporal super resolution that awesome upscaling algorithm that's uh Native in Unreal Engine 5. so everything is super sharp and very crisp and awesome especially like if I go down to 50 resolution like this is very sharp it's not looking like 50 resolution at all and right now I hit n and this is dlss2 so Nvidia upscaling and it's quite different actually so uh they have different strong and weak sides Nvidia is not looking great when we're standing still so everything is like swimming around all the samples everything is just never standing still and overall things are much blurrier comparing to it's important super resolution like all the leaves are much thinner and we can see much crisper details on them while Nvidia is much more like smoother look and yeah temporal super resolution is a lot more static it's a lot more stable so nothing is supposed to be moving and almost nothing is moving a little bit of sampling here and there when there is very tiny details but with Nvidia everything is constantly moving it's like all the bushes are covered in tiny bugs that are constantly moving around so yeah Nvidia is not that great on keeping things in place but when you move around things are a lot more consistent like no ghosting or any kind of noisy details overall everything is just looking like normal 3D Graphics so that's great for movement and if we switch to temporal super resolution and start moving around you can see all those super tiny crisp details that are looking so good they kind of stick around when you move the uh the details especially when like rotating or moving forward so everything is kind of like covered in this fuzz or something like that like a static noise a little bit and again Nvidia is just a lot cleaner movement which is I think much more important for a game although it's a horror game with puzzles maybe a lot of static stuff will also be included maybe there will be actually more static situations but what I was thinking about since I can so easily like switch between different samplings maybe it makes sense to like program it to like react to the amount of the movement of the camera and space if you're moving around like immediately switch to like dlss and when you're like standing still just switch back to TSR the only thing is I don't think you can fade it in very easily it will always have like this little twitch like it's looking very different that's why it's uh not that easy to do and yeah another thing I have right here set up as a hotkey is actually a cool maybe even machine learning power sharpening so it's not really upscaling but overall since things are a bit uh like soft with dlss I'm also using a barrel effect on the camera that's like a post process Shader so everything is kind of like a bulged a little bit like this I might drop this idea of using this effect later but like I should I I think because things are very smooth like soft just because of that even at 100 resolution but anyway the point is the sharpening really comes in handy if we look at the leaves on the ground right now the ones that are in Focus like this is becoming much clearer much sharper and this is a much softer look and with that temporal super resolution if we turn on the sharpening it's like super clear look oh my God you see how like is complete mess of a noise when you move around things just stick around some kind of white noise like all the sky that was showing through the branches they just keep showing around like sticking to the edges of the details when they're already not relevant but yeah another cool thing I have here is this button so you can at any point turn on the sun meaning you turn on the sun shadows and remove the clouds and that's how things swish so yeah and things run pretty fast this way as well like I'm generally at 60 FPS in the sunlight when I'm not recording at least so that's pretty cool so I've been running around in here shooting enemies and everything with this very questionable quality of the image because it's upscaling and all you can run around at like stable 30 FPS at almost full resolution at like 75 percent upscaled in the sunlight with even a flashlight we're still above 30 right now with occasional dips like this now that's something that starts happening but yeah I've been shooting enemies around the only problem is they won't walk towards me anymore they never will like they both move right now as if they noticed me but they won't be able to come close and even like if I'm nearby they start attacking so they're like they fully work but they can't move and that's because the nav mesh doesn't work when I package the game if I open this same build of the game in the editor they will totally walk around so the navmash for some reason doesn't work maybe because I'm using the world partitioning now so yeah that's the way things are right now uh I guess the only solid progress I had so far is how I can blend the stones with the ground that's really cool looking like I don't know I'm just loving this appearance and the fact that it's um not that expensive to do and easy to set up as well so cool so yeah this is it uh I wish I had better news about performance today but alas no such luck mostly like I I'll still have to keep digging into how to make nanite work better I I really like I still have hope that it's about like making assets more optimized for nanite like the best thing to do like the best asset for nanite is the one that has just one shape one surface that is like it can be as complex as you want but it should be one shape with no overlapping polygons hanging polygons maybe even open loose Parts all of that is like it creates extra work for nanites each frame each asset so yeah I'll see about that and maybe I'll do more as I said some of the assets I managed to do that but with the trees it's really hard to rematch that kind of stuff so I don't know so yeah for now this is it let me know what you guys think optimizing the game is a hell of a thing but I'm sure as hell not dropping this endeavor because I need my game to run at solid 60 on Nadia's laptop that has 3070 TI mine is running 3080 ti so I'll make sure it's 60 FPS on 37 ETI a laptop that's my goal for now at least but yeah let me know what you guys think and I'll see you all in the next video bye oh boy oh boy a lot of stuff to learn here funny thing how nanite works for me in practice originally is like unlocking Paul accounts you can do whatever you want but at the same time like not really at all literally if I would remove trees things would work a lot faster isn't that kind of poly count related question mark

Channel: BoroCG

Views: 47,263

Keywords: game, environment, ue5, unreal, engine, 5.2, nanite, performance, 60, fps

Id: mk2BaeNr_p4

Length: 29min 19sec (1759 seconds)

Published: Sat Jul 15 2023