W Love Games Conference 2022 Keynote Session with Ari Arnbjörnsson, Epic Games

with a great great pleasure because this is one of the best speakers out there in the games industry and I'm I'm daring to say this it's my honor to welcome to Stage Ari from epic games [Applause] ah thank you so much [Music] foreign [Music] [Applause] [Music] so hi everyone thank you so much for waking up so early after hopefully not partying too much yesterday I hope you're not hungover um yes it's always nice to be on the second day of the conference the first talk and everyone was like and I'm like don't worry this is very technical also sorry about that this is going to get a bit technical at Parts but I'm also going to try to keep it kind of fun I'm gonna try to um make it generic at part so that even if you don't kind of like do Graphics or programming you'll at least be able to have enough takeaways that you can bring this to your team one thing that I want to do no because I started doing this and I really like it is that I like to take a selfie with all of us to uh then I'll share it on my Twitter and if you don't want to be on the internet you can just go like this but it'd be great if you can go look at this and also wave to make it look a little bit more alive so I was thinking we could all say hi from Helsinki because that's where we are so I would count down three two one and then we all go hi from Helsinki let's practice it so three two one hi from helsink e oh yeah again but this is our morning voice I want to hear like party voices like hi from Helsinki okay let's try once more three two one hi from Helsinki much better that's the energy I want and then I'll put it on my Twitter philosophy and maybe in a touch of a retweet it we'll see so let me see him selfie mode let me come a little closer so it looks like we're way more than we actually are I'll come down here okay are you ready three two one thank you so much now that was a great start to the day so you might have noticed I have some up here I'll explain that in a while um so this presentation is called maximizing your game's performance in Unreal Engine I've been doing a little bit of road tour with this talk and um nextly actually next week is where I'm gonna have the grand presentation on real fast so you're actually getting a sneak peek um you're also getting a practice run so I'm practicing on you so if you see this online at unreal person I'm going to be reusing all the jokes so um sorry about that not sorry and um I have a bunch of things in this presentation that I just really wanted to kind of like um teach people about but there's this one thing in this presentation that I just I I like it so much I put it right at the start I've been looking forward to showing this thing to everyone it's me hello everyone so I am Pearson um I'm an Unreal Engine evangelist at epic games I often get the question what does an evangelist do it's a really weird name for like what I do some people would call it developer relations but I do more than that I go and speak at conferences so I guess to summarize it would be I try to drive up adoption of Unreal Engine and support the studios that already using it and then you kind of fill in the blanks how do I do that well I go to conferences and I speak about unrelenting about how you can make better games I talk to Studios I'm basically a first point of contact two epic games or if you have questions about unrelenting the store Mega grants whatever you can always hit me up and uh yes I am I you might have noticed my first name sounds because I am Icelandic so my first name sounds Finnish and my last name sounds a bit Swedish so often in Finland I'll go people go like Ari Zoom lesson no no no no just English please I am a I am a game programmer I've been 15 years in Game Dev I've actually been programming for 20 years and um every time I get stuck with my own code tripping me up doing something and making me spend hours on it I get one gray hair uh common question I get is my hair color real it is this is genetic me and my father and all my brothers we all look like this and we also have a quite a bit of spring in our steps so when you see like four white heads going like this you can go a lot of Nissans so that's us and um I put I put my personal domain there are that games because uh all my presentations both of these are unreal related uh often I want to share it with people I want to go hey go check out my presentations I'll put them all on there I also put my unrelenting notes there usually because unrelenting it's a vast software and every time you learn something new you're probably not going to be like remembering all of this so I always recommend just make your own notes like put in a Google doc or an ocean or notepad even or something like that and uh I did that and then I realized you know what actually I could just make this public clean up a little bit so I did that all of my animated notes are there uh if you're getting started with unwell I really recommend you check this out there's also a presentation there called Lessons Learned From A year of under Legend 4 Triple A development and that was my first year of making Eternal before it was announced and basically I started using Unreal and I was like there's a lot of stuff no one tells you I need to tell other people about it so check it out also this domain cost me about the same as a Netflix subscription per month and the only reason I had this domain so I can tell you to go check it out so please don't make that money be in vain and yeah this is my contact info so for this presentation um three topics that I want to talk about we have profiling in general and this is what everyone can like get something out of then I talk a bit about problem with animal engine if you are using other engines this is where you're going to go a little bit like who you're on and then lastly I'm going to go into practical examples I'm going to this is the meat of the presentation this is uh what I feel other presentations don't do that much it's like showing you actually wrong things to do and how to find them and how to fix them and of course because that's the most fun part left until last um yes and then I brought some stuff here I brought some swag so these are epic games keychains and then if you squeeze the sides it's a little cloth there to clean your glasses or if you don't wear glasses clean someone else's glasses you can just click let me they'll love that and um and I have one uh pack or one t-shirt also because one question is going to be like um a big one but whoever guesses it gets this one if no one guesses it I'm taking this back home so yeah you better get it and um I'm gonna be asking questions yes throughout the presentation well not lemony like for example when I'm related there's not gonna be any questions there and whenever I ask a question if someone gets it I'll just throw you some swag I'll try not to throw it too hard if you're in the back I'm sorry it might hit someone in the face um when I ask questions don't raise your hand uh just yell it out I'm gonna go like what's this and you just go like and then I'll go like yes and uh I think that's a good format so let's go with that I think we can already start with a pop quiz what is this that's good are your reaction time we actually take less percentage than that we only take 12 on our store it's okay this is uh if you haven't been doing that much profiling you this might look a little bit uh Alien but there's actually another number there also to give a little bit of hint does anyone recognize this um well I'll tell you these are milliseconds these are the amount of milliseconds in one frame from 60 frames per second and for 30 frames per second so uh for CPU and GPU you usually have you know first experims you have 16.66 milliseconds and then for 30 frames per second you have double that although nowadays gamers are getting more um well depending if you're doing a mobile game you can get away with 30 but for control games they usually want 60 unless you're going for something looking really really good but next 10 has always been about 60 frames per second and those milliseconds you should basically treat this as a budget treat it like money so uh some features they look great but they are expensive so you have to decide when it's worth it and to give you an example it would be like for example if you have a wedding and you have 16 000 Euros and um the one of you decides hey I want 5000 to be spent on catering the other one goes no catering is going to be the thing of the wedding I wanted to be 10 out of the 16 Ms or k that we have and some of my my goal that's really expensive that's a lot of times less money but if it looks good slices very tasty it might be worth it to you so for some games some some big chunks of that uh budget are going to be worth it because if you wanted to go for the quickest like the most fast frame you'll be just not draw anything zero there you go great game though so for consoles you can actually close spend close to all of your budget you can get really close to that number of like how much computational time you're spending because they're all cooled well they all are about the same Hardware or not about they're exactly the same Hardware however for PCs you cannot just use all of that you need a little bit of leeway also on PCS the The Gamers they are the users they're going to be the ones that want to say hey you don't get to decide how you spend your budget I want to decide how your game spans the budget so I want Shadows to be more or I want them to be less detailed I want the they want to go into the settings and configure all of them but the thing is not all of them so some users just want to be able to launch your game and start playing it maybe you'll have a little bit of order detection or something like that but the thing is you need a little bit of breathing space so um yeah don't go all in on like stuffing your frames too much and then lastly mobile um you might think oh I can get quite close to mobile for my target platform and not really because mobile phones don't really have a fan on iOS they're like technically more underpowered than Android so you can actually use a lot of that but the thing is on Android there was this kind of competition where all these different Android devices were like we have a bigger GPU no we have a bigger GPU we have a laptop TPU in our mobile phone it's like there's no fan in this how are you gonna cool it down so it will run your game super well super fast while it's cool it's for like under a minute and then it will start going on too hot thermal throttling because that's the only way it can not burn and then it will run much slower than um iOS devices for example so take that into account give your phone a little bit of breathing space like take only 70 print or something like that of the frame budget I often guessed the question when do I optimize when should I optimize and if you've done any kind of software developmental optimization you will often hear this quote it's like premature optimization it's the root of all evil blah blah blah I say about that because it's a bit of an absolute statement and I'm not a huge fan of absolute statements because we know who it is who only speaks in absolutes right yes who else only speaks in absolutes come on hate leads to anger anger leads to suffering it's the sin the Sith only speaking absolutes so uh but we're not going to speak in absolute so I'm not going to say like you shouldn't prematurely optimize at all um but uh like sure you shouldn't spend time on actually like premature optimizing but you should still be performance conscious during development so that means for example if you're coding don't sacrifice code readability to do some micro optimizations before you even know if they're even even affecting anything so I'll often I know of programmers we want to be challenged we want to make sure we're doing everything and super optimized um they'll start writing these weird for Loops because like it's a bit faster and you're like does it will you be able to measure the time and see a difference that isn't negligible usually the answer is no you won't see any difference so um basically yeah don't do it in advance so if you're for example are you already running on 60 frames per second on the target Hardware like if you're aiming at you know the nexting consoles then you're just you don't optimize but let's say that you are making something and you're like well I know I'm not supposed to prematurely optimize but then when am I supposed to optimize and then you uh my answer is this when it shows up on the profiler that's when you optimize it not before um we were making return all and we were trying to figure out our budgets but it's hard to do to figure out budgets on a console that hasn't released yet so we're trying to say like should we have this vertice like limits or like how much the thing is like we didn't hit those limits we couldn't even uh find them there were these uh usual things that slowed us down but the other things that we were actually trying to limit they never became an issue so um if you're doing games for nexting consoles I think it's great to just like make it as High Fidelity as you want but then when it starts slowing down then you see what you should slow down so then everything will look the best it can so practical optimization is basically only optimized if you're not hitting your frame times and usually you go for the lowest hanging fruit first so you go for the biggest impact for the least effort and that's what you use the profiler for you see something that takes a lot of time on the profiler and you're like okay well this is actually quite easy to fix it's just a simple check mark do that wow we got like a whole millisecond of the frame time those are the things you could go for first and then um you go for the next lowest hanging fruit first that would take the least amount of effort to optimize and then you do it a few more times and now it's time to ship that's okay environment goes you're not going to have time to do all these things you want to do you have to go you should go for the things that you have to go to First not the ones that you want to so also uh don't blindly optimize you should figure out the what why and how of optimizing games are they what are the what why and how excellent question I'm really glad you asked so the what why and how this is uh I came up with this uh it's like I don't know this is kind of a no-brainer but I thought you know what I'm just gonna put it down so you at least have it so first when you're trying to optimize a profile first you have to figure out what is law see what is causing the slowdown in a profiler because if it is time consuming it will show up only when you know what is low is when you start asking the question why is that thing slow and once we know the why and the what that's when we ask how do we fix it we ask how do we fix it when you know what and why is low um it sounds like an opener but I like some of the smartest people I know will not do it in this order because they will see something and they will immediately go huh why is this low they just run the game why is the game slow they don't even know what is slow they just think like why is the game slow in this part of the game I'm gonna just start fixing things I think this is affecting so they'll go directly for the how do we fix it then without even knowing what and uh can blame them but I always say like no figure out what first you'll learn more and sometimes you spend a lot of time fixing something and turns out it wasn't even that and then you might break something because whenever you write code there's always this like percentage chance of something breaking what why and how this was part one now we're gonna go more unrelenting specific I'm talking about profiling in unrelenting so uh we have these two programs that I usually recommend people use uh that we have more ways of uh profiling in the editor but these are the ones that give you great numbers on the what so under insights uh it's a program it comes with unrelenting it is not inside of it it kind of stands next to it so you just start it but you can start from the editor you go to tools and then run on unreal insights uh but because it's a standalone application you can also just go into the binaries folder where the unreal editor is and there's an um executable there called unreal insights.exe and then once you have unreal insights running you can just start your game and it'll just automatically connect and it'll just automatically start profiling and uh super easy so it will automatically connect to it but also there's this Trace command parameter that you can start your game with and you can use it to overwrite which channels you are measuring so by default you're measuring CPU frames log bookmarks and you can use test Trace equals to override it you can even put default in there to get all of these default channels put comma and then keep adding to a GPU memory file a little time something like that and making a shortcut to your game with those parameters is Handy but if you're working in a team you might even make a bat file and then you don't even have to profile it on your own machine you can use Dash Trace host and then put IP to connect to another computer and this is great if you're running this on a console or a mobile device or even just on another computer but the developer who is profiling it wants to do it on their work machine then the other command I want to talk about because unreal insights is great for CPU and memory and all that kind of stuff but then dpu that's always its own kind of monster and uh well we have a separate command for that also it's called profile GPU and this you can launch both in the editor and in development build using Ctrl shift and comma or if you don't have a keep like for example if you are running this on a development kit for control and you don't have a keyboard connected to the console but you can remote into it then you can just give it the profile GPU control command and what it does is that it will profile the current frame on the GPU side and this is this command is profiled out in shipping build so you can't use it there so if you want to use it you can you just gonna have to make a development build and if you're running in the editor it will open up the GPU visualizer it looks like this if you're not running it in the editor it will print the same information out to the console in packets builds so these were the first two sections and I will go to what in my opinion is the most fun part where we show you the Practical examples um but before yeah so what I've done with the Practical examples is that I use the lower starter game so the Lyra start the game it's a sample gameplay project that you can um well as it succeeds shooter game shooter game used to be our old like hey here's how you can make a shooter game like how epic would do it you can download it and use whatever you want out of it you can even build your game on top of it and um it's well what Lyra does is that it's our new version of shooter game we're not working through the game anymore now everything is in Lyra and it showcases a lot of Unreal Engine 5 features basically if you want to know how am I supposed to use all these new features in un25 to make a game check out Lyra um we are working on getting it console certified so you know that it's full of all best practices how epic games would use our features and um yeah I wouldn't be surprised it's even encouraged that you can just download Lyra and just start modifying it until you have your new game so you can download it from the marketplace the team is really happy with it because it's such a great example of everything and how you should do it well and I just took it and I was just like you know what I ruined it I'm gonna make an example of how not to do everything well about how to do everything horrible so I put some horrible um performance issues into it and now we're going to fix them together the first curse that we're gonna fix the first issue we have four of these that I made two of them are GPU related so for the graphics people one of them is CPU related for gameplay or uh programmers then the last one is going to be a memory League low editor frame rate someone comes to me let's say I'm a lead programmer or something hypothetically what is it what is it the editor used to run at 60 frames per second but now it runs only at 30 frames per second it is post low I don't know what happened it was working just fine yesterday and I go like okay well let's see if I can fix it so my first thing that I'm going to do is that I am going to verify just what the frames per second are in unrented uh before I continue how many people here are using unrelenting one two three okay so wondering how many questions should I be asking or should I just be telling you all the answers um I think I'll just I'll skip the questions I'll just um I'll just throw this out at the end to random people I have actually so many that I think all of us might be able to get one so low editor frame rate yes uh we have a way in unrelenting to see the frames per second does anyone want to guess how to does anyone know okay it's okay so in non-religion this is a little hamburger icon there in the top left and you could just click on it and say show FPS and uh you also have a keyboard shortcut Ctrl shift and eight what you can also do is you can use the little control command uh input there and write stat space FPS and then when you do that we'll get this there's a number uh FPS and then also the milliseconds for how long each frame is taking and usually when someone gives me a relative number like array the game is now two we lost two frames per second let's say we lost five frames per second it's five frames per second slower this number means nothing to me because it depends on which computer was running and it depends on also what was the frame rate it was already running at so if someone tells me we lost 500 frames per second this can mean very different time spans depending on for example if we were already running at 30 frames per second and we lost five frames per second that's like quite a good as 1 6 of the whole entire budget but if we're running at 120 frames per second someone says we lost 5 frames per second that's like negligible it's like a margin of error so usually it's better to say our frames frame times are 10 milliseconds slower and I'm like wow that's a lot of time I know exactly how long it is it's the same like no matter what the frame rate was before but the thing is I just showed you two different tools I showed you how to profile a CPU problem and how to profile a GPU problem so we need to figure out first where is this time being spent is it being spent on the CPU side or the TPU site and unrelenting we have another command to show more info on that it's under that in a hamburger menu again top left stat engine unit in the unit stats so you can also again also use the console command start Space Unit which is usually what uh more technically minded people will do because it's much faster just write them and then we'll get also these numbers so uh the first number is basically the same as the one here on top it's the number that the entire frame takes the entire frame is taking about 33 milliseconds the game thread although it's known as the main thread on the CPU that's taking about five milliseconds that's actually very good I would say we should be putting more stuff into the game to try to get closer to that 16 milliseconds so like we have a more High High Fidelity game or like better AI or something like that but now the draw threat taking 30 milliseconds and we know that 16 was the max we should be going for so something is happening on the draw thread and the GPU itself is looking at 33 milliseconds so this is obviously a dpu problem not CPU so what I'm going to do is that I'm going to run the profile GPU command now so I'm just going to write Ctrl shift and comma on my keyboard and then this pops up sorry about how small it is but I'll just walk you through it top we have the graphical representation of an entire frame one frame and then you can click the little white triangle to see like more what's happening but what how what I like to do is I like to use the numbers down below it's the same as above but in a tree format you can sort it by duration and you'll see exactly how long everything takes so here you can go into 0 does light slides taking five milliseconds lumensy lining tagging three okay like some of these are really slow so I'm going to just start going into settle this random virtual Shadow Maps bats raster passes okay so there are some things they're called cursed Point lights and each of them is taking half a millisecond and that's quite a quite a few of them um okay that doesn't really look right we have some rectangle lights that are taking like no time and then we go into the lights themselves direct lining on patched lights then the cursed lights are there also and they're taking like one third almost of uh millisecond each of them so we just used a tool to measure what's happening during the frame and it showed us exactly what it is exactly like these these Point lights so now we know the what what is low next because we know the what we can try to figure out the why why are these slides slow and I'm just going to try to find the lights so I'm gonna go in the outliner oh there they are convenient I'm going to click one of them and uh okay it's right over there there's like 10 of them right next to the door looks fine uh like how someone would use lights I'm just going to start searching for the details panel for anything that looks weird so we can see that they have cast Shadows on I don't think they need caseros because there's they're so small and then the attenuation radius is 5 000. it's quite I don't know what that means like let's zoom out and see how big this light is okay okay okay okay each of those lights is lighting the entire level even though we can only see a little dot there so I'm gonna fix it I'm gonna start by selecting all this curse Point lights turning off uh turning down the attenuation radius to about 100 you can see the outline now and I'm going to turn off Shadows and the frame rate jumped straight up from 30 to 60. single check box and one input field that are changed from 5000 to 100. this is what I mean when I say lowest hanging fruit first this was a small change that gave us a humongous amount of frame time back for and it didn't change at all but the game looks because all of these lights were so small anyways uh they were just misconfigured maybe the artist uh pulled them from some other part of the level and just adjusted the lighting intensity without doing anything else um but yeah this shows you you don't need to guess you can just see the numbers find it estimate why is it slow and then fix it now I'm gonna run the profile GPU again just to show you the difference of what it looks like so I'm going to press Ctrl shift and comma and now if you look down the tree you can see the duration of the whole frame is 12.96 which is under Earth 16 millisecond budget so everything is now running at 60 frames per second I'm going to go down into light start lighting and best lights we have the directional light we only always have one of those so that's fine that it takes a lot of time because it's lighting our entire game and I can't even find uh the curslet anymore they've just been bats together so that's the second part I said about when you should stop profiling when it doesn't show up on the profiler anymore it doesn't can't even see it I think the first problem out of four has been fixed now so I think we should go to the next problem tiny actor huge hitch um this is like a tricky one uh anyone who's made a game well I'm not going to spoil it let's just go into it so uh wait actually before I go there let me I want to have this doing this okay beautiful so QA comes to me this hypothetical lead programmer and says something's wrong and I'm like oh oh okay well I handled that other problem quite well give it to me I think I can handle it so we have this thing where um in one part of the level we'll start streaming in the next level in advance and then we have another part that also uh has a little synchronous loading meaning that it loads before the frame continues the next one of a tiny actor and it usually always loads just fine you can't even notice it it's imperceivable but sometimes there will be a huge hit depending on how you do this and I'm like well that's not like a pain in the butt to replicate but they say don't worry we have a QA engineer who made a rep Pro case for you and I'm like okay they say go into this level and there's two actors there oh wait before I continue uh I don't want to show you this in the editor I want to show you what it looks like when we have a packet built so I'm going to start by packaging the build so you'll have like so like how the game would be running on steam or something like that but still in development build so I'm gonna just do platforms build and as you can see package supported for Windows and through the magic of pre-recorded presentations I can now skip two packets incomplete some of you might be asking well Ari is Unreal Engine really that fast at packaging the project and uh no yes let's say yes okay so okay so now we're done packaging I'm gonna open the folder where I save the game to and I'm gonna make a shortcut to Unreal insights from uh the unreal editor thing so I can just start it from there instead of always having to open the editor I'm also going to create a shortcut for the Lara game executable and I'm going to put in uh trace the trace overrides their um so that I kind of have better control of what I'm tracing and then I'm gonna just open the properties for the shortcut and start writing in dash trace and in this video I'm doing it super slow so I'm just going to fast forward so this is what I put in so for CPU and loading profiling you can use Das Trace equals CPU GPU frame log bookmark and then I'm also adding in file in load time because they said there's something to do with loading um and also if you want to do memory profiling it's just uh I just put it there also you can use default to memory um I was told uh you should not usually put all of these together because each of them has a slight overhead and you want to be as accurate as possible so uh yeah just make each tray session kind of looking for only what you really want to find also you noticed here I used stat named events you don't need to use it um I like to use it when I'm doing CPU profiling it just activates a bit more CPU timing events so that you'll see a bit more what's happening it's like it adds a tiny bit of overhead again uh but it's fine because it's all relative anyways um so if something is really slow like it's going to be really slow even even without this overhead so you can see all the trade cells we have available on our unrelented five documentation page that has called unreal inside reference the thing is if you go to it from Google they will usually still link to the old 4.26 and those trade channels are quite old so if you do open the page make sure you change the url to say for a 5.0 instead of 4.26 or 4.27 or you can just use secure code to go there directly so now that I put this into the shortcut I'm going to start first unreal insights just have it there and then I'm going to start Lara game and you can see uh already that the trade session starts in the background and I can even double click on it if I want to and look at it in real time as it is capturing data so for now I'm just going to minimize it and I'm going to go to the Repro case of the queue engineer made for me so they have two volumes there they made okay so first I'm gonna try to load the tiny actor that happens synchronously and yeah it was instant cool how it's supposed to work and then I go into level asynchronous it says started loading map and then in a few seconds downloading map that works perfectly and I mark this as can't reproduce fixed good job but the Q engineer becomes a little bit angry at me says I told you you should do it in the opposite order I'm like fine okay let's try that load level asynchronous and then immediately run over oh there's like a three second hits there I'm gonna so this is actually part of the presentation where if you could guess this I would give you an Unreal Engine t-shirt so can anyone guess why this is happening it's fine it's fine I will tell you anyways but I will take this one with me so don't worry don't worry also because it was a trick question I would have given you the T-shirt but I would have told you you guessed the why instead of using the what first so using my what why and how we're first going to use the tools to figure out what exactly is happening um so I'm going to open the trace file and so this is unreal insights it's a rainbow of colors we have a few tabs up there timing insights acid loading you have the time of the game there here we have the frame times and then you can turn on other tracks like graph track that shows you again how long its frame took by default it's docked and it takes a lot of space so I like to right click and make it scrollable so it goes up with everything else also because I have Hans is something to do with asset loading I'm going to turn on the asset loading track which shows you at which time every single asset is loading then I'm also going to turn on the frame track which shows me what number or frame it is that is running at each time and then here we have the CPU stack at each time of the application and it's inversed so the bottom here is like the first function that's running that's usually the engine loop on the base and then you can see which function that calls and which function that calls stuff here we can see we started loading the asynchronous map and here we can see the frame number five for eight we can scroll to scroll wheel to zoom in a bit and also here you can use shift and scroll to see the time three seconds this one frame took here this is one frame during like from 19 to 22 seconds all the assets being loaded here and as soon as the level is done loading it continues so something is happening there that is waiting for the entire level to load so let's just go down the stack and see what's happening so it's kind of hard to see so I'll just say it out loud so we have here execute uh blueprint function trigger curse little tiny actor that is calling load asset blocking load object load packets internal and then it's calling a function that's called flush async loading plus async loading why is it calling that and why is it taking 2.5 seconds this now we know the Watts now we can start acting the why why is this happening why why is my game so slow why is that slow good thing about online insights and also about on residents in general is that you have the source code for the whole of unrelenting and unreal insights uh connects you to the source code so you can even right click on the functions just say go to source code and then we'll get this so here's the source code we have a function called load packets internal getting really technical now and inside of that we have here loads pack load packets asynchronously and then what it does is that if it is doing something it will call a function called flush async loading and floss basically means load everything and wait while it's loading and if it's waiting on the main thread the game can't continue it's just gonna stop so it's that's why we have that hitch that's very interesting so apparently we cannot call a synchronous loading function something that makes the acid come immediately while something is asynchronously loading in the background because it will just suddenly make everything uh flush and I don't honestly I don't know why because it it gets a request ID and then it says like here's the request ID so it should in theory only like do that one but the thing is the uh unrelenting has its own threat for loading assets and that one happens and uh serial so not parallel so it might maybe be loading all the assets before that until it loads that maybe anyways we cannot have that hits so what I'm going to do is I'm gonna just change this tiny actor loading to be synchronous to asynchronous so I'm going to open the blueprint called the trigger cursed low tiny actor and there we can see load asset blocking that's the what why it's being slow and I'm gonna just call async load class asset and use that instead and I'm going to put BP tiny actor there instead and then I'm going to connect that up but I'm going to connect the output pin not to the exit pin but to the completed pin so this one over here and that means that this print string will only happen when the actor is actually done loading and not just when it has triggered the loading of it because then if you do the upper one it will immediately call a string while it's still loading in the background so I'm not going to show you what it looks like now because it basically doesn't hit anymore but the thing is if you start loading the level first and then the tiny actor it will still wait for the um whole level to be downloading before it loads the tiny actors are still waiting for it a little bit but the game itself running there's no hitches so yeah people sometimes ask why is there nothing we can do well actually there is this I was going through Lyra and I noticed this class called f streamable handle and that one has this function called wait until complete so you can do that while loading in a single asset I was like that's interesting and then I'm going to read the comment over that function now and it says blocks until the requested assets have loaded like the other one or the functions had and then it says this pushes the requested asset to the top of the priority list oh sounds good but does not floss all isn't loading it's like they knew like oh this function doesn't call Floss async loading and then the other function doesn't mention that they're just like well this does something so yeah so you you have this so it says usually resulting in a faster completion than a load object called so you can you can try this one if you want to load a small asset and you want it to be done faster than everything else but without the blocking so okay we can do that and then let's just call this fixed for now okay curse oh don't stop the video Yeah curse number three of four bulky Horror Picture Show that was the last pun only pun of the presentation I promise so um QA comes to me and says something's wrong no I'm like okay I can do it yes tell me what it is I'll fix it they go as soon as you start a game and turn around the frame rate crashes and I'm like okay sounds easy enough so I'm gonna just go in and I'm gonna turn on this that FPS turn around and okay this is really slow there's like eight frames per second but I'm gonna use the new control command I just showed you profile GPU and press enter it hits us for like a second while it profiles the frame and then it keeps running the game what it did now was that it saved it because this is not running in the editor this is an executable that I showed you and um we don't get the pop-up that I showed you in the first cursed example so it goes into the logs instead so now we have to dig up the logs so the logs are here under uh saved folder logs and there's the log file and I've scrolled down already to where the profile capture was you have to scroll a little bit and the thing is this happens in the order that they happened on the GPU not sorted by how long they are so you're gonna have to go down a little bit until you see the percentage in the frame time uh show up so here we can see translucency pass is taking 90 of the frame time translucency after death of field parallel sticking most of it and then down here we can see the culprit ninety percent of the frame time 135.96 milliseconds is a material called M underscore cursed fog card particle and the good thing about using unreal's own profiling tools is that it gives you extra information that maybe other profiling tools would wouldn't show you so we have this um particle material but the thing is unreal CO2 and this is the particle system the agrosystem cursed fog that is using this material ah fantastic so now I can open this particle system called cursed fog and then check out the material to see exactly what's happening foreign there's the cursed fog right there I'm gonna open it and I'm gonna just start looking because we know the what I'm gonna start looking for the Y spawn rate is 50 and not unusual for a particle system initial particle initialized particles has quite actually a big initial size 500 times 800 and it's about 300 particles for a particle system is not that much but for some big particles it is quite much and this is the material so we have a noise note there and it's set to computational instead of a texture sample levels 10 larger numbers cost more performance oh there's a noise note also levels 10 and then there's a water noise there that says quality 10 higher numbers are slower and higher quality and then we have Alpha only one percent visible so maybe we have some graphics people there because this is this will be you would fix this no matter what game engine you'll be using so can I just have some suggestions like how can we just make this perform better just any suggestions there are many right answers to this one yes everyone who said lower the quality keep your hand up let me see you said who also said lower quality ew what else should we do lower the quality on the notes yes plenty of other things to optimize Yes actually do you want two because those are two really good answers you stole two answers uh what's that yes we could just make the material more simple thank you thank you thank you all of these good suggestions all of them at work so um by far what I would start with was what you said the alpha doesn't need to be one percent visible that's barely visible so someone is just like I want this to look like a real Fork there's actually a um fog component in unreal that you can use uh so you don't need to like take it like that by using cards on top of each other and uh there's this thing called fill rate on all gpus uh that just means how many pixels the GPU can draw and that is always limited that's the first thing you'll hit a limit on and that's usually when you draw multiple things over and over on the same uh area and that's what you do with transparent graphics so just do less of it because this will put a PS5 or export series on its knees doing uh overeating the fill rate so much so I'm gonna put the alpha up let's say 10 or even more also um it's kind of making these three notes to make a fork that looks kind of like this the thing is none of this is dynamic you can't really adjust the parameters it's always going to look like this so why are you why and like so instead of you know calculating how this would look every single frame for every single Pixel just make this once in a or photoshop and then just make it a single texture sample instead and also like if you do want to make this Dynamic you can just make all of these because I I wanted to make a really good example of something that takes a lot of time and I don't know what to take a little time like no one would realistically do something like this but still um I just wanted to show on the profiler so you can make all of these texture samples by default but it does by default you have to be really dumb to be to switch it over to competitional and put up the level so much um but yeah those are a few things and then no that sounds good so uh yeah I don't have a video of it being fixed just imagine a fork that looks about the same but it runs at 60 frames per second wow now we're at the last practical um example and now we're on a roll we've been fixing everything using my what why and how let's say QA comes to me something's wrong and I'm like ah what is it my child I will fix it and I say we have a memory leak and I'm like memory like unreal Engineers tools to fix those things awesome okay give it to me how do I reproduce it okay what tells me well it's actually quite easy to reproduce and I'm like huh if something is easy to reproduce it means it'll get fixed it's like because you can find it every single time it's just a matter of time but they say you start the game and you just don't do anything I'm like okay perfect and then you leave the game running for three days and I will crash only on consoles and I'm just like do you little that doesn't sound fun and I don't want to have this presentation run for three days while we wait for it to crash but the thing about memory leaks is that if even the slightest amount of memory is leaking we can find it and nothing should be leaking in the game so because of this very awesome practical example you're never going to have this but for this presentation let's say that you just hit a gold mine I'm gonna start the game and I'm gonna um profile memory for a few seconds and then I'm just gonna be done and also so I'm gonna launch the game now with uh insights and now I've changed them to be running the memory and I'm going to run the garbage collector every frame so we know there's for sure nothing that should be persisting between each frame so I'm gonna run uh type GC dot collect garbage every frame space one now that's running uh there's a good command to know you should just uh write it down into your notes like I'm recommended to do and now it's done uh that's all I needed to do in this perfect example where everything is like you know dancing on roses so I'm gonna open the trace file and uh here you can see the trace the memory Trace uh here we have the flags you can see the garbage collection here running every single frame if it's in the array you can press that little white triangle up there and it will hide the excuse me hide the lines but I want them shown for this example and also it one of these is the category of where the memory is being allocated this is our old low level memory system but I'm just going to get rid of them because I'm going to use the new system which is per stack allocation so I'm going to select the time period here and then run a investigation on it so here we have rule let's go with memory leaks rule and I say give me every single memory allocation that happened to be feed between a and b and then C is when that allocation was released so give me every single thing that happened here and then was never released and then I run it and I say I'm gonna sort those memory allocations by the call stack and then we have here on the top 22 megabytes leaked and I'm gonna right click on it and say expand critical path and it will expand down the stack all the way to when the memory allocations are happening so I'm gonna toggle this away so here we can see exactly where those memory allocations are coming from so this is uh the stack again um and we can see that it's happening in a function called a cursed actor tick you know I don't think we need to go any further because I think I solved it every single memory they kept one thing in common they were all in an actor called something cursed just change the naming conventions fixed just kidding no it's not that easy so but the thing is um because this is showing us the entire stack we know that it's not happening in a sub function of this it's happening in the function itself how many here are programmers by the way one two three four oh yeah this is just for you the next part sorry for the rest but now you get to see a little bit into the world of uh memory leaks and programming stuff so um I'm gonna show you the code for cursed actor Tech this is an actual memory leak that happened during the development of eternal and uh I was the one in charge of finding it and when I found it I was like this is so sneaky and clever and what a fun memory League that could definitely happen to someone I have to put in a presentation someday ah so I'm going to show you the code I'm going to start explaining what the code is doing okay we're almost done this is the last thing I'm going to explain what the code is doing and uh if at any point you think you know why it's leaking just shout it out here's the code this is the header file we have a array of structs and the struct is called f stats capture so these are value types and then in the tick it will call the super tick it will add a news that's captured just that captures that add new stats capture and put the time there and then if we have more then 60 stat captures in the array we will send those that captures to the analytics server and then we will empty that array we're still emptying the array every time it's more than 60 so why is it leaking memory hmm why is it leaking what I'm what yeah but say it again there's a new there's a news that's capture actually you know what you will get the T-shirt you will get it no comment well there's something strong going on there definitely it's like your Hunter's right then why is that new there um so in C plus uh this is how you should have written it because this is a value type so you just call the Constructor immediately whenever you do a new you're creating a new thing of the object to memory and then if you use the star it's gonna dereference that and create a copy and uh put that into the array while the original new is floating in memory it's like if you're coming from C sharp you might write you know there and then you will try to compile it and also you can't put a pointer type in two value types you're like what's that then you Google it and Google will say how to convert object pointer into a value type and also you just use the d-reference operator there the star and then you put it there but this is not how you do it in super plus when you have a value type you just write the Constructor immediately so that was the last curse um I learned everything about online search for this presentation from this presentation called unreal insights live from HQ inside unreal on YouTube so these are the two Geniuses that made these tools Stefan and you can see this presentation by following the QR code or just Googling that thing and we are out of time so if you want to reach out or if you thought wow these profiling tools are amazing I want to use unreal but I want to know a little bit more about it let's contact the array so there's my email and there's my Twitter and Linkedin I'm quite easy to contact we have even time for questions we will make time for questions thank you so much to everyone thank you

Channel: W Love Games Conference

Views: 2,041

Keywords:

Id: cGx4q5E8xqA

Length: 55min 10sec (3310 seconds)

Published: Fri Nov 04 2022