This is a potato. 289 bucks from Amazon, it's
just about the cheapest Windows laptop that money can buy. But for gaming, it's terrible. Intel i3 CPU, 4 gigs of RAM, and no GPU. I'm making a game using the Godot
game engine, and six months ago when I tried to run it on this
machine, it looked like this. That's about 9 FPS. To make matters worse, because
Bloodthief, my game, is aesthetically inspired by retro shooters like Doom
and Quake, many players expect to be able to play it on one of these. Needless to say, I've been working
super hard to optimize my game. Using this computer, which is
my wife's computer, by the way. as a benchmark. And after six months since taking
that initial 9 FPS measurement, I'm proud to say that this same potato
is rocking a steady 50 to 60 FPS depending on the user settings. And on my actual gaming rig, I've managed
to increase FPS from barely touching 100 to well over 200 consistently. So in this devlog, I'm going to
be going over all the things that I've done in the last six months
to make my game more optimized. From the raw framerate improvements
to fixing things like Random stutters when the game first loads. The optimizations range from classic
techniques that you've maybe heard of to one thing that is honestly so
absurd that I can't believe it works. All right, let's dive
into some optimizations. Occlusion culling is just a fancy way
of saying don't render things that are blocked from view by other things. To demonstrate how this works in
Godot, I've set up both of these cubes with occlusion culling. Each mesh has an occluder
instance 3d node on it. And it's exactly the
same size as the mesh. If I hide the visuals for this mesh,
but we go behind its occluder object, we can see that the other one isn't drawn. That's occlusion culling in a nutshell. I've been putting off setting up
occlusion culling in my own game for a long time, because painstakingly
going through my entire map and creating all these occlusion objects
just seemed like a massive pain. But lucky for me, I'm using Qodot, which
is a plugin for making maps in Godot. And back in December, they added
this handy dandy checkbox that generates all the occluders for me. Now whenever I update a level,
I just have to click this build button and all the occluders
will automatically get generated. Only problem is there's this
weird bug where sometimes things get occluded that shouldn't. Like right here, right here,
right here, And a million other places that playtesters reported. This really breaks immersion, so I
wanted to fix this as soon as possible. I started looking into it, and
yeah, it's a bug with Godot. I ended up working around this bug
by modifying Qodot to add an extra cull margin to all my map geometry. But this will now make it so a bunch of
stuff that should be culled won't be. culled It's also just super jank
to modify a plugin like this. Luckily, the Godot devs have a
fix for this in mind, and I'm hoping it shows up in Godot 4. 3 or 4. 4. Regardless, adding occlusion culling
improved my frame rate by a whopping 50 frames per second in certain areas. Alright, we're off to a pretty good start,
and now on to the next optimization. Baked lighting is a lighting technique
where you basically pre calculate all the lighting on all the surfaces of
your game and save all that lighting information to a file so it doesn't
have to be calculated in real time when the player is playing the game. This is awesome because we can make all
the lighting super realistic and beautiful with very little performance impact. The only kind of sizable downside is
that every time you update your level you have to re bake the lighting and
light baking can sometimes take a long time depending on how big your scene is. Think on the order of minutes. I went ahead and set this up for
Bloodthief, and I gotta say, I'm a huge fan of Godot's baked lighting system. I mean, look at this before and after. The emissive light coming from the lava
just looks amazing and the indirect bounce lighting really helps all the
surfaces look more realistic and pretty. Best of all, depending on what room I'm
in, I'm getting around 30 to 100 more frames per second because baked lighting
is just way more efficient than what I was using before, which is Godot's Voxel GI. I do have to rebake every time I update
the level, but even this isn't so bad. I usually just turn off baked
lighting entirely when I'm working on a level and only set
up baked lighting at the very end. Plus, for me, baking only
takes like 15 seconds per level at medium quality setting. Why bake at medium quality settings? Well, Godot crashes if
I try anything higher. Ah, yes, another Godot
bug opened since 2021. Okay, medium settings it is. If you've been playing the Bloodthief
playtest, you've probably seen hundreds of these guys running around. And even though their AI is nothing
fancy, having dozens of these enemies on the screen at once
can really tank the player's FPS. One user reported getting as low as
18 frames per second at this part of the game where there are like
15 enemies coming at you at once. One of the main offenders here is the
fact that these guys are all using Godot's navigation mesh system to find
the shortest path through the game. To the player and any sort of
path finding in any game engine is gonna be really expensive. In this case, every knight
was recalculating the shortest path to the player. Every physics tick, which is just
a crazy amount of unnecessary work. I updated the Knight's code to only
recalculate the path to the player once every 20 physics ticks, which
is literally 20 times less work. Then what was being done before
and I can literally not tell the difference in their AI at all. One big gotcha with this method is
that if I were to have multiple enemies updating their path every 20 ticks If
those enemies are all activated at the same time every time that 20th tick
happens All these enemies will be updating their path calculation at the exact
same time Which basically defeats the purpose of this method entirely ideally
if we have multiple enemies chasing us at once All their path calculations
are spread out across time, so they basically happen at different frames. So what I did was add a random
integer between negative 6 and 6 to the 20 tick update interval. With this method, some knights
will update on the 20th tick. Some on the 19th, some
on the 21st, and so on. This makes all the path calculations
nice and spread out over time instead of calculating everything all at once. Which would surely cause a big
stutter every once in a while. The next optimization I got from
Miziziziz, which is basically just to turn off the knight's animations when
they're not visible on the screen. These knights have some really
detailed animations with like a bazillion keyframes, so limiting
how much the game has to actually animate them seems like a great call. The idea here is to use Godot's
visibility notifier node and basically just disable the Knight's animation
tree when the knight is off screen. So they basically look like this
when you're not facing them. After all these knight optimizations,
the results speak for themselves. That player that was reporting the
18 FPS before now never goes below 110 FPS at this part of the game. So at this point the game is running super
smooth for me on my personal machine. Rocking a solid 200 FPS consistently,
and even when I ran the game on my wife's computer, we're looking at that solid 45
frames per second that I mentioned before. So, imagine my surprise when I
checked the feedback survey from the playtesters and saw that dozens of
players with better hardware than me were experiencing terrible stutters
when they first booted up the game. Ah, shit. Here we go again. Enter the granddaddy of
all optimization issues. The one that stumped me for the
better part of the last six months. Shader Compilation. Based on my research, Shader Compilation
is basically the bane of every 3D game. On basically every 3D game engine. Digital Foundry even called
Shader Compilation out as the number one issue with modern PC
games that needs to be fixed. And Bloodthief is certainly no exception. Look gameplay I was able to capture
riddled by Shader Compilation issues. Stutter here. Stutter here. Stutter here. Stutter here. And then when we go to kill this
guy, we have a massive stutter here. So what gives? How come I wasn't experiencing
this issue on my machine? Well, the thing is, shader compilation
needs to happen on the player's machine. And once shader compilation is completed,
the results are stored in a shader cache and never need to be computed again. So, in other words, shader compilation
only really happens the first time the player boots up a new level. And I wasn't experiencing it,
because shader compilation had already happened on my computer,
so everything was silky smooth. So, unfortunately, as it turns out,
even though your game works great on your machine, when you give your game to
players, it may run terribly at first. Which sucks, because those first few
minutes of gameplay are the player's first impression of the game, and
are by far the most important. So this begs the question,
how can I clear this out? Aforementioned shader cache so I can
reproduce the stuttering on my own machine and go about debugging it I scoured the
web looking for an answer as to how to do this and some people did give hints as
to where this shader Cache may be stored like this reddit post here and this github
page here But the problem, I think, is that different GPU drivers may store
the shader cache in different places. I also think Steam, which my game
is using, is doing some funky shader cache stuff, because I found
another shader cache folder right next to my game in my Steam library. So, in the end, after basically manually
combing through my computer for suspicious looking files, I determined that these
four folders contain shader cache related stuff, and if I delete them, the entire
contents of all of them, it will replicate the shader compilation stutter on my end. Side note, it's kind of
terrifying just deleting random program files on my computer. Like they say nothing about
Bloodthief or anything on them. Like this file right here might be
needed to boot up my OS for all I know. Well, we do what we gotta do. For science. You, are finally awake. Also worth noting, since these files
aren't Bloodthief specific, this probably triggers shader compilation issues for
all your other games on your computer too. So, maybe don't do this if you just
downloaded The Last of Us and sat through that infamous shader compilation
loading screen for two hours. Alright, so I can finally
reliably reproduce the issue by deleting the shader cache. It quickly got painful having to go
through these folders over and over again manually, So I wrote a Python script
that just deletes all the files for me. Now it's finally time to
actually solve the problem. So what I'd like to do is precompile
all the shaders during like a loading screen or something. That way they don't have to
be compiled during gameplay. Ideally Godot would provide a function
like compileShader that we can call, but unfortunately it's not that simple. According to the Godot creator
himself, You don't know what to compile until you render it. In other words, in order to compile a
shader, Godot has to actually render a thing that uses that shader on the screen. Turns out this has been a known
issue in Godot since, oof, uh, 2022. And the prevailing strategy right now
is to literally take all your materials, shaders, visual effects, and display
them all on the screen at once in one big explosion right before the game starts. Since we definitely can't let the player
see this nonsense, we're We put up a fake loading screen, and then maybe put
up some fancy words that make it sound like we know what we're talking about. Like, compiling shaders. So, this giant explosion
at the beginning approach, unfortunately didn't work for me. The problem, I think, is that shader
compilation can be triggered by, like, a bazillion different things. For example, according to this guy,
shader compilation can be triggered by changing your lighting setup. According to this guy, updates to
transparency or skeletal animations may also trigger a shader compilation. And according to this YouTube video,
changing various settings on your shader may trigger a shader compilation. And there's almost certainly a bunch
of other stuff too that I'm missing. The little script I wrote basically
tries to simulate all the things that the player could encounter in game. It instantiates a quad for every
material that is in the map. It also displays every
particle effect in animation. It does all this while
progressively adding lights. But nothing that I do in this
initial explosion seems to make the stuttering completely go away. So I opted for a different and
even more ridiculous solution. Record myself playing through the level. Exploring every nook and
cranny, triggering every effect, and killing every enemy. And then, when the player starts up the
level for the first time, the game plays a replay of my playthrough at 10x speed. Not a video, but literally it is
replaying all the inputs that I recorded, and playing them on your machine. This guarantees that all the
parts that usually trigger the stutter will be pre compiled. Of course, similar to the other approach,
we'll put a loading screen up to make sure that the player can't see what's
actually going on behind the scenes. I also turn off the volume at this
point, otherwise the game hilariously sounds like it's like frantically
cleaning its room while it's loading. Check this out. Yeah, we'll keep the volume off. And oh my gosh, after months
of working on this issue on and off, this finally fixed it. Silky smooth. No stuttering at all whatsoever. I tried this out on my wife's
computer, which had stutters that would last like five seconds and
sure enough, silky smooth now. Funny enough, I checked Godot's
GitHub about a week ago and oh, of course Godot has an elegant
fix for this slated for Godot 4. 4. Well, FML. At this point I hit a wall
in my optimization journey. No matter what I did, when I ran the game
on the benchmark machine, I couldn't get the game to consistently reach 60 frames
per second, which, let's be honest, is the real goal for serious gaming. Also, as development has progressed,
on about a weekly basis I'll get a frustrated player on the Bloodthief
Discord unable to boot up the game at all. This leads me to an issue that I
don't really see talked about enough when it comes to deciding if you
should use the Godot game engine. Support for graphics APIs. For a little bit of background, there
are three main graphics APIs that allow a game engine to interact with the GPU. DirectX, OpenGL, and Vulkan. Unlike OpenGL and Vulkan,
DirectX is closed source and proprietary, developed by Microsoft. And since Godot is an open source engine,
Godot doesn't currently support DirectX. Further, if you want all the best
features of Godot, you're probably going to need to use Vulkan. Vulkan is the modern open source standard,
and OpenGL is kind of being phased out by the industry in favor of Vulkan. At face value, the fact that
Godot is a Vulkan based engine is not really a big deal. It is cross platform, and it is a modern
standard that should allow games to be more optimized than with OpenGL. The problem with Vulkan, however,
is that it's still pretty new. Since Vulkan came out in 2016, if you
have a GPU that was made before then, there's a solid chance that it just
doesn't support Vulkan, and in that case you wouldn't be able to run Bloodthief. The GPU manufacturers would have had to
retroactively release a driver update for their GPU that Does support Vulkan. Further, and this is just speculation
on my part, but I would guess that, it actually takes time for the
industry to make high quality drivers that actually work well with Vulkan. For example, the CPU for the potato that
I've been referring to this whole video was actually released right about when
Vulkan entered the scene back in 2016. And it's Intel UHD Graphics 620 driver
does technically support Vulkan. The problem is, it's freaking terrible. Like I said before, if I drop
my player in this empty scene, I'm hovering at around 56 FPS. But if I force Godot to run my game
using open GL instead of Vulkan, I'm suddenly getting like 140 FPS
in this simple scene on the potato. And just to fully prove to myself that
this is an issue specific to the potatoes Vulkan driver, I did the same thing on my
gaming rig, which has an NVIDIA 3060 ti. A pretty modern GPU. And if I force Godot to use OpenGL. Aside from breaking half the
visual effects in my game, OpenGL had worse performance than
running with the default Vulkan. So that was a lot of technical nonsense. But, in summary, in the current
state of Godot, you're kind of forced to make a choice. You can use the default Vulkan
based renderer, Forward Plus, which will have all the newest
stuff and all the best features. It will give you the best performance
on modern computers, But, at best, will probably run terribly
on old computers, and at worst, won't run at all on old computers. And then your other option is to use
Godot's compatibility OpenGL based renderer, which has less features, runs
worse on modern hardware, but has speed. better support for old computers. I'd be curious to hear how
things are over in DirectX land. It's possible that you may have to make
a similar choice with other game engines. I'm not sure though. If there are any Unity or Unreal
experts in the comments, I'd be curious to hear what your experience
has been with other engines. Anyway, I opted to go
with the Vulkan render. Which means that my dreams of one
day running Bloodthief on one of these are pretty much shattered. It also means that there's probably gonna
be some portion of players who just can't play the game, which of course sucks. Annoyingly, if the player's computer
doesn't support Vulkan, Godot will pop out this pretty misleading
error message telling them to try to run the game using OpenGL. Which, as I showed before,
is just not gonna work. Godot doesn't seem to have a way to turn
off this message, so if you get this error message saying that Bloodthief supports
OpenGL, just know that No, I do not. And don't you ever say I did. So, moral of the story, it's not all
rainbows and fairies over here in Vulkan land, so if you're considering using Godot
4's Vulkan render, just keep this in mind. Also, it's definitely a good idea
to try to test your game on a wide range of computers if you can. Okay, before I wrap this up, I
want to list off a bunch of other miscellaneous optimizations I did. I disabled screen space ambient
occlusion, which in my opinion does almost nothing in my game, but actually can be
pretty expensive on low end machines. Eventually I'll add more customization
in the graphics settings where players can choose if they
want this type of thing or not. I set up distance fade for a whole
bunch of stuff like decals, particle effects, torches, lights, and enemies. When loading a new level, I set
it up to load in another thread while this loading bar fills up. This is way better than just showing
a grey screen for 10 seconds while the level loads and the player just
thinks that their game is broken. I also added the ability to change
the resolution the game renders at. This definitely helps the game run
way better on low end machines. And it also has kind of an endearing,
chunky, old school aesthetic if that's what you're into. And that concludes everything I've
done so far to optimize Bloodthief. It runs consistently around 200
frames per second on my main machine, and 50 to 60 FPS on a potato. Well over 60 FPS if you
run at a potato resolution. Most importantly, if I look at the surveys
of players playing the game, Almost all the feedback says the game runs smooth
for them, which was definitely not even remotely the case six months ago. So overall, I'm feeling pretty
good about things for now. By the way, if you're interested in
Bloodthief, go wishlist it on Steam so you get notified when it releases. There's also a fully open
playtest going on right now if you're interested in that too. One last thing, I'm
super active on Discord. So if you want to chat with me
about the game or interact with the Bloodthief community, join the Discord. I'll be leaving a link to
that in the description. Alright, that's all I have. Thanks.
