Improving My Game With Tester Feedback

Video Statistics and Information

Video
Captions Word Cloud
Reddit Comments
Captions
Hey what’s up guys and welcome back. Today we’re  going to be working on Steam Engine Simulator’s   new DLC. But first I’d like to thank BeamNG,  Reyna, Maisteer, Joe and Bacteria Rancher for   supporting me at the Master Mechanic level on  Patreon and Brilliant for sponsoring this video. Brilliant dot org is a great platform  that offers a wide variety of amazing,   interactive courses. These courses have  high quality, hands-on practice problems   and demonstrations to make sure that you fully  understand the subject matter they cover. So   it’s not just memorization or information being  presented on the screen, comprehension and real   learning is the goal. Learning a little each  day is a great habit to develop and will help   you take your software skills to the next level.  Probability is an important concept to understand   for a lot of different software techniques and  this course, “Predicting with Probability” is   a great introduction to the topic. Overall, it’s a  really great service and I highly recommend trying   it out. To try all of Brilliant’s features for  free for 30 days, visit brilliant.org/angethegreat   or click on the link in the description.  You’ll also get 20% off Brilliant’s annual   premium subscription. Once again, thanks  to Brilliant for sponsoring this video. Alight so this is what the DLC looked like the last time  we talked about it. I said at the end of   that video that: “we’re only missing a small  gauge cluster and the mailbox feature and   it’ll be ready for release.” I probably  shouldn’t have said this so confidently though. Shortly after that video, I released a build to my  testing group and what ensued was complete chaos.   The Discord chat was immediately flooded with  irate gamers. Some of the complaints were minor,   like “the wires being too wobbly” or confusion  about why the generator kept spinning even   when the engine was disconnected. Things that I  thought would be intuitive turned out to really   not be for the majority of players. For example,  I thought it was obvious that profitability would   depend mostly on fuel economy. But virtually no  one from the testing group picked up on this.   Instead they accumulated 100s of thousands  of dollars in debt while keeping the heat   level way too high, which inevitably ended up  blowing up their boilers, making them even more   frustrated. And yes, you can blow up your boiler  now but we’ll talk about that later in the video. In general, testers found the game stressful  and unintuitive and complained that the   difficulty progression was suboptimal. In case  it wasn’t obvious, this game was not supposed   to be a rage game, so the way players were  reacting to it was not at all what I wanted. It was a complete disaster. This group  was about as biased in my favor as   possible since they were my own Patreon  supporters. Paying customers on Steam who   were not so invested in the project  and weren’t as knowledgeable about   engines as my testing group would  likely have torn this game apart. Now, getting negative feedback like this  wasn’t fun, but that was far from my   biggest concern which was the effect this was  going to have on Engine Simulator’s timeline.   A large proportion of you watching are more  interested in my main project Engine Simulator,   which is what my channel is known for. A lot  of my Patreon supporters are eagerly awaiting   Engine Simulator updates. The simulation  prototype that I demonstrated in my last   video works really well and it’s just waiting to  be made into the final product. No one wants to   continue working on Engine Simulator more than I  do, but the reality is, I just couldn’t do that. This DLC already had around 40,000 wishlists on  Steam and at least a few thousand people actively   following its progress on YouTube, Steam and  Discord. Canceling it wasn’t an option without   upsetting a lot of people and throwing away  hundreds of hours of development time. However,   releasing the product as it was, would  have been even WORSE than canceling it.   The hostility that I deal with on Steam is  already enough that I dislike posting on the   platform. I can’t imagine what I’d have  to deal with after people both paid for   this product and continued to spend 12 hours  getting nowhere before demanding a refund. So I was stuck with a set of bad  options and multiple large groups   of people who all wanted different things  from me. It was hard to tell if ANYTHING   I was doing was making anyone happy. For  example, in my last video about this DLC,   the top comment was a suggestion for a  completely different DLC. For the first   time in my short game development career, the  pressure was getting to be a bit too much. Alright, so now that we know what happened, we need to figure  out how to solve this problem. And to do that,   we need to come to terms with some  realities first. No matter what I do,   I won’t be able to keep everyone happy. I know  this might seem perplexing to some of you,   but it’s important to me to release this  product in a state that I’m happy with   and I want to do that before  moving on to Engine Simulator. If you’re frustrated that I’m not working on  exactly what you want me to be working on,   I get it. I do get a surprising number  of people asking or demanding that I   work on a particular thing, which is a  bit disappointing for me but it shows   that people actually care about  my work which is not a bad thing. Likewise, it seemed like the testers really  didn’t enjoy this DLC, but at least now,   they actually cared about the outcome of  the game. In all previous versions, testers   were mostly neutral, maybe even disengaged.  The game was at least engaging players now,   which was a step in the right direction. We  just need to take the hundreds of points of   feedback, and try to improve it while  keeping the schedule under control. We’ll focus on its two major issues:  The initial learning curve being too steep The difficulty progression not being very   satisfying and leaving players bored  at some points and frustrated at others Let’s look at the learning curve first. I tried  to avoid including a tutorial in the game by   making intuitive mechanics that required little  to no explanation. The game’s primary loop that I   mentioned in the last video mirrored real-life  economic principles and worked like this: Lower fuel usage allows lower energy prices  Lower energy prices increase  demand for migration to the city  Greater populations require more energy and larger of energy amounts can be produced more   efficiently due to an economy of scale effect Higher efficiency allows lower fuel usage  and the cycle continues, until  the maximum population is reached I really overestimated how “obvious” this  mechanic was to players. The game seemed   perfectly understandable and logical to me since  I developed it, but expecting players to figure it   out immediately was probably not reasonable. Most  kept the electricity price too high for too long,   accumulating 100s of thousands of dollars without  progressing. Others focused on just running the   engine at whatever heat level worked, which was  usually many times the required level for the   conditions they were operating under. Some players  completely misinterpreted the goal of the game and   instead of maximizing population, maximized profit  by adjusting the electricity price to always   exceed their expenses, instead of adjusting their  fuel use to keep expenses below their income. It was also unclear to players how all the  controls were meant to be used. The intention   was to present a set of basic controls and leave  it to the player to figure out how to use them to   achieve the game’s goal. However, most players  didn’t seem to enjoy this process of discovery. To fix this problem, we can do what  most games do and start with an easy   section that introduces the player  to the game’s controls and goals. I added a task list and a letter viewer to  this empty area. Some of the visuals are   still temporary so just ignore those for now.  Having a list of tasks that need to be done and   completing those tasks seems to be enjoyable  and much easier to understand for players. I   also created a simple mission tree to organize all  of these tasks like most other games do. This is   a generic system which will make it easier to  implement the rest of the game’s story later. The first section consists of a few tutorial  missions. The goals are mostly understandable   from the task list alone but the complete  details are given to the player through   short letters from the game’s NPCs which will also give some background and history about the town. The town’s mayor and a local mechanic are   the first two NPCs to make an appearance and  they run you through some routine tasks like   fixing the power lines, fixing a ruptured  boiler and adjusting the engine’s controls. To teach the player about operating the steam  engine, I went a step further and added an AI   that can run the steam engine autonomously.  I could have hacked the simulation to force   particular speeds or values at particular times,  but I think the AI approach is much more powerful. Firstly, it proves to the player through  direct demonstration that the game is   possible with the right inputs. It can also work as a copilot,   allowing the player to control a few of  the settings while it handles the rest.   This allows the game to slowly transfer  responsibilities to the player as they   become more familiar with the game. The AI can also be used for automated   testing that would have otherwise  taken me hours of manual gameplay. It’s oddly satisfying to watch the AI adjust  the steam engine in response to changing load,   so I might use it for something in the  future, maybe as a separate game mode.   If you’re wondering how this AI works, it’s a  pretty simple feedback loop system. The output   parameters that we are interested in are  water temperature, engine speed and water   level. Each of these outputs is primarily  controlled by the heat level, throttle valve   and water valve respectively. We also have two  safety devices, the relief valve and the brake,   which can be used to prevent boiler overpressure  and engine overspeeding. These safety controls   do reduce efficiency when they’re enabled, so  they’re only used in exceptional circumstances. Each of the feedback loops senses the  output and incrementally adjusts the   input until that output value is reached.  In the case of engine speed for example,   if it starts dropping below the target speed,  the throttle setting will be increased. When   the throttle valve is open, more steam will flow  out from the boiler which will cool the water,   likely causing an increase in the heat  setting from the heat feedback loop. The   systems mostly do not know about each other and  work independently to keep everything balanced. Next we’re going to take a look at fixing the  game’s difficulty progression. One of my testers   kindly put together this chart illustrating the  problem. We can safely assume that the left half   of the chart refers to the steep learning  curve which should be mitigated by the   tutorial that we added earlier. The progression  from 4000-12000 people seems pretty reasonable   until we see some of the modifications  that another tester added to this graph.   It seems that the real issue here  is that the game is too easy and   does not present enough challenges  in the middle of the progression. So why did this happen, especially when  I put in so much effort in the last video   to specifically prevent this from happening?  Well, the early game frustration definitely   didn’t help but beyond that, I think there  were some flaws in the approach I used. So,   I developed a new model which  should solve these issues. To understand this new approach, we first need  to understand what we’re even measuring here.   The game is about engine efficiency, so we  need to compare the player’s performance   with some sort of benchmark. The problem  is that we’re using a real simulation,   so we don’t know or have direct control over  its exact performance characteristics. Just   like last time, we need to study the engine  and characterize its behavior. I added some   debugging features to help with this and  eliminate as much human error as possible,   which I think may have skewed some of my  results from last time. The end result was   a function that maps power demand to the  minimum required heat setting at steady   state. This establishes an upper limit for  how efficiently the engine can be operated. We can use this to calculate the theoretical  lowest cost that energy can be sold at. The   problem is that this value varies with power  demand. Just feeding the current power demand   into this model to get the lowest possible  price gives us an instantaneous value   which can fluctuate drastically. I tested many  solutions to this and ultimately decided on using   a rolling 24 hour average. While implementing  this, I realized that there was a serious flaw   in the previous model. Simply time-averaging  the power demand and then feeding that into   the market price model doesn’t give the average  market price over that period, which explains why   the game was so easy before. I won’t go into the  math behind this but I will heart any comments on   this video which give the correct mathematical  explanation for why this is wrong and how to   correct it. It’s not too hard to figure out  and it was a pretty careless error on my part. The NPCs will explain how this energy price  mechanic works as part of the player’s training,   but to make it even more clear, the metrics  are visualized with these approval bars,   showing affordability, reliability,  popularity and overall demand. Using the AI that I created earlier, I was able  to verify that the model was correct and roughly   followed the AI’s performance. The AI isn’t  perfect but likely performs better than most   humans. Now that we have a precise benchmark  to compare the player’s performance to,   we can precisely adjust the difficulty  by varying the leeway given to the player   as the city gets bigger. The game might start  with a 20% unfair advantage against the market   which goes to 0% as the city grows, effectively  making the game harder and harder. I don’t want   to leave the difficulty progression entirely up to  this though which brings us to our next feature. I won’t give away too many spoilers, but  let’s just say, the player will need to react   to problems that become progressively more  challenging. For example, a drought natural   disaster will prevent the player from being  able to fill the boiler with water. They can,   however, prepare for this by filling the  boiler beforehand enough to survive through   the disaster. This is just one of many  other challenges like throttle failure,   sudden increases in demand which  will force the player to think   creatively and break the monotony of  the predictable day/night demand cycle. These challenges will follow the game’s storyline  which I don’t want to reveal too much about,   but it’s another way of more precisely adjusting  the difficulty curve as the game progresses. There were many other minor points  of feedback from testers which I   also addressed. I’ll list just a few of the larger changes in this video: like for example I added some sound effects for power outages, repairs, gear shifts and other events.   [Powering up sound] [Powering down sound] [Clicking sounds] [Lever sounds] [Ratchet and impact wrench] Testers often missed cable snapping events and   an audio notification should help with this. [Cables snapping] I added boiler explosions because they’ve been heavily  requested since the game released and I   felt that the game didn’t have enough negative  consequences for overheating the boiler at least for the DLC. For the base game mode it doesn't really factor into it that much, but it is there by popular demand. In total, this project consists of around 86,000  lines of code. About half of that is for the game   engine and physics engine which I had already  written before. The remaining 42,000 lines is   specific to this project. Line count isn’t a  great metric to evaluate project scale, but   the point is that creating highly quality software  with attention to detail takes a lot of time and   effort. I know it looks simple but I really don’t  think development could be sped up, at least from   a single person team like myself. Using a  game engine like Unreal wouldn’t help since   this project is not game engine bottlenecked.  My game engine has performed perfectly well and   this game has been tested on 100s of thousands of  computers at this point. I have all the debugging   tools I need to make the game, it just really  does take a lot of effort if you want everything to be perfect and I really want people to feel like (for the money that they're paying) they're actually getting something good, especially considering that the base game was free. If you're interested in what my day-to-day development progress looks  like, I post detailed daily updates on Discord,   so feel free to join the Engine Simulator discord  server. The link to that is in the description. I’d like to thank all my Patreon supporters  for being very patient with my perfectionism   and making this project possible. They  will all of course be getting a free   copy of this DLC. And of course thanks  to Brilliant for sponsoring this video   and thank you for watching,  I’ll see you all next time.
Info
Channel: AngeTheGreat
Views: 174,254
Rating: undefined out of 5
Keywords:
Id: 3aFYEyCoC4A
Channel Id: undefined
Length: 17min 49sec (1069 seconds)
Published: Sat Jun 29 2024
Related Videos
Note
Please note that this website is currently a work in progress! Lots of interesting data and statistics to come.