Know Your Market: Making Indie Games That Sell

[Music] hello everyone we're back here after lunch welcome again to the independent game summit my name is Yann I'm one of the advisory board members of the independent Summit and I have the pleasure to introduce you to Erik Johnson who's going to be talking about knowing your market let's give him a big round of applause real quick make sure your phones are on silent don't jiggle your badge too much and make sure you fill out the evaluation thank you thank you so yes knowing your market making games that sell apparently this is a popular topic who knew people would be you know interested in paying their rent I want to start by telling you a little bit about how I got into this topic I'm a developer I worked on a game called life goes on and this is a puzzle platformer where you have to sacrifice your Knights and use their dead bodies to solve puzzles and so we released this on Steam in 2014 and then in 2016 we did a major content expansion and a release on the PlayStation 4 and during development we had a lot of indicators that maybe we had a hit on our hands we got media coverage we want awards we we got on Steam back when that was more difficult thing to do but if you've heard this sort of story before you know where this is going neither the initial neither the initial release nor the 2016 follow-up ever really lived up to our expectations and in trying to understand the sales performance or the lack thereof I've developed an interest in market analysis that borders on the obsessive and so I've done lots of research to try to understand this stuff the factors and what drives game sales and I've learned a lot of things that I wish I had known before we started on life goes on and because of all of this I have developed a shocking amount of enthusiasm for data analysis and statistics so I'm not saying that you should be looking at everything from the point of view of making money and if you're making games to get rich you're probably in the wrong industry and there's also lots of non-commercial reasons to make games but if you're planning on selling a game you have to at least look at it from a business perspective to some extent and be aware of this stuff so what causes games to sell I want to present the argument that it's not just luck it's not just how good the game is it's not just your connections and who you know it's not just your ability to promote and advertise and market your game these are all super super valuable factors but they don't account for market fit which i think is also something that is super super important and so a lot of what I'm talking about here is trying to figure out what people want and particularly if enough people want what you're making to justify your making it and this is something that we we never really thought about it from this perspective when we were working on life goes on we knew we had this sort of clever idea we knew it was really cool work to work with from a game design perspective and we knew that people thought it was neat but we never really like gave a lot of thought towards whether this was something that people were gonna be really excited to actually buy to give you some context I'm talking about indie games on scene and selling them and what I'm gonna say is going to get less applicable the further away from that you get so free to play is different console the consoles have a different marketplace mobile is even more different and factors like Steam Direct and the growth of the Chinese audience on Steam have really changed things in the last year and things might be very different a year from now so let's start with tools and techniques how do you keep on top of this stuff how do you know what has sold and what hasn't turns out there's loads of information readily available and if you're interested in this stuff at all you probably already know all about steam spy which is a service that scrapes steam and creates a statistical estimate of how many people have a game in their Steam library Steam Spy is an absolutely fantastic resource super super useful but you have to be aware that owners on Steam Spy are not the same thing as sales there's lots of different ways a game can get into a users library you've got bundles giveaways free weekends all sorts of stuff like this that can inflate the owner account and also that accuracy is low for new games and games that don't have very many owners something that turns out to be surprisingly useful is the review system on the steam store page itself particularly it shows you account of reviews by steam purchasers versus other sources and this gives you a lot of insight into how well a game is actually sold on Steam at least there's a technique called the Box later method where you can estimate the number of people who bought a game on Steam and played it by multiplying the reviews by 50 though some games mush might have a much higher lower multiplier than that so this is a pretty rough estimate and in addition to steams Buy and the review counts you can get discount and bundle history from is there any deal you can get concurrent player history from Steam charts you can get regional price differences and loads of package details from Steam dB okay let me talk about what I've learned from all of my obsessing have you heard this before if your game didn't sell well you just need to make a better game next time quality certainly does matter but to think that game quality is the only factor or even the dominant factor in in the game's success is the fallacy of believing that the world is fair and just and this isn't this isn't always the case in the context of life goes on I want to say that it is far from a perfect game there are many things that I wish we had done differently there are a hundred things that I wish I could change and improve on right now but we did have some external indications that people might like it and there was something that really came as a surprise to us and that was that when we did the 2016 content expansion and PlayStation release we significantly improved the quality level of game but we didn't see sort of a corresponding increase in sales or revenue and this contradiction was one of the big factors motivating me to try to better understand the market ok time to get into the numbers I hope you like stats i've done some rough revenue estimates for every non free-to-play game on Steam this is based on the steam spy data plus some additional data I've collected from Steam store pages these this estimate is rough but I think for a broad sort of trend analysis it's very useful something to note this doesn't include DLC or in-app purchases this is estimating initial purchase revenue only the revenue is shown on the y-axis here on this chart and this is shown on a log scale so points at the top of the graph have several orders of magnitude more revenue than the points at the bottom and this chart in particular is comparing this revenue to steam user score so I've commute I've computed Spearman's Rho which is a correlation measure that basically measures if X goes up does why I also go up and a 1.0 would indicate perfect correspondence whereas a zero would say that we're looking at random noise with a point three one this means that we're sort of looking at medium to low correlation so but we do have some correlation between user score and revenue so hurray there is you know some justice in the world but this isn't a super strong correlation but maybe we shouldn't be surprised by that GTA 5 has a 68% positive user score pub G is only a 56 percent positive these are incredibly popular games but they have these somewhat unhappy player bases and so while I'm very proud of life goes on is 96% positive I really can't argue against pub G success let's look at a similar data set run related to Metacritic score this time we see a stronger correlation with a row of 0.44 which is getting into sort of medium to strong territory and so we are seeing evidence that game quality does matter both with player and critic reviews so if you do make a better game you will probably trend towards making more revenue the next thing i want to talk about is genre this is sort of the far out view of how viable different types of games are or should you make a puzzle platformer in 2018 the answer is No so once upon a time games like Fez braid and limbo sort of defined what indie success was but the market has dramatically shifted since then there have been really very few breakthrough puzzle platformer successes since about 2014 inside and never alone are a couple of notable examples that I'm aware of and these both have really incredible visuals excellent marketing stories and had I think fairly large production budgets I did some johner analysis based on that steam tag set this is super super noisy data but I did some filtering and cleanup and I've taken the taking the median estimated revenue for puzzle platformers on Steam excuse me and I've defined this as the metric standard unit of my disappointment with the games marketplace so we're comparing it to other other tags and genres here and puzzle platformers are near the bottom action RPG has a median revenue 24 times the median puzzle platformer there's another other indie favorites here roguelike and Metroidvania and they're only at three and a half times the revenue of puzzle platformers so I think we can see evidence that different genres have got dramatically different market responses so sometimes we get the impression that visuals might be less important for indie games we have you know pixel art low-poly styles and that sort of thing and it's not all about the cutting edge tech and the photorealistic visuals like you get in the triple-a space but I think visuals still matter a ton when someone's looking at a store page and deciding what to buy or what not to buy I think there's no coincidence that my two examples of recent successful puzzle platformers both had incredible visuals and we got a lot of positive comments about how life goes on looks but we also got some criticisms about it looking generic and so I think that visual quality isn't the only factor there's also something to be said for distinctiveness so are you showing something players unique or new there's a master's thesis done by Michael trainee where he developed a machine learning model to predict success from Steam store pages and he found that a screenshots average saturation and distinct colors was one of the top factors in predicting success unfortunately I've read his thesis and it doesn't say either way while they're having more colors and more saturation is good or bad so I guess more research is needed there an idea that I've encountered sort of from other indie developers is that the Steam audience wants dark and gritty games so do we have any evidence for this valve has shared the top 100 games of 2017 by revenue on Steam and when you look at this list I would say that these games are pretty dark and gritty there are some obvious exceptions to this rocket League and cuphead are bright and cheerful at least in appearance but I think they also have a seriousness in their tone in some ways they aren't goofy or silly games and I think this is a factor that made it more difficult to sell life goes on we have this really dark concept with the sacrificing your characters but we approached it in kind of a goofy silly way let's look at the data here we're here we see ESRB recommended age and again this is a log scale regiment a log scale revenue estimate and here we've got a row of 0.45 so this is about the same correlation that we saw with Metacritic score so getting a higher Metacritic score seems to have about as much predictive power for success as getting a higher ESRB rating and so this is comparing median revenue by ESRB rating and this time we're looking at a linear scale because it's multiples of the median and you can see that the median mature game has 36 times the revenue of is the median a game on steam so rocket League planet coaster Kerbal space grant program these are all rated games and they are some of the top selling games on Steam so it is possible to go against this trend but sort of the typical a rated game has comparatively little revenue and when you consider that you know steam is an online store mostly driven by credit card purchases it's a lot easier for adults to buy games than it is for kids I wanted to look at this by tags and again comparing two puzzle platformers and so family-friendly has a medium revenue of 0.8 4 so if you think puzzle platformers are in a tough spot family-friendly has it even worse crime on the other hand is at 23 times puzzle platformer somewhat less directly related tags like war and military are even higher we can see colorful at 1.3 vs. dark at 5.9 and if you need a more direct comparison than that I'm not sure I have one I'm sorry I also want to note again to remember that this research is focused on steam the story for family-friendly games is probably very different on a Nintendo switch so a really big factor in commercial success is how long people play a game for is a game sticky is there a lot of content is it highly replayable as their multiplayer steam systems reward long play times for example I can't tell you how many times I've seen steam pop up a window telling me my friend is playing spelunky and I think maybe I should play spelunky if we look at a verge play times for some classics like braid Fez and limbo and you compare these two more recent hits like star D Valley or more old factorio people are playing these newer games or 10 to 20 times as long as they played those older games um sadly life goes on as play time is with the first groups not the seconds okay let's look at average play times plotted against revenue this time our X scale which is at the average play time in minutes is also on a log scale so play times are increasing exponentially as we move to the right with the row of 0.55 this indicates a super strong correlation this is the strongest correspondence that we've seen so if you're looking at this chart you are probably wondering what's up with that cluster the strange vertical line that we see and this sits at the 3:00 hour and 20 minute mark and these games all have lots of players but it looks like relatively few sales directly on Steam so this is the same data but this time the games in red are games that have steam trading cards so this shows evidence for the games that are created for exploiting the steam trading card system and this was a problem that Valve cracked down on about a year ago so this is a super neat thing to see in the data but don't let that vertical blob distract you from the really strong trend line going sort of up into the right where as revenue goes up sorry as play times go up revenue goes up very directly this row value is really significant and that's somewhat of a scary trend you know there's a lot of new games with it ever evermore increasing games entering the market well at the same time the most successful games are holding on to their plane holding on to their players for a very long time so we know that YouTube and twitch are huge for marketing and for indie visibility but not all games benefit equally from streaming when a player watches your game watches someone else play you're damned do they say oh I have to play that for myself or they come away from it thinking I've seen enough I've seen what this game is I don't need to play it now and I think that short narrative heavy games and puzzle games that have consumable content might actually be hurt by streamers and Let's Plays I can share some examples PewDiePie is the number one YouTube channel and he played the extremely awesome crypt of the necrodancer and this video has about 2.4 million views jacksepticeye played life goes on jack doesn't have quite as many followers and his video sifted about 1.8 million views but I think you know these are sort of ballpark comparable Ryan Clark Krypton good answers developer shared that the PewDiePie video resulted in direct revenue for his game of about sixty thousand dollars comparatively Jack's video for life goes on we saw a revenue bump of about five hundred dollars another example comes from x-gen Studios youtuber Matt che did six episodes on their game super motherlode X Jen estimates this drove about ten to fifteen thousand dollars in revenue for them the same youtuber played their new point-and-click adventure game the low road they saw as zero results so crypt of the necrodancer and super mother lode both feature procedural generation they're both highly replayable games whereas life goes on and the low road are far less were playable so next thing I want to talk about is community can community get involved in your game do you have level editors you have mod support you have leaderboards is there multiplayer so that players can drag their friends into the game to play with them life goes on has none of these things you may be noticing a trend here looking at this question in the tag data we see some of the top tags that I found moddable sits at almost 140 times the median revenue of puzzle platformers this is a bit of an outlier but I think it is indicative of the trend level editor doesn't look great in this chart but it's still double the median puzzle-platformer you also have to be careful though at point 4 3 4 player local is one of the lowest revenue tags I looked at on Steam game designers seem to really love to make for player local games but it seems like they're really hard to sell on Steam so the factors that I've just reviewed aren't the only contributors to market fit these are just the ones that I could show you good data for and I'm not saying that it's impossible to buck these trends if you want to make your own short colorful family-friendly puzzle platformer please feel free but you will be fighting an uphill battle and there are lots of lots of super successful games that don't match these trends so if you're going to try to defy them though be aware of what you're doing and approach it thoughtfully so you might think that I'm just looking at the top sellers list and just chasing the hits and chasing the big bucks and that's not what indie games are about the indie game should be targeting a niche finding finding an audience that really loves what you're doing and going after that audience and so yes finding a niche is absolutely important but buys indie games on Steam turns out to be a surprisingly narrow niche to begin with there's an article titled your target audience doesn't exist by a cervical can the creator of schemes by where he SME it's only about 1% of steams user base regularly buys new games every year so you have to make sure your niche is viable and if you can tell me that there's this target audience out there that really wants more puzzle platformers and you know how to reach them please please come talk to me I have a game I would love to sell them to give an example of a narrowly targeted but viable niche let's consider Zack Tronics games these are basically engineering challenges for programmers which seems about as nice as it gets they recently launched their new game opus Magnum and I saw it at the top of the global top sellers list on Steam that's a good sign that a niche is viable and it turns out that the medium programming tag game is almost as 11 times as successful as our standard puzzle platformer so yeah programming seems like a little bit more viable despite its very initial peel I'm also not saying that you should you should not be looking at the top selling games and trying to clone them again if you're just making games to make money go do something else it will be easier most designers have a lot of different ideas so when you're choosing between your ideas evaluate them for market fit look at the factors that correlate with success and make something unique based on that for example are you targeting a genre that's in demand or if you're breaking new ground does your design at least align with the trends for success like remember the strength of correlation between success and play times will people want to play your game for a long time what would take to make long playtime's part of your design also can you find games they're doing similar things to what you're doing can you explain the successes can you explain the failures do you think these games that you're looking at had comparable marketing and development budgets to yours to give you a bit of an epilogue for life goes on we've continued to put a lot of hard work into the game we've continued to support it localization has been really big for us particularly in Asia we got kind of lucky we didn't have a lot of text and so localization turned out to be cheap we'll probably never really get our investment back out of it but we've done much better than we expected - based on our launch even though it's been a bit of an uphill battle if you want to learn more about this Ryan Clark - the crypt of the necrodancer developer used to run a regular twitch stream where he would do market analysis for a couple of hours unfortunately he's no longer regularly streaming but you can find back episodes on YouTube and they are definitely worth your time Lars de sete the developer of defenders quest has a project called steam profit where it looks at upcoming steam releases and tries to pick which games will be successful which is a very useful approach to try to test your understanding of the marketplace and you can also watch totalbiscuit schoo optional podcasts this features popular streamers and a regular segment is where they look at upcoming games on Steam and you can just sort of see their reaction to their trailers their concepts their screenshots and see what they think of them it's gonna be very informative so thank you for your time if you want to talk about this stuff hit me up on Twitter Eric Johnson and Eric with Kay if I if you want help if you want to talk if you want just to nerd out about stats I would love to please please reach out to me and I think we have some time for some questions yeah sounds like we have time for two or three questions if you could line up at the mics on either side and you look like you're there first so please go ahead yeah quick question hi I'm Dave from no one cares um you mentioned that replayability was an indicator of revenue yes possibly correlated but then you also mentioned that roguelike is like way down as far as tag goes which is interesting you have any sort of insight on that um roguelike isn't popular with Triple A and roguelike is a pretty broadly applied tag as well and so I think I would say that it would be a big driver there the two categories which seem to rank the lowest and the data you collected our puzzle platformers and rated e for everybody could that possibly just be a result of how many games fit those categories I puzzle platformer or something you could bang out in an hour whereas a action RPG or amateur game is not likely you know most indie games re for everyone right yeah so I would say that there is certainly an element of of not just do people want these things but also how saturated are these areas how many people are already filling that need and how easy is it it's you yeah there are people who love puzzle platformers but they are drowning in selection versus you know the people who want the top-end triple a crime action you know it's harder to meet their needs so I think that is certainly a big factor No hi do you have any tools or tips on how to counter survivorship bias and find the games are not successful how do you counter survivorship bias and find the games that are successful so I mentioned steam profit and playing steam profit is the process of looking at basically every new game that is coming out on Steam every week and you will very very very quickly find innumerable examples of games that don't succeed that don't generate revenue and I found looking at all of these games that there would be many games that I thought you know maybe they won't be the next hugest hit but certainly they will see some reasonable amount of revenue and it turns out that you know the their Mahr their revenue is within margin of error of zero so yeah I would say just looking at the game store will very very very rapidly dissuade you of the survivorship bias one more I think sure do you have have you have you done and are they existing somewhere and if they are will you you know post them somewhere have you done time and money adjusted comparisons like take into account how you know development costs and price point for selling and yeah so there's no I mean there's no public data set on development budgets you might be able to do some interesting and data based on like you say price point you know higher price games probably we're more expensive to develop something that you will certainly see as a trend on Steam is that games below $10 very rarely have any sort of significant revenue yeah okay thanks thank you [Applause]

Channel: GDC

Views: 138,399

Rating: 4.9414172 out of 5

Keywords: gdc, talk, panel, game, games, gaming, development, hd, design

Id: uy0Dfr-mnUY

Length: 27min 2sec (1622 seconds)

Published: Thu Aug 23 2018

Reddit Comments

[u/justkevin]

Some really good information in here. A few take away points for those who don't have time to watch:

• Quality, as measured by either Steam rating or Metacritic is correlated with estimate revenue, but the correlation isn't as strong as some developers might expect.
• Playtime showed a much stronger correlation with revenue than either Steam rating or Metacritic.

[u/His2W]

One more case of a bad game name standing in the way of success.

Life Goes On sounds like a terrible self help book, and the expansion Done To Death is literally calling itself out for beating a dead horse.

• Flappy bird
• Cup Head
• PUBG (commonly pronounced "Pub G")
• FortNite
• MineCraft
• Battlefield
• The Witcher
• Counter Strike
• Rocket League
• Ghost Recon

All of these wildly successful games have names that are verbally expressed in 2 words, while being both descriptive and unique.

When people are scrolling through steam, or when friends are jumping into a game, they can see your game's name and one picture. Make it count.

[u/Orange_Hour]

Now I want to play The Low Road and Crypt of the Necrodancer.

[u/themoregames]

I am not affiliated.

Video description:

In this 2018 GDC talk, Infinite Monkeys Entertainment's Erik Johnson @erikejohnson analyzes trends in the Steam marketplace and explores how his game, 'Life Goes On', failed to match the market.

[u/ARN64]

I don't feel like he arrives at the correct conclusions with that data. Looking at the revenue by genre especially. Of course genres that include AAA games and no indies are gonna do better, and preferred indie genres untouched by the giants do worse. It's no breaking news that AAA make more revenue than indie games