Exposing BIAS in Game Review Scores

actors troy baker and ashley johnson returned to add so much heart honesty joy and sadness to the roles as joel and ellie respectively it's uncomfortable and outdated mechanics make you feel frustratingly trapped the artfully detailed and realized locations ellie travels through are layered with backstory it just feels cruel those clever designs are dressed with some of the most beautiful environments i've seen from the playstation 4 generation this game just feels constantly punishing for no reason while part two is a thrilling adventure it still makes time for a stunning nuanced exploration of the strength and fragility of the human spirit ultimately the game winds up being an unpleasant nostalgia trip that nobody should pack their bags for [Music] given the recent controversy about the last of us too i bet viewers assume this is a video digging on the game or its fans well i actually really don't care about the last bus i just don't really like cinematic games very much to begin with so had very little investment in the series one way or the other however i predicted something like this was going to happen the issue of the last of us controversy isn't fully the developer's fault or the fault of the players who were disappointed by its sequel either at least not completely this controversy hits something deeper than that an issue the gaming industry has had for a long time now and that is put simply that video game critics and the video game fan base at large don't share the same tastes now this isn't exclusive to video games anything from movies to food has critics that's not actual size is it maybe it is no chance no chance that's the actual size the argument for why these people exist is that a critic is supposed to have a finer palette than that of the average consumer the assumption goes that the uneducated masses don't really understand what is good and what isn't the issue is and i have statistical data to back it up in the video game medium critics aren't more discerning than the average gamer in fact it's the opposite the power the metacritic score has on a game's success cannot be understated of course there are successful games with low scores and unsuccessful games with high scores but to a video game publisher a review score is a piece of marketing and the higher that score goes the easier it is to market their game and it makes sense why video games are expensive unlike say a movie ticket which usually only costs about 10 of viewing a video game often costs 60 or more to purchase so given the increased financial investment it makes sense that while movies have a simple thumbs up or thumbs down style of review most people aren't willing to purchase a game unless they are certain it isn't just good but excellent so a non-binary style of scoring system is required but there are major issues that come from this style of system and the result of this is developers pandering to reviewers above that of their actual fan base the idea of developers changing a game to a piece to game journalists likely isn't a novel idea to anyone watching this video there are multiple stories of games getting censored or changed after a game journalist makes or tries to make a controversy over a subject what people forget is that outside of free-to-play games which tend to not get review scores most game developers have to think about game critics during nearly every step of a game's design and their specific tastes are a big reason why the current video game market is the way it is today just using my home country of america as an example video game fan bases come from all cultures and interests as well as ages you have the culturally diverse old guard the fighting game community then you have more casual gamers that just play whatever is advertised to them the most there are also people who only like a specific genre of game for example i know a lot of people that just play league of legends or dota and nothing else tastes also change based on the demographics we're looking at things such as age gender or where you live is likely going to be a good indicator of the types of games you prefer the diversity of the many tastes of players isn't reflected well in games journalism however as i will show in a bit journalism bias doesn't affect all games equally like fast-paced complex action games sorry most game journalists don't like old 90s style fps well tough luck how about 3d platformers sorry having to move vertically in 3d space is too frustrating for critics you might just think i'm cherry picking but no i'm not after going through the critic vs user scores of nearly 4 000 games there are a few clear trends i found that i would like to share which i think begin to show us the increasing divide between video game fans and the video game critics as well as starting to explain why things have gotten to where they are today the first thing i did was compare the difference between the average metacritic score a game gets versus the same game's user score and the average difference of this by year from 1996 to 2018. the results of this is a trend that as time went on review scores became more and more positive compared to what the general audience felt on a game starting in 2009 reviewers started reviewing games more positively than that of user reviews before that year from 1996 to 2008 every year showed reviewers criticizing games more harshly than that of the average player as you can see after 2009 there wasn't a single year where critics reviewed games harsher than fans this positivity bias was greatest in bigger budget games or games made by large publishers and smaller in indie games so am i implying reviewers are bought off by large corporations after all why would games made by megacorps get higher review scores well no i don't believe this is the case at least not directly you see 2009 was around the point where print media was really starting to die and online game critics were starting to become the bulk of game reviews unlike print magazine which generally shipped on a monthly basis and so had a set schedule of when a review had to be finished by with online reviewers releasing scores on the night of or even before a game's released suddenly getting a review in on time or among the first became extremely important to review sites since whoever released a review first is more likely to get views while a video game publisher can't prevent reviewers from releasing a bad review of their game after a game is already released though they have tried they sure can determine who gets to review a game before said game releases through the process of handing out early review copies to specific online publications what this results in is journalists tending to be nicer when reviewing big releases or games from large corporations because as long as they aren't too harsh on a game they'll be selected next time it's time to give out early review copies this is why early review scores of major games are almost always better than the eventual metacritic score a game eventually gets after it's finally released so that more niche game you like made by a no-name company it's not going to get the extra bump sorry and again while early review copies weren't exclusive to just online websites it became far more important once magazines started getting phased out and replaced by gaming websites this is likely why when comparing the difference between user and critic scores of top rated games as in games with a score of 90 or greater from the five-year period of 1996 to 2001 versus 2013 to 2018 we see a drastic difference in agreement during the turn of the millennium there was only a 0.5 scoring difference on average compared to a whopping average of 1.8 point difference 1.8 might not at first sound too bad but that's the difference between a game getting a 7.2 out of 10 versus a 9 out of 10. now i don't think early review copies are the only issue causing this huge difference in opinion going back to what i said about diversity the simple fact is that video game critics don't have the same player diversity or experience as many of the fans of the medium looking at available information from the writers of gaming websites it quickly becomes clear that the average video game critic is a male in their mid 20s to early 30s that live in a major city on the west coast most likely san francisco this might at first seem like a trivial bit of information but it's important when understanding the specific bias game reviewers have the fighting game community for example is much bigger in some areas of the united states than others it's also no secret that texas is the holy land for quake and doom players taking this further these sites serve the entire english-speaking world not just a single country different countries contain players with differing tastes if you're a developer that lives in say the uk for example you're going to have your game reviewed by majorly west coast americans this is just conjecture on my end but could this explain why most video games these days take place in america despite video games being developed by people from all around the world now maybe what i said before wouldn't be so bad if video game reviewers were highly experienced players with a breath of knowledge in the medium if this were the case maybe they could handle their innate biases to what games they do and don't like but the sad fact is is that nepotism not knowledge of the subject matter is often how people get hired in this industry given this it makes sense that most game critics live within a stone's throw of each other it's a group of people that for the most part happen to know each other this also explains the embarrassing lack of skill you often see from video game reviewers or their preferences for easier games remember that spongebob review at the beginning of the video here's that same reviewer not being able to figure out a quote unquote puzzle made for literal children not only could a child solve this puzzle actual ocean invertebrates solve more complex puzzles as has been shown time and time again a lot of these people aren't fit to be the vanguards of the gaming industry and this shows them the data spongebob wasn't the only platformer to get reviewed negatively by critics despite most games getting reviewed higher than fan reviews 3d platformers get nearly a .5 reduction on average in review scores compared to user assessments when we take nintendo platformers out of the picture this raises to nearly 0.6 the final and in my opinion greatest bias reviewers bring to a game review is that honestly they didn't really get a chance to experience the game the way they are meant to be played when you get a game to review you have a deadline what does this mean it means beating the game on the easiest difficulty turning on cheats or avoiding combat altogether because gosh darn it it's tuesday and i need to get this review up by tomorrow or i'm in serious trouble with my boss so think i'm just talking out of my hat reviewers admit to doing this all the time and not just for the last of us either pretty much for any big game here's one of the worst examples in my boot camp reviewers were charging through missions wearing the chicken hat which makes you invisible almost completely ignoring mother base and all the side ops in the race for the end many if not all of the reviews that are already online were written by journalists who were forced to play mgs 5 for 8 hours every day in regimented time slots while under instructions to share only the information that was deemed necessary but konami higher ups and when they don't go out and admit it often times it's found out anyway if you beat a game but experienced less than half of its content or just rush through its content do you really think you could give said game an honest review deadlines suck but that's no excuse for rushing review which will ultimately create an uninformed assessment of the product being reviewed so let's put this all together what do you get well i did notice unlikely others have too that game reviewers tend to really like certain types of games over others among the genres that showed the highest positivity bias third-person cinematic games and walking simulators showed a 0.8 and .95 bump respectively so think about it if you're a person who likely only plays video games casually and are given a strict deadline of when you have to be and review a game what game's going to be the most enjoyable to you let's say you've got to get your review out by tomorrow so you enable cheats put the game on the easiest difficulty avoid most content and just rush to beat it as fast as possible what in the game can you actually honestly review you can't give the combat or balancing a fair shake you likely aren't an experienced enough player to understand the ins and outs of the game's combat system either so what's left well maybe the graphics were nice or maybe you love the story or voice acting maybe a certain character was funny to you so the most enjoyable games are going to be the ones that deliver on these aspects even i admit if i only had a weekend to play a game i probably wouldn't pick the most technical fighter or some 200 iq rts by the time my time is up i wouldn't have even understood the most basic gameplay elements yet and i definitely wouldn't have enjoyed the experience much i probably would have enjoyed a nice story focused game more instead you can see this in the reviews for the last of us too despite the fact that you spend around 90 percent of the game in combat or sneaking or managing resources the story animations and characters make up the vast majority of the content of these reviews why because they're the only parts the critic got a decent taste of everything else is the reviewer playing the game of guess on very little actual experience what you think the things you're reviewing is like if you like the last of us too or any other games reviewers also like i'm not trying to insult you i'm just showing that these are the types of games critics by the very nature of the review system are going to gravitate to and because of this these are also the types of games big name developers are also going to gravitate to as well because they know it tends to lead to better reviews this is almost certainly why the last of us 2 has the option to enable a button that literally makes your character invisible to enemies because they know this is something that a reviewer almost certainly wants and makes the playtime of the reviewer more enjoyable this is also likely why the new crash game is getting an unlimited lives mode in its new installment the sad fact is is that because review scores mean so much for the sales of the game it simply becomes riskier as a big time publisher to make the games you know reviewers aren't going to like it stifles the diversity and creativity of the games publishers are willing to fund because they know critics have a narrow range of games they are willing to like this hurts gaming because it means a wide array of diverse tastes and people that play games aren't being properly catered to it's well known that game journalists tend to really care about diversity and representation or at the very least they say they do but because of the bias that comes from game critics being such a homogeneous group themselves many of the cultures that reside within gaming aren't getting the representation they really need and that's a sense that the outside world cares about your hobby in the same way you do think for example of the fighting game community a community made up in large part by poorer communities that started as gatherings around arcades how often do gaming critics show an appreciation for the deep and complex mechanics this genre has to show or what of eastern european games it's no secret that western games media has a bias against what many have considered euro jank often critics see these games as frustratingly unfunctional and up to snorefests but to their fans and the cultures that create them their unique and creative experiences made resourcefully and out of love and don't get me started on the cultural atomic bomb that is game critics and their disdain for the expression of japanese devs reviewers need to see genres on their own terms rather than through the lens they often get both from their background and the short amount of time they get to assess a game a lot of these reviewers are predisposed to not liking certain food items and i don't think that's right i mean they're they're the reviewer after all they're supposed to have the professional palette and be able to tell me if a tomato is crisp or not crisp they can just throw it off and be like i don't review tomatoes oh oh what are you [ __ ] gay so what's my solution to this should only gaming experts like me get review copies no i don't think so there are some types of games i simply don't care for but that doesn't mean i think these games deserve a low score because of it the major issue of the metacritic is that the best games aren't going to be the ones that are liked by everyone but instead the ones loved by the specific group of people they pander to a piece of art that speaks to you specifically is always going to be the piece of art that we personally like the most as i said earlier i don't like cinematic games does that mean all cinematic games deserve low scores of course not rather than traditional metacritic reviews we should transition to more accurate and personal review systems traditional game reviews are kind of obsolete in the year 2020. most people don't even read gaming site reviews they just checked the score sites like youtube allow people to broadcast detailed opinions of games and unlike random ign reviews you likely know if said content creator has taste similar to yours but an even better review system in my opinion is that of the user review system not necessarily the user review system that metacritic has but the system sites like steam have steam used to display the metacritic score on a game store page but later replaced it with user scores but more importantly these are from users that have already purchased and played the game this means that the reviewers have already been self-selected for being into the specific type of game or genre the product is from and likely have more expertise in it not only that but unlike game journal scores steam review scores can update continuously which better reflect the state of a game as time goes on and gives players as much time as they want before reviewing a game sure this might result in games that the general populace wouldn't be into getting a score far higher than what reviewers would give it but let's be honest if you're looking at the user review scores of a japanese dating sim it's because you already have an interest in japanese dating sims this has worked wonders for the games that the major gaming websites aren't willing to touch or games that are in early access do i think steam's user review system is perfect no but i do think it's a step in the right direction and definitely a better option than what the current dogma is to close i feel there's a lot of information that can come from this data set i've been collecting and going through as time goes on i hope to update it and make a more comprehensive list showcasing reviewer buys as it applies to genres publishers and developers if nothing else maybe it could help players determine a better assessment of the game's quality than just what gaming review sites decide [Music] you

Channel: WelfareWalrus

Views: 723,496

Keywords: spongebob squarepants, the last of us 2, the last of us 2 review, social justice warriors, SJW, spongebob, THQ, video game reviews, spongebob squarepants battle for bikini bottom rehydrated

Id: YGfEf8-SNPQ

Length: 19min 25sec (1165 seconds)

Published: Sun Jul 26 2020