Game Theory: Did Dream FAKE His Speedrun? A Final Analysis.

oh hey matt oh hey you it's me robert we sat next to each other for like three semesters in college yeah richard robert robert yep how have you been what are you up to these days done with college i presume uh technically no since i'm a professor now teacher humor anyway what are you up to well uh my life definitely took an unexpected turn i'm uh i'm actually a professional youtuber now oh god don't tell me you're setting couches on fire in your backyard no no those people are the worst no i have a show where i actually talk about video games well someone got lucky so what do you do all day it's about what you'd expect play video games make fun of video games read 29 page papers about binomial distributions in a series of minecraft speed runs oh wait really no no i'm just kidding only part of the paper deals with binomial distribution the other sections describe modular arithmetic and the properties of linear congruential generators hello internet welcome to game theory the show that speeds through scientific topics for the benefit of those with short attention spans yet still manages to be longer than a minecraft speedrun it's true as of november 3rd the world record for speedrunning minecraft 1.16 is 14 minutes 39 seconds and 520 milliseconds milliseconds that is how granular we're getting here people and that should show you that minecraft speedruns are a serious business which is probably why a minecraft speedrunning controversy set the internet on fire all throughout december because what better way to capstone a year full of crazy headlines than people taking the world's most relaxing game and turning it into a war zone dream smp excluded of course that thing is always a war zone in the best way possible but let's just say when the minecraft speedrunning team puts together a 29-page paper claiming that dream cheated while speedrunning minecraft and dream in turn hires an anonymous astrophysicist to offer a 19-page rebuttal all of it dealing with sampling bias and binomial distribution and computer modeling and statistics it's kind of a big deal when people are busting out latex you know it's serious i'm glad that maybe one of you got that reference for everyone else i'm not talking about like latex plastic i'm talking about latex a document preparation software that allows academics to get all those weird mathematical symbols into their docs because if you've ever tried to use word it don't work so why then am i here well honestly i wasn't sure i wanted to cover this i don't want to insert myself needlessly in drama also this is future matpat dropping in we're finalizing this episode now to release in about a week but dream just issued his final word on the matter on twitter i don't want this video to be seen as beating a dead horse or reopening old wounds just took me a long time to work on the math alright and these videos take forever to edit and while i could have just shelved the episode and taken the l on this one now that the discussion is officially over i think us being late to the party is actually a good thing since there are a lot of interesting takeaways in all of this which i'll get to later in the script anyway just wanted to drop in let you know that i recognize this is coming way late and i think this video especially the conclusion is still very valid now back to the episode as scripted personally all of my experiences with dream have been very positive dream has been fantastic to work with on both our collab video and as a guest in our charity live stream last year plus i really admire the work he and the crew are doing on the dream smp but it's not every day that people are excited to talk about statistics and a lot of these discussions have been very technical and complex in a way that's pretty difficult to understand if you're not comfortable with math and honestly that's the foundational reason i created game theory this show was originally designed to be a place where i could break down a lot of big and seemingly complex topics and fun more digestible ways these days it's mostly big complex topics like the fnaf timeline but also things like physics and history and yes even math do you want to know more about the games you love without having to put in any work of your own well now you can play and learn we have created game theory gaming's tangential learning experience oh the cringe it is so bad it's so bad it's blinding me anyway all joking aside i was lucky enough to have teachers that got me excited to learn who were good at explaining things but i know other people aren't so lucky so i always wanted this channel to be that place for whoever found it to explain why these sorts of topics are actually a lot more fascinating than you might think and this is absolutely one of those situations it's also one of those situations where i think we all might be missing the bigger point in question but i'm gonna get to that one a bit later understand we all okay here i'm not trying to attack anyone this is purely meant for education there's a lot of misinformation circulating here and as that cringy old promo of game theory said knowledge is power oh i i played the cringy promo again what was i thinking so i think it's worth starting from the beginning dream is a minecraft speedrunner that's a bit reductive since these days he does a lot more than just speedrunning but the first time he appeared here on game theory he introduced himself by saying i'm dream i'm a minecraft speed owner so yeah i'm gonna go with that anyway dream at various points in the history of minecraft speedrunning has held world records back in march 2020 he completed a run of minecraft 1.14 in 32 minutes and 21 seconds which at the time was a world record by 16 seconds it was beaten several days later so dream came back to reclaim the crown with a time of 26 minutes and 44 seconds in june he once again claimed the title of world record holder with a run of minecraft 1.15 in 22 minutes 4 seconds [Applause] in fact to this day dream's run is actually the fastest 1.15 run on record though in an interesting twist he's not actually number one on the leaderboard since there are five people that have actually completed minecraft 1.14 with faster times why would that matter well 1.14 and 1.15 are similar enough that they're grouped together as a single category though fun fact 1.15 is actually considered to be the harder for speedrunning due to the removal of a trick that lets you infinitely restock villagers and even amid the current scandal dreams 1.15 remains on the leaderboard without any disputes over its authenticity or validity and this leads us to the controversy which didn't start until last october when dreams started playing minecraft 1.16 that's the one with the nether update an update that was big enough to core mechanics to get it listed as an entirely separate category of speedrun in the best of these runs dream was able to complete the game with an impressive time of 19 minutes and 24 seconds putting him at fourth place on leaderboard however it also involved him getting very lucky in fact during his october speed running live streams dream got so lucky that it attracted the attention of several people who started to question whether his live streams were proof of more than just good luck had dream actually modified the game now a lot of people might be asking at this point why scrutinize dream in particular after all while his 19-minute 24 second run was certainly impressive and very lucky it's far from the fastest minecraft speedrun on the books in fact he's several minutes behind the 15 minute 12 second run that was the previous world record at the time if you're looking for suspiciously good luck why wouldn't you focus on the fastest times well as the minecraft speedrunning team explained in their 29-page report the problem wasn't with any one of dream speedruns in particular rather it was abnormally high drop rates of key items across multiple consecutive live streams what kinds of key items so one of the biggest bottlenecks in any minecraft speedrun is crafting eyes of ender to use the end portal to reach the end which requires both ender pearl and blaze powder to craft hence the importance of these two particular items and dream across these six live streams in question was getting them at suspiciously high rates specifically the investigation team noted that quote 42 of the 262 piglen barters performed throughout these streams yielded ender pearls and 211 of the 305 killed blazes dropped blaze rods this means that for this specific set of speed runs dream had an ender pearl barter success rate of 16 when the actual rate is actually much closer to 4.7 percent and a blaze rod drop rate of 69.2 percent when actual drop rate is 50. that might not seem like a big deal after all a 16 drop rate is obviously higher than 4.7 but are the numbers really that different to merit a cheating investigation i mean these are random events after all sometimes you just get lucky well to illustrate this pop quiz time which is more likely flipping a coin 10 times and getting heads at least seven times or flipping a coin a hundred times and getting heads at least 60 times pause the video if you need to think about it scroll down to the comment section to post your answer and while you're down there consider subscribing it's that red button right there under the video it's really satisfying to click and i cover a lot of minecraft lore fact i've probably put more lore into this game than mojang has anyway you all ready with your answer great here it is your chance of flipping a coin 10 times and getting heads seven or more times is 17 your chance of flipping a coin a hundred times and getting heads 60 or more times is 2.84 yeah it is way way less likely now that result might seem counterintuitive at first but as your data set gets larger and larger the more likely it is to reflect the true odds which when you're talking about coin flips is 50 50. you want an even more extreme example your chances of flipping a coin ten times and getting heads eight or more times five point four seven percent are actually significantly higher than your chances of flipping that same coin 10 000 times and getting heads 51 or more all you're talking is 5 100 times 100 over half the probability of that one happening is 2.32 at a glance you might say hey 51 odds that's only one percent away from 50 why would it be so improbable but this is a phenomenon that's described in statistics as regression towards the mean or sometimes reversion to mediocrity which is how i like to describe most of the content on the youtube trending tab oh get dunked on youtube trending tab watch this episode is gonna wind up on the youtube trending tab so how does this all apply to minecraft well for example you only have a 4.7 chance of getting an ender pearl from a piglen barter you might get lucky and get an ender pearl after say four attempts leaving you with a 25 drop rate but the more times you try the closer and closer you should be getting to the true drop rate of 4.7 percent and when you're talking about over 20 hours of live streams and hundreds of barter attempts you'd expect the number to be pretty much closer to that expected value which and i'm just gonna call this fact out means that if cheating was involved in this situation it was the fact that dream showed his work that got him flagged it wasn't the individual record setting run itself that set off the alarms it was that plus all the other 20 plus hours of run attempts aired online around it an individual run wouldn't have had enough drops to show any sort of subtle tampering or bias in other words if there was cheating involved and this was just an individual run that happened offline completely removed from any context and then was submitted for consideration none of this would have happened there'd be no way to verify the validity of the run unless of course i'm missing something here just thought it was an interesting loophole that this whole situation brought up that probably should be closed up now let's think about an extreme example of luck winning the lottery the chance of you buying a lottery ticket and winning the mega millions jackpot are roughly 1 in 300 million those are some incredibly low odds but obviously some people do get lucky enough to win the lottery i want a lottery that's great and it wouldn't make sense to track down that lottery winner and question that person about cheating on the other hand if the same person were to win the jackpot multiple times well that's when fraud investigators are going to start getting involved this example illustrates one of dream's early responses to being singled out for the review that he just happened to win the lottery he got extremely lucky and by looking only at those instances of extreme luck the review team was seeing skewed numbers quote from his response rather than taking a truly random sample the sample starts when the luck seems off and the sample ends as soon as i hit a personal best goal of 19 minutes although this can be touted as a random set of data it is most definitely not end quote this is an issue known as sampling bias referring to the fact that dream speed running streams weren't randomly chosen for review people spent hours pouring over the data specifically because someone noticed that it seemed to be getting luckier than normal and obviously if you scrutinize someone specifically because they seem to be getting lucky when you look at the results well they're gonna seem luckier than normal so to combat bias what everyone has been calculating and arguing about in their massive research papers are the odds of drops this lucky happening not specifically to dream but to literally any minecraft player in existence you know the old saying about how if a monkey is hitting keys at random on a typewriter for an infinite amount of time it'll eventually write the complete works of shakespeare it's based on the idea that something super improbable will almost surely happen given enough time and enough attempts and it's an actual theory a statistical theory called the infinite monkey theorem anyway same thing applies here dream's luck was extreme but how extreme was it both the minecraft speedrunning team and the anonymous astrophysicists that dream hired to write a rebuttal have submitted their numbers but people seem to be getting those numbers confused to directly quote from the original report the minecraft speedrunning team presented a number of 1 in 7.5 trillion well that number is them basically saying hey let's assume that there are a thousand minecraft speedrunner monkeys at a thousand keyboards there's still only a one in 7.5 trillion chance of recreating the luck that we saw on dream streams conversely dreams the odds to be 1 in 100 million the number 1 in 10 million gets thrown around a lot from dreams rebuttal but that's actually a different number comparing the two is actually like comparing apples to oranges if you're looking for the best comparison it's the speedrunning teams one in 7.5 trillion versus the astrophysicists 1 in 100 million but still that leaves us with a massive difference in calculations why is that well as much as i'd like it to be it isn't something that has one big answer but instead a lot of smaller answers which you know is why these arguments are happening in 20-plus page documents it includes things like differences in the application of the stopping criterion different estimates for the total number of live streams other differences to account for sampling bias and differences in accounting for p hacking bias we've already covered the sampling bias discussion in broad strokes and we're about to touch on that last one p hacking but as you can see it's a lot much more than i actually want to get into here if you want to get into the nitty-gritty of it i highly recommend mathemaniacs video breaking down all the arguments point by point from a mathematical deeply statistical perspective good job mathemaniac keep doing the good work but all of this i just hope illustrates the wider point here that this is a complex topic with a lot of different questions to consider all using numbers that at the very best are rough estimates that are very flexible which leads us to the last point i want to talk about p hacking bias p hacking is sometimes also referred to as data dredging it's basically the practice of looking for patterns and selectively presenting data to come to a conclusion it's practically the principle that this channel was founded on for instance look at our mario ismental series of episodes from way way back in the channel's history in these episodes i looked through hundreds of mario games to present you the viewer with selected data points to lead you to the conclusion that yeah mario is indeed a sociopath and to get there i ignore decades of him acting heroically and selflessly to save the mushroom kingdom though to be honest this is actually to this day one of the theories that i still believe i just can't ignore the fact that mario whips dk in his early games or he grinds luigi's foot when he wins in the tennis match anyway this process is sometimes described as post-dicting you get it it's the opposite of predicting instead of coming up with a hypothesis and then doing an experiment that confirms the hypothesis you start with a pile of data and then you use the data to come up with a hypothesis to explain it after the fact so how does this apply to our minecraft speedrunning controversy well in any game of minecraft there are lots of sources of randomness but thus far we've only been looking at two pearl trades and blaze rod drops but what about other sources of luck like the frequency of blaze spawns or the number of piglets that you're actually able to barter with these are all random too just because two variables happen to be higher than you'd expect how do we know that we're not just selectively choosing the data to come to the conclusion that cheating happened well it all depends on the number of variables if there are only ten factors to consider and two are unusual well then yeah it's hard to deny that something suspicious is going on but if there are say a thousand possible variables well then it's very easy to cherry-pick two of them and come up with a false correlation in fact there's a really fun website dedicated to exactly this for instance did you know that the declining divorce rates in maine are actually tied to the decline in people eating margarine it's true there's a 99 correlation between the data points the chart doesn't lie but obviously our conclusion does divorce and margarine are completely unrelated unless of course marriages are suffering due to poor butter substitute choices but when you're looking at a huge pool of data you can come to practically any conclusion and that's the other major point of contention between dream and the speedrunning team how many sources of rng are there that could be manipulated as possible sources of cheating how large is this pool of data the minecraft speedrunning team made a quote-unquote generous estimate of 10 different factors their words not mine dream claimed it was closer to 37. and honestly the frustrating thing about this is that there's really no clear-cut answer because it's not just about how many variables there are but also how measurable those variables are i bring all of this up one to explain it a bit more clearly and two to point out that some of these calculations and differences between the numbers are based on numbers coming from assumptions that can feel kind of arbitrary and that's where there's room for debate here but all of this for as great as it is to talk about statistics i think is missing the point yes it is essential to uphold integrity on speedrun boards a hundred percent absolutely super important but are we sure that this is a game that should have a speedrun board in the first place for all the back and forth on this topic one thing that's not in dispute here is the dependence of luck in minecraft 1.16 speedrunning well the nether update was much beloved by many minecraft players outside of the speedrunning community myself included given the huge amount of theory fodder that it added to the game the speedrunning community hasn't exactly welcomed it with open arms and it's led to a bit of a divide with minecraft 1.16 considered so different from previous versions as to warrant its own leaderboard the update substantially changes the amount of time needed to complete minecraft under ideal conditions as i'm recording this the fastest pre 1.16 minecraft speedrun has a completion time of 19 minutes 36 seconds whereas with version 1.16 the fastest time is a mere 14 minutes and 36 seconds so why would 1.16 be so divisive well it all comes down to how you go fast in 1.16 the thing that makes 1.16 speed runs so much faster saving as much as 5 minutes compared to previous versions of minecraft don't have to do with tricks like bunny hopping or diagonal bridging or other methods that minecraft speedrunners have spent years perfecting it all has to do with what we've been talking about this entire time random events and most specifically the piglin barter that ender pearl trade is where the extra speed of minecraft 1.16 comes from compared to previous versions and unlike bunny hopping or bridging this isn't the kind of thing that you can get better at by practicing whether the bartering system gives you the items you need mostly comes down to pure randomness and bad luck can ruin the most skilled of runs now obviously rg has been a big part of speedrunning and especially minecraft speed running since the start after all the most popular speedrunning category involves playing on worlds generated from a random seed but that sort of randomness is seen by many runners as the interesting kind the kind that forces you to improvise along the way to me speed runs like that test the most important skill that you can have as a gamer something that i call sight reading a game having enough fundamental skill and knowledge of the game's mechanics that you can just be dropped into it and know what to do in order to put down a solid time no matter what curveballs the game may throw your way but barter rng doesn't really challenge the player that way it's more like a slot machine where you just shove gold at an npc and hope for the best and the swing in times that it can create is massive larger than many other rng elements from the past maybe even speed run breaking which merits the question does it defeat the underlying goal of what being a speedrunner is it's not surprising that dream's exceptional luck is certainly one of the most attention-getting and controversial parts of this whole incident after all he's a youtuber with 15 plus million subscribers and even more who aren't subscribed to his channel but that doesn't change the fact that with version 1.16 minecraft has become a game where your ability to set a world record is mostly up to random chance it's partly a measure of your skill absolutely but arguably the biggest thing that's being tested is your patience do you have the persistence to keep playing for the hundreds or thousands of hours it takes for the rng to bless you with the luck that you need to get the best time is it even worthwhile to invest those hundreds or thousands of hours when the most important factor in your success is something that's completely out of your control so then what would the alternative be perhaps a more accurate yearly indicator of the best speed runner for that version of the game would be a tournament of speed run races dropping two or more players into the same random world seed and seeing who can roll with the punches the best who can use their knowledge of the game to get to the end of the end the fastest you would probably have to make piglet bartering off limits but would that be any different from a smash brothers tournament where certain characters aren't able to be used because of imbalance or being broken i don't know it's just a thought that i wanted to bring up and heck i just really enjoy watching a competitive version of minecraft because let's face it if that comes out of all this controversy well i'd definitely call that a silver lining well that and everyone being forced to learn statistics in order to talk about it but hey like i said earlier learning is and should always be fun and lucky for us today's sponsor skillshare makes it fast and easy to learn new skills from the comfort of your own home skillshare is an online learning community with thousands of inspiring classes for creative and curious people just like yourself explore new skills deepen existing passions and get lost in creativity they have classes for all interests coming from industry professionals in all sorts of fields like photography graphic design animation music production hit those sick beats rap pat is coming at you in all seriousness though i am a person who can't keep a plant alive to save my life you've heard of a green thumb my thumb is gray and sludgy and kills everything it touches i am literally the muck of gardening but one of my new year's resolutions is to remedy that and thanks to chris sach and his class happy houseplants i am well on my way to turning my house into the studio ghibli greenhouse of my dreams move over top plant youtuber plant arena there's a new youtube botanist in town and his name is plantpat matplant skillshare is curated specifically for learning meaning there are no ads and they're always launching new premium classes so you can stay focused and follow wherever your creativity takes you all for less than ten dollars a month with an annual subscription and best of all they've been kind enough to offer me a special offer to give to you the first thousand of my subscribers who click the link in the description below are gonna get themselves a free trial of a premium membership unlimited learning absolutely zero dollars all at your fingertips so try skillshare today find yourself a new hobby master a new skill and as always remember that's just a theory a game theory thanks for watching

Channel: The Game Theorists

Views: 7,288,334

Rating: 4.662467 out of 5

Keywords: minecraft, Dream, dream speedrun, speedrun, minecraft speedrun, dream minecraft, dream minecraft speedrun, dream smp, dream team minecraft, dream team, minecraft dream, minecraft dream smp, dream smp minecraft, minecraft 1.7, minecraft update, minecraft lore, unsolved mysteries of minecraft, dream unsolved mysteries, game theorists, game theory, matpat, game theory minecraft, dream fake speedrun, dream, minecraft mod

Id: VczZCoZ21jk

Length: 23min 30sec (1410 seconds)

Published: Sun Jan 17 2021

Reddit Comments

[u/TheVostros]

Conclusion: Minecraft 1.16 shouldn't be speedran

Did I listen to that right? Was that his conclusion out of this? Lmfao

[u/CallMeNobody016]

If Mat didn't want to offend anyone, he shouldn't have made this video in the first place. He had the opportunity, to talk about this topic, and maybe end the whole drama (since he is the science guy, with 13 million subscribers), but he wasted it. It became some clickbait shit, with an awful conclusion... Sad

[u/Nemenian]

He dances around the question without actually answering it, like Mat always does, to try and avoid offending anyone.

[u/Cyber-Gon]

What a terrible video.

Give a conclusion.

"minecraft speedrunning shouldn't exist" well guess what - speedrunners still have the option of playing on 1.14. Where they don't have to deal with that kind of RNG as much (other than the 1/3). People who run 1.16 choose to play with that RNG.

Also speedrun tournaments already exist??

So bad.

[u/Nothisisweird]

The title of this video is pretty misleading. He’s explaining why it’s hard to conclusively say whether Dream cheated or not, he’s not telling us if he actually cheated. Don’t bother watching if you expect a clear cut yes or no answer.

[u/sapio42]

First time i disliked a game theory video

[u/MyoTheHamiltrash]

i really thought he was going to answer the question

[u/Pastapalads]

Just finished watching it, it explains the math pretty Well, though it didnt even come to a conclusion and kind of clickbaited

[u/Bulka_losu]

I really liked how he explained the math, but the fact he never answered the big question is so infuriating, ugh. I was so dissapointed when instead of answers I got only 'just stop speedrunning block game lol' and then skillshare ad. Also, I hoped he would mention at least Karl Jobst's video, but welp. Definitely below my expectations.