1,500 Slot Machines Walk into a Bar: Adventures in Quantity Over Quality

[Music] welcome everyone to the 1500 slot machines walk into a bar talk adventures in quantity over quality a couple notes before we get started please silence your phones also you may have heard that there is a event given store by Microsoft with some servers that need some extreme cooling so it's extremely cold in here that's untrue we told them to turn the heat way down because the heats gonna get way up in this talk am i right yes so my name is Alex warts and I found it in ran alchemy labs the folks behind mobile games smuggle truck Jack lumber discourse we also made job simulator and Rick and Morty be our we were required by Google a little over a year ago and I left recently to found an unannounced new venture and I am Xaba I run pop cannibal makers of such gems as Adult Swim's girls like robots the IGF nominated LG for dead world and boatbuilding crowdfunding success make sale now in early access so let me first start off by you may have read the description you may have come to this talk thinking what's going on here let me tell you what we did in 2013 as a joke slash experiment the two of us built a robot that builds slot machine games a lot of slot machine games and this robot went on its journey to automate the process of creating rescanning theming and launching those slot machines on the Android Google Play Store we eventually created 1500 slot machines way beyond our expectations we received over a million installs of these slot machines and made actual money so let's talk about this so we are going to tell our tale today in four parts first alex is going to describe the seed of our discontent which blossomed into this enterprise second you will ride with as we roll this snowball downhill collecting mass and speed until third we realize this may not be a joke and finally we will enrich your lives with conclusions that can only be arrived at by people who have breathed the rarefied air from Heights such as we have achieved so let's go back to 2013 just a quick heads-up if you're watching the slides during the talk at the bottom and the footer every once in a while will have actual slot machines that were created and released and down loaded with real stats displayed next to them so keep an eye out for those gems like 3d puffin slots with 42,000 downloads so 2013 here's a picture of myself just two indie game developers with whole cans and we were working together in a co-working space making video games and at the time we had actually just launched premium mobile titles and we're kind of coming across subpar sales and discontent with the whole market 2013 was a different time and we felt like we were doing everything right based on what we knew about the mobile market at that time everything within our power to succeed so by that I mean we're hitting all these kind of like we're doing it right checkboxes on mobile we were working with big brand name publishers which was Sega Adult Swim games we were winning awards we were getting good reviews and none of that seemed to actually make a difference in you know making our money back and being able to make enough money to make the next game so at this point we're pretty much concluding that the mobile space is dying in at least four premium games and we weren't feeling too happy about that and sharing that unhappiness with our other indie friends and seeing that you know hey this is not really a great market and we were looking through the App Store and realizing how kind of scummy and gross some parts of it can really be and there's clones everywhere there's garbage Rhys Kinslow effort shovelware and you know Casino phone bill I've no idea like what is this gummy bear card game I have no idea jackpots all around and so we thought you know this garbage how do we how do we do better how do we do gross better how do we hit rock bottom and find that spot so global Game Jam was coming up and it was at MIT and we're feeling so burnt out on mobile and we just thought you know let's just build something really really out there so I did most of the tech on this project I'll admit to that so at this point we want to make a terrible game I didn't know if I was capable of that but I was going to try really hard then we need to clone it over and over so this involves like pictures and sounds and assets and it just sounds exhausting and like how even does slot machines work not let's not bother with any of that sorry there we go we just bought one off the asset store $15 so that's right our whole talk whole enterprise is all based on a $15 game that we did not even make ourselves it was just this generic Shane you put the handle you get noises many things and coins sometimes I don't know I don't actually really know much about all these slot machines work just don't judge us too harshly though because we did put an effort to make it our own we stuck advertising on it thank you you're welcome play Haven ads to be specific and Alex will tell you later how they eventually felt about that so this is going great but we only have one game and I needed infinite games which required further and deeper customization so we swap the name curling slots becomes cantaloupe slots curling stone becomes a cantaloupe are you feeling that yet this is okay wait there's more so we also added in an about for every slot it's it's it's this flavor text it just really brings the feel of the theme to you in this example here we have hair slots which did you know hair is a protein filament disease these are practically educational games at this point and the piece de resistance every slot gets a custom theme song so we use text-to-speech for the vocals and then we deftly mash that in with our phat beats are you ready for this let's do it cantaloupe oh so then we just swamped those assets about a thousand times by hand of course not we wrote a unity editor script that would just take one word and then it would use Google to download the text-to-speech for our hot track it would go to Wikipedia to pull in a blurb for the about page sometimes we'd end up with a disambiguation page because like GDC also means general dental council and other stuff I didn't really care because sacrifices to quality weren't just made they were sought after we used Google's image search api to find the images until we hit a rate limit we use bings image API to find some images Thank You ping so at this point we can just type themes into unity and it pulls in all these images it makes the app and this is amazing but that does not cover publishing these in the Google Play Store a Google Play Store it wants screenshots it wants Desi's checkboxes and blurbs and more screenshots and dropdowns and google you are asking too much of me are you trying to ruin my entire day well privacy policy yeah so the screenshots and the icons were not a real problem we just go into unity we move the camera around we throw up a sassy frame BAM that's quality but for the actual Android developer page enter selenium so selenium is this super potent web automation tool that lets you record customize and playback these web browser interactions so we connected that into our unity editors group script and we got it to open up chrome login is our Google Account create a new Android app fill in all these text boxes screenshots blurbs more nasty nasty check boxes upload the apk and it's live so at this point we have reduced the gap between conceiving of a game and having it live in the marketplace to typing in one word and pressing enter so you might be wondering yeah now what well some stats for you first two months we launched 126 games had 32,000 downloads some some old classics such as Richard the third slots 71 downloads in the US great we've got the Harlem shake slots 2013 remember it was cool then sure it was so you know this thing's chugging along we launched some games we had our fun so we did what any self-respecting game developers did is we walked away and forgot about it and we actually came back many months later after the holidays a lot of times app stores have kind of a bump during the holidays but I wonder what I wonder what happened and we saw the Downloads went way up but then we checked our play Haven ad network back-end and you know we were surprised as you we first saw that in January and throughout the holidays we're making 210 dollars a day on ads not only that we saw that the eCPM for those interstitial ads was at $59 and so I you know we I think we promptly googled what is it eCPM and it is that's way too high we also saw and I'm you know I'm no scientist but 27 percent click through on the ad there's no way that humans click on ads anywhere above somewhere around 1% so we needed to step back and think what if we what's happening here what the is going on and after thinking for a bit we have to kind of step back and think about the basics what are video games and why are we showing why do people show ads in games so the typical mobile model for ads which I'll dub the retention model is as I see it you make something of a game for the intention of having human beings have fun in said game so that they will stay inside of the app and if they're playing for longer you could show them more ads right this is in one game now what if you took this model and you turned it completely on its head ladies and gentlemen I present to you the signal-to-noise anti retention model you have an obscenely low quality game thousands of them in fact that are found via hyper specific keyword searches if you really love unicorns we've got like nine slots for you it's great then you get the game and you're instantly bored disappointed dismayed disgusted and then BAM we show you an ad but not just any ad a perfect ad this ad looks so damn good the idea the idea that a real game a real game of quality is just out of reach that with a single click you could be on to greener pastures the contrast in quality between where you are in this garbage pile and where you could be is so high that of course you click you click to escape you click for a better future and maybe that's why our games are making a lot of money I don't know so it's just a theory so clearly you know chugging along thanks Shutterstock the next thing is what are we well clear we have to put more time into this obviously so next step got to incorporate signal to noise LLC you're welcome we are the noise and yeah okay so technically what we had was good but my life was not yet all just champagne and Three Musketeers bars I had two major problems one we had to sit there and type in themed after theme wait for the process type another theme ever imagine sitting at a computer for up to 20 minutes a day just so you could publish a half a dozen games no second problem we discovered Google had a max 15 app a day upload limit I mean come on well it's like they're not even trying to support our workflow it's time to react attacked so I wrote a web service and a website middleman we call it the goose Golden Goose and it's a PHP my sequel server with this HTML front-end and it takes keywords and then it lets me pick from several things finest images and I can tweak the blurb if I'm feeling artsy and it lets us build up a long queue and just pile a hundred different ideas into this most importantly I stole unities documentation style sheets for this site so at a distance my activities look very legit to my office mates please note there's only one language yeah so we put these hundreds of apps into the goose and then back in unity it doesn't have to talk to all these crates of different API is it just talks to the goose does hey goose give me an egg which describes describes one of these apps and then it builds its 15 the day and now publishes it and we can now do this from a web browser anywhere in the world we are just typing in our ideas and willing these slots into the world but what I don't like is that I have to leave a unity license open and a machine running that's capable of building all these and it ties up my license and I have other side hustles that I need to be getting on with and I'm not trying to break any Terms of Service so the next idea is how do we free up the unity license we realize we only need unity to build the slots not to publish them so we have the website where we just throw a hundred ideas and we go to unity and then we just build hundreds of slot machines all at the same time but instead of publishing them we just shove them back up into the goose and moving on to the next step we built a standalone game that does the publishing and says hey goose do you have anything needs published it opens a chrome does it sting and publish so now any cheap computer can be a publishing terminal and so you can think of this very much like steady at home there's this central repository of important work that any computer can now process to advance the state of mankind what you're explicitly seeing here is that every time Marcus Fenix from Gears of War gets hit in the head by an image from Bing a slot machine is born into the world so with this we could keep a stream of 15 games a day being published as long as we had this queue of them ready to go in a garbage laptop waiting to do its bit on a good day we did absolutely nothing so you might be wondering what what resulted of this next adventure 10 people still playing lobster slots one year in we had launched 750 games with over 200,000 downloads across those games so a couple quick stats we've got bowling slots a simple simple bowling game with 8000 downloads we've got dolphin slots just killing it 17,000 downloads just for dolphin slots it's very america in malaysia you know and we've got now hundreds of apps on the store we're bound to get some takedowns or arbitrary takedowns from google right first you know we saw apparently sideboob is no longer okay which that image alone if I were to show it to you with no other context no one would know that that is that's sideboob once you add the text maybe we're starting to get bug reports from users in emails so that means that someone's so engaged and so upset about their diaper slots having having crashed that they opened up you know Gmail screenshot it sent it to us and had a whole whole thing about it we're getting user reviews back these are actual user reviews I swear to you why are people enjoying any of this I don't understand okay some fair fair content this person really wants to read it lower than one star that's fine but this person really loves dolphines and they were very satisfied this person also loves dolphins and you know what I think we served them well so at this point the robots arrived and this is in the form of automated email outreach so let me clarify our automated b2c casino app company is now starting to attract automated b2b outreach so here's a good screenshot a Peg's was very interested in promoting 3d cabbage slots I could see why they said this baby's headed straight for the top but we're also getting a couple other pieces of outreach from that company and you know we both run actual companies and we get a lot of spam and our detector is really high and a lot of you could tell like hey they're not actually talking to me that's just a robot that said this and when I get those at the real company that I run I feel like man I don't deserve getting all this spam but here I kind of feel like we deserved absolutely deserve getting all this spam and you know that's fine okay so third push already on a good day we are doing nothing but we still have to occasionally shovel keywords back into the goose so it's time to optimize question is what makes great garbage why our dolphin slots hot and Hot Tamales not well when we started making these slot machines we were just kind of randomly coming up with these themes willy nilly it's kind of like Keyser söze we're looking around the room we're making microphone slots handsome co-presenter slots beautiful audience slots whatever we want we're just throwing them out there but it occurs to us that most people are probably getting to our apps by searching for something they actually wanted and just stumbling over us instead so how can we figure out what it is they're searching for so they are sidetracked by us more often well we couldn't find any actual search lists for the App Store but there does exist Google Trends which is just a global of what people are searching for on Google so here's a shot of Google Trends last week the stuffin read to the stir the problematic things it's a bit of a copyright landmine so once you've filter out all the celebrities and sex offenders and litigious institutions what you're left with are mostly current events and often they're actually very weird current events which of course we used this led to an early favorite of ours deer antler spray slots so this was at the time there was some sort of NFL doping scandal where people were spraying themselves with deer antler derivatives I don't I don't know I'm not a reporter I don't have a responsibility to actually carry this story to you I just made the slot machine and so this one has a particular spot in our hearts but surprisingly it actually failed to garner an audience so if Google Trends isn't working for us there's not enough of them and people aren't into it what can we do to automate this more how can we think less word lists so I downloaded several large lists of words and you may also have noticed that we have a bit of a formula to our slot so let me give you the tour of that first 3d because 3d is hot it's the future also it's really good for the alphabetical optimization it's right there on the top of your phone and then an adjective any old adjective will do I have a very long list of them and they're not all great and after that put in a subject every country got a slot machine and slot slit note it is because with this audience there are some questions and free so you know what an incredible value this is then we made a button in the goose where I could just click try again over and over and it would just iterate through this shuffle them together go through the api's and put stuff and this is how we came up with genius such as Viceroy butterfly and tremendous face pain slots I did love that one which culminated in the creation of a masterpiece by far my favorite slot maybe the glade the greatest slot of our generation something so specific that one pure soul out there might see it and think wow they really know me they know my needs and they know my generation and I present to you 3d inexperienced great horned owl slots free I mean I was training a great horned owl recently and I had some difficulty with they're all related drills and I thought oh man I should go to this loft for comfort but yeah maybe it is slightly too specific or too targeted so at this point we thought we need a professional we need to bring on a contractor who can help us decipher all this data and optimize yeah let's do it if you recall from our presentation description on GDC we said that we involved in MIT statistician I regret to inform you that we forgot what university they were from and that is incorrect they were actually a Harvard psychology PhD I regret that small error so she came back to us with some real scientific looking data and I won't bore you with the statistics but I will call out one of my favorite parts in the middle which was that in the spirit of Ottomans automation automation she employed Amazon Mechanical Turk to do all the mundane tagging and sorting of the slots which for those who are not familiar Mechanical Turk is a farm of human beings who complete tasks that robots cannot yet get to the level of automating which is just perfect so our consultant on her own without you know inquiring with us outsourced monotonous work to essentially a human robot farm so that we could better automate creating robots anyway the final slide of the presentation was the most important one what are the takeaways what have we learned about all this data we learned that sex is great people hate other people and they hate objects but they love dogs it specifically dogs so you're welcome everyone this is the hot juicy data that you could use in your games moving forward so yeah we're giving away for free which is incredible we also after seeing all this are wondering about all the inputs and there's so many things we could test against one thought was what if we did a side experiment to see and try to answer the question does quality actually play a role in all of this so we thought let's just rewrite the entire slot machine from scratch and make a really good slot machine and see how it compares to a really bad slot machine but we clearly were not incentivized to do that so we used some of the money that was coming in to outsource the entire process to Romania and you know what we reached out and find a fund of fine Romanian developer who well we reached out to him under a pseudonym wrong and paid them hourly to mock up art design program a new slot machine temple from scratch you can see this looks very complicated I have no idea what it does no idea apparently you can bet on 15 separate lines and there's all these kinds of knobs and buttons that people who play slot machines like so we did that and it got built and it's beautiful and it had audio and all these things and we never ended up launching it because we're too busy creating the the low quality ones but I regret that part of the story so at this point we have lots of questions and opportunities what to do next and what do you do at your company when you need to make high-level decisions company retreat so just like our original slot machine we did not set up this retreat but our selves we use somebody else's it was time coincidentally for the annual Indy cabin so every year for almost a decade Indy's we pull in about 10 to 20 game developers in a cabin in Vermont or New Hampshire and we work in play for a week so this particular cabin we show up we have this growing awareness that we've kind of been torching our friends with our hilarious slot stories and how easy it is to make money for years now so we came with a pile of booze sort of as a bribe or apology to forgive our exuberance and as a stunning counterpoint to demonstrate our inability to stop celebrating ourselves we showed up wearing deer antler spray t-shirts by the way we created one extra and this is the last and final shirt that exists in the world for deer antler spray slots it's a medium raise your hand if you would like it and we will throw it directly into that pile of hands somewhere oh yeah we had the picture of theirs that's it for all you didn't get one this is what you're missing out on it's a really comfy - all right so at this point we are literally drunk on our own power here it was excellent I regret nothing but this is when a lot of the negatives and the opportunity costs start to come into focus for us so let me paint a picture of this pivotal point so this cabin we've rented is actually an offseason sex therapy retreat that was last rented by strong and vigorous people so Alex and I are in a room with beautiful tapestries have barely abstracted genitalia and a shattered bed and we're there so where you hold your business meetings we are there with a friend who is a perennial advisor and investor in the Indies fear and he's telling us this really isn't a joke looking your growth curves and what's happening and if you actually paid more serious attention to this you could pull in real investors you could maybe sell your company sell your data or just go further yourselves and maybe make piles of money but you'd have to take it seriously so we're looking down that road and question is are the people down that road still us would this be our first steps to wild success as a business person or is it going to convince us to cross lines we maybe didn't want to cross and we become super villains should we really go all-in on this crap Factory so this is the point at which a number of mounting issues and back-to-back kind of surprise issues arose all adding to the overall friction of this enterprise that we're trying to do on the side so keep in mind first off this is several hundred hours of work so far the entire enterprise is a fragile scaffolding with so many points of failure it's incredible and hard to even explain so the whole process end-to-end relied on so many external services over a dozen all remaining functional everyday for this to keep going their servers databases text-to-speech the developer back-end remaining consistent being Wikipedia unity updates ad network stuff in fact if a new language or country is added to Google Play or a checkbox gets moved 10 pixels to the right the entire thing falls apart and you know that's not to mention when unity updates and need to fix things and Android SDK updates that are required for the Play Store hosting issues that come up and so in addition to all of that pain there's now competing interests in our lives as developers doing real things outside of the side hustle VR is just beginning to take off as the oculus dk1 Kickstarter happened and alchemy was jumping on that and focusing there LG was getting some great attention I believe my sensitive poetry-writing game is in the criminal gameplay workshop I'm an artist and right around this time this is all happening very very quickly after the advisor meeting is we were contacted by our ad provider play Haven and they said hey could we have a meeting and we were like why are we so nervous about this did this be because we did something wrong did we do something wrong and so we had this call and I'll describe it to you they said we're seeing some fairly erratic data coming from your side of the world you did not break any Terms of Service will be very clear we did not break any Terms of Service you're not doing anything wrong per se you're just inconvenient to us and you know they said we're not interested in continuing to serve ads in your game games plural and also by the way you have the worst users somehow no one who comes from your apps wants to spend any money and you know I think there's another way to think about that which is you're welcome for segmenting your users so finally and you should be able to capitalize off of that but you know what fine we're off to chartboost immediately no worries and you know what that relationship turned out quite well we even received this little digital postcard from them after a few months so how sweet thanks to our boost at the same time Google anyone who is a Google developer knows that you just get emails about the store and what's changing and new new Terms of Service updates and they send out these periodic emails and they seem to start kind of mentioning some common topics which are about repetitious apps and duplicate content and then specifically unique functionality and being maintained within you know the ecosystem and so it sounds increasingly like they're talking directly to us and we're wondering you know okay we need to actually make that decision now as like where are we going what are we doing so you might be wondering what happened next this thing is making real money some there's some obvious issues to deal with but they're not insurmountable we could fix them it requires serious dedication pulling back from the main things that we're doing so what did we do we walked away again that was that was our big decision yes lazily stop touching the entire thing in action was absolutely the decision that we took and I think the weight and the responsibility of all the things that we could not they still could not automate away from ourselves was what ultimately drove us from not wanting to continue don't get us wrong don't get me wrong we left the keys in the car and the car running you know we didn't we didn't shut it down we just let it die slowly over many years so you know we watched the empire slowly crumble and just focused on making real video games but the enterprise lasted way longer than we thought it would so when you log in to Google Play now it alerts you at the top very helpfully that hey one of your apps needs attention at its the understatement of the century there it is a ghost town hundreds of apps all removed that Ghost out of its previous grandeur you know of course installs dwindled over time down to zero but it lasted years past the mission it was constructed for much like some other proud robots out there and you know I think it's worth going over a couple fun facts about the ashes of our Empire first the Middle East loves puffins useful the u.s. loves Obama 62 times more than Malaysia and India is super into sexy librarian slots we see you India and so here we are present day and I think it's worth reflecting on the journey about what we built why we built it and how it changed the world so clearly you're thinking well what's the what's the legality of all this I will answer that we're always within the legal lines to our knowledge our auto generation algorithm did occasionally stumble upon some trademarks sure if we ever got a takedown from Google we would of course comply for example the time that we stumbled upon someone who had a very aggressive trademark for Bible slots you know we received to take that of course who removed it we want to let the Bible slots person continue on their way they work hard for that yeah so if it's legal sure but is it ethical Ziya so first question did we harm players well we never took their money that was a big line for us it's it's gambling some people have problems with that I'm not really educated in that area and I don't want to be involved in these problems we only took money from ad providers and I don't feel bad about that and in fact I think that puts us a cut above all other lot of other slot machines and we're actually creating a safer place for players to come and enjoy this kind of entertainment similarly we never took anybody's data have you ever seen an app as low-quality as ours and felt safe that it's not stealing your data we were a real cut above in being a steward of our players money and in privacy of course you saw we had quite a few good reviews not all but some people were happy with this we were bringing joy into the world and as a tangent if you remember that is actually against our business model if we really wanted to go for this we would have brought more disgust into the world but the virtuous beings we are could not bring ourselves to do that and we couldn't help but make people happy did we harm game developers I love you all I wouldn't want to do that to you so we had the same tools as everybody else we didn't have some secret back-end we just used them a little differently possibly more aggressively than the rest of you but we generally played by the same rules you know who drilled the speed of traffic if you look particularly at the Android store it is a pile of unlicensed Star Wars things and and weird clones of your games we did none of that and as Alex alluded to earlier you know when play Haven dropped us because they didn't like our users coming out of there what we had done is we had taken chaos and created order we had had filtered filtered people through into a group and if they couldn't figure out how to use that knowledge we presented them that's their loss but I say we made this ecosystem better by doing that did we harm platform holders we never knowingly broke the Terms of Service we didn't argue any of the takedowns I will still contend that tasteful side boob slots deserves its shining moment in the Sun but I'm not going to engage in that email thread with anybody at Google they don't have to worry about that and I think we expose some insufficiencies gently often when you know a Terms of Service have to change it's because some sort of catastrophe happened somebody really gamed something did something bad there was a leak but we were I think a really gentle partner in this period of discovery about what they did and did not want on the store kind of like if you have like a rash on your arm a doctor may draw a line around it just so you can see if it's growing outside of that line or not but we were respectful of that line we were like a very manageable rash that just it helps to remind you that you should wash your arms more often here's a shot of what our actual impact looked like on the store on an average day when we were operating roughly at capacity so here we are we're all the way down on like page six of specifically top new free casino everything outlined in red is ours and yeah if you go down to seven eight nine ten they're probably very similar there's a lot of us there but we're not right at the top and I mean imagine we were not on this page at all number one fan marijuana light might have been half a page higher and I hope they can forgive us but I don't think we did too much damage there so what I'm really getting at is every hour we spent on this was an hour we didn't spend competing with the rest of you for a feature on Android or Apple or steam we didn't spend our runway funding on a four-year passion project that nobody but us wants underpaying contractors and promising rev-share filling every subreddit and twitter thread with our gifts and elbowing to get cozy with platform holders we just quietly put out crap if if you spend four years making a game that nobody wants and I spend four seconds who squandered more of their precious life who used more of the world's resources and who took everybody's attention in time we are the good guys you all are animals yeah okay so that last bit was half disingenuous and we can argue about that later I don't really believe all that though I'm afraid some of it is true so let me leave you with a bit more of a sincere lesson which is that this whole project was an itch that felt so good to scratch we conceived of this improbable idea and we used our talents to make it come true and we received awards and we filmed our noses at the mobile market of 2013 that broke our hearts we got to share these joys with dozens of our friends and it culminated in this overblown spectacle right here and the itch is scratched and now we are done we have not put out a new slot machine well over a year they're all gone from the store because all it is left would be maintaining this machinery and turning the crank to poop out more slot machines and you probably realized by now I do not care about slot machines so we've moved on and if there is a business lesson in that I would say scratch that crazy itch and you know try try your dream idea there but be prepared to walk away even if it is successful because it may just be the attempt that made you happy and not the result well said cybo I think it's worth going over some final stats to close this out how many downloads did you actually receive in total it was 1.6 million downloads of slot games over the course of the history of the company what was the number one slot I'm sure everyone wants to know what was it 3d sexy skin slots free 200,000 downloads and how much money did you guys make over the course of all this about 50 grand off of ads that is what happened of course it is worth saying oh yeah sorry click through 50 we did it what is the weirdest stat that you came up with the sexy skin slots which is number one huge in Iran we see you Iran and you know I think we need something special to play us out so here we go thanks for attending the GDC table so we would love to take questions now but before that just one more thing okay so I lied a bit we have revived enough for the machinery just for today and today only if you email slots at signal-to-noise apps calm with your idea in the subject line it will reply to you with a custom slot machine today only you get a slot you get a slot etc thank you very much please fill out your surveys and don't say we never gave you anything questions hi I'm Simon rickshaw I work at a company that makes real actual slot machines in casinos in Las Vegas for us it costs literally tens of thousands of dollars to get a new game approved so is there any takeaway that I could get from this have so the question is I'm doing this for real and it's expensive like theoretically let's say the App Store had a publishing cost how you know would this work I mean how much work are you doing can you do less we still spend a lot less time than like actual games it's been like a couple of months have some pride these are all actual games I mean if I don't know what's going in to your your budget to do all that but if you just try and try harder to make worse things you'll have some metrics to see if which which path is going to lead you to victory remember it's dogs and sex and not people objects hi I'm also a slot game developer did not expect this talk to be what it was actually about have either of you considered a career in comedy slots pays better I have nothing to add hey there oh my name is Daniel I'm a business consultant and on Monday I spend my day watching mostly mobile talks and they were quite depressing the way that people are monetizing games and I got to say that this is way better than all of that stuff thank you I think the depression cycle is like a 360 loop and if you get too far it becomes yeah it's like yin and yang you know one becomes the other and I'm wondering with kind of somewhere the trends that you're seeing in the mobile market and automation through artificial intelligence do you see this becoming a growing problem with game developers in the market this type of thing will you repeat the question do we see what we're doing in automation becoming a growing problem in the mobile market in I've been so far removed from the mobile market for for many years because personally as someone who focuses on weird new innovation in experimental hardware and in new markets mobile is a more mature market where you need user acquisition and you need you need much bigger budgets so it's not really well-suited for me will machine learning changed the entire ecosystem of I think it will for targeted user acquisition I think that will get more and more honed but you're still relying on an ecosystem of quality and so I don't know that machine learn is gonna help with the creation of higher quality art if there are a way to optimize for that that would be very interesting but it seems very far removed from the end goal of getting it on to mobile and making money through ads off of that art so I don't see I mean spam spam will get more until it gets better yeah weird is all just terrifying yeah and it's not just mobile it's everywhere everything is going to be susceptible to bold new methods of creating things in volume and it's we're just on a wave of that it's gonna happen let's all enjoy that well I think that leads to a curation just becoming more and more important right every app ecosystem and so the whole free wild west of like well whatever gets downloads is going to be the top thing that you open and see I think maybe we're starting to realize that that's not the greatest method of creating a good user experience for for players also more curation she's not where a lot of the markets are going anyway they know what the platform that is are doing this not what platforms are doing so maybe automated creation maybe and then yeah you subscribe to things they'll just make automated purchase free like humble monthly that's just them maybe they have algorithms deciding what's in there and you just own their money and it's automating the purchasing it's automating the advertising it's automated in the stores we don't have to do anything anymore and we're great consumers it's it's we don't even have to play the things that we automatically bought it's it yeah it's actually interesting you mentioned automating curation because another side project they did over a weekend a few years ago was when steam curator program was announced built a website called steam curator curators so you can curate which curators you want to look at so I think we could there's a couple levels deeper that we could go there and I think that's a ripe opportunity thanks guys this was a great way to start my day okay so I'm a I'm a huge slots fan are you drunk right now [Laughter] maybe no I was gonna ask both of you if you could speak to apart from all of this related to mobile if you have a mobile game in the last year that has really stood out to you just to give a moment of curation check my phone polytope you guys incredible that's what's called right yeah no one no one went whoo am I the only one who likes polytope yeah is that it's not a thing did I make that game up did I imagine this in a dream it's like a little sieve like game it's incredible go play it a real recommendation I would say anything that Zach gage makes I was been playing pocket pool on my phone for four months now it's absolutely incredible yes single pouch is that's a good one anyways how many times did a Google wake your automation for force polishing well one too many at least how many the question was how many times did Google break our automation repeatedly cuz I mean it's probably in the somewhere in the low double digits I would say we had to go in and fix it because things like a new country comes in or a country would change its policy about gambling so we actually had to be aware of like you know Korea's no longer allowing gambling apps we had to go in and add a privacy policy to everything and we had actually there were changes where we would have to actually add a privacy policy linked every app so we wrote the infrastructure for updating every app not publishing which is a whole different set of api's and once if you update to just add app at all they're gonna require you to catch up to all the not required updates before so they introduced new screenshot sizes which was fine until you actually updated your app now you have to also do new screenshots I had to find these images from it's it was a hot mess there was actually a lot more weird moving bits than we talked about because this isn't a Tech Talk so I feel like you've found this gem of the anti retention model do you ever you ever feel like there's some new way that you've wanted to wield it in the future or in any ideas that in the past you've had what more can be done with the anti retention yeah yeah but that's the question uh I mean like escapism as a business model I think it's travel agencies know what do we I feel like Microsoft's doing that right now with the refrigeration and chasing people out of the room somehow if they can benefit from that you know if we can apply that to other spheres of our life like talks and somehow profit from scaring people away from things well keep an eye on that yeah I don't know that anti retention scales across multiple people trying to do it haven't thought about that is that a one-one takes all type of model I think we have a lot to think about as a society and as a group here you need another retreat yeah uh yeah it seems like the only way you could make this more legitimate is to add a blockchain I had like a slide that came after this that says what about blockchain question mark and then I deleted it but I would have shown it right then so I would posit that anti retention is actually just curation because you're sending people somewhere well hopefully after the curation process and then you get the thing you enjoy it Oh fair enough um so I had a real I had a real question as well did you actually do any source control for these things you were generating or was it sort of a once-off and they're all gone or is there a github with these things somewhere is there the question is is there a pile of slot machine somewhere yes it's a big pile I managed to get I don't dream host is regretting their unlimited space offer right now on the hosting a lot of slot machines thanks guys this was extraordinarily entertaining just a quick thing I I don't know if there are people from Google play in this room and or watching this on the GDC fall later in the future but I hope there are and on the off chance that there are what would you tell them they should do to fix this Alex was here what is your message to Google do you know as Google as an ex-googler okay there was a point in here that I don't think was drilled home as hard as it could have been which was you know you got into arm rashes and the the idea was if people are sneaking by with garbage the thing that we did was to put a huge neon sign with an arrow pointing at ourselves and the garbage next to us illuminating the entire underbelly of dark weird AppStore and so the idea that if you do it at scale clearly they're gonna notice and clearly they're gonna change the terms of service slowly to ice you out you know hopefully others went down with us in the week of that like we provided the perfect targeted spot to drop the missile so hopefully hopefully that you know that helped get rid of duplicate apps that we're also in the area but I mean that's that's the like billion dollar question is in an open market how do you how do you prevent bad things and actually I think that's the wrong question it's like the bad things should be there but they should just be so invisible that it has no consequence to someone who took their phone out opened Google Play and then saw things of value that were we're great and to me that is curation yeah but curation like the argument that everyone makes is we're a group of semi homogeneous people in one part of the world how could we be the tastemakers for the entirety of all cultures and all people of all backgrounds we can't just say these are the best things because we believe that we have to like figure out what is best to you individually in this targeted curation and like somehow I think it revolves around curation but it revolves on a way better version of curation that is different for each person yeah I mean you guys mentioned earlier very early in your talk I mean this obviously is an extreme example but early on you're talking mentioned that the Google Play and App Store's in general are filled with clones right like not this extreme but nevertheless filled with clones and I think it all ties together and I was hopefully there's a message here anyway thank you if I could suggest to Google that they employ their own small team inside whose task is to grief them themselves I think you might seriously work they could you know they they it's the equivalent of security it's like any machine learning of these adversarial networks that I don't know much about machine learning but I'm saying they should just do it themselves they should do what we did to themselves see how they like it [Applause] hey first-time caller longtime fan is this and are we currently in frog fractions three so if I could repeat the question it was how did you two gentlemen get both so handsome and so clever I think that's what was with Brody ask roughly velvety soft boys is this frog fractions three that's the question just email slots a signal-to-noise apps comm with the subject frog fractions three and it will come back to you and you've got it as many frog fractions games as you want right now on your phone this is the rabbit hole everyone go go go we have I think time CAS we have time for we have three minutes a couple minutes hi if you would have taken on this project without scruples and I don't know how much data you collected if any but do you have any idea how much money you could have made if you allowed your users to play with the real money what if we'd gone full force no scruples collected money from users what I think the outcome would be like real money gambling wagering etc yeah some of you here who actually makes a lot machines you tell us I guess I don't know if you if you took what we could talk about that later if you if someone who has a sawed machine that does those things and then we multiply them by the number of times we put out these slot machines we may be able to come up with a number I don't know if I want to see it yeah it'd be interesting to see that that thinks oh there was nothing when we started we actually would have had to take money from users we would have to put in more personally identifying information to Google about what we were doing and we didn't want to do that first yep and we'll take one more I'm curious if you tracked how many times people pulled the lever on average total Oh so I have the answer to that it is definitely did not actually but yeah I think we're in the same page we forgot to even add save data at all to the app and I think one of those reports we only sought while we're putting together the slide deck years later it said when I close the app and I open it back up all my data's gone and I looked at sup and I said did we add saving I don't know and we're like I don't I don't know and it's really not important to the philosophy you know so you don't know this actually I did implement stats for tracking every time a slot machine was pulled I don't remember on what stat platform and I haven't looked at it in years it's somewhere out there it's gonna be huge I don't know we'll do a whole talk on just that number next year thank you fantastic question I learned a lot from it

Channel: GDC

Views: 93,673

Rating: 4.9359999 out of 5

Keywords: gdc, talk, panel, game, games, gaming, development, hd, design, video games, game design, selling games

Id: E8Lhqri8tZk

Length: 60min 26sec (3626 seconds)

Published: Tue Jun 25 2019

Reddit Comments

[u/GrantSolar]

This is an entertaining talk and is well delivered, but there's basically nothing on the design side occurring. It's mostly about Google Play Store and ad revenue politics. Definitely worth a watch, but don't go in to the hour-long talk expecting anything design-related

[u/Bmandk]

Before I watched this a few months ago, I had always thought that at some point, entertainment would be automatically generated by machine learning at some point in the future, maybe like 50 years or so.

After I watched this, I'm thinking it might be a lot closer than that. I realize that it isn't a fully ML approach, but it's the first steps towards that. Should I be worried for my job as a game design student? Nah, probably not. But I'm pretty sure I'll be delving into using AI and ML more, to use them as game design tools. I mean, it's sort of already happened with procedural generation. The next step is to be able to guide it so much that we can create environments like Rapture from BioShock, and I think that's not too far off.

[u/inbooth]

Watched this a little while back. It's definitely somewhat entertaining....

Though they were a source of "spam" they seemed to have good intentions