About a year into the development of Portal, playtesters kept giving
Valve very similar feedback. Something along the lines of "that was a great
tutorial, I can't wait to play the actual game". *VCR Pause Sound Effect* Uh. Slight problem. That was the actual game. Despite playing through 14 or so
hand-crafted puzzles, players were clearly missing some vital element that would
tell them this was, indeed, a video game. And so after a lot of discussion, Valve
decided that the game needed an antagonist: someone to push back against the player, to provide motivation to keep moving,
and to put these puzzles into context: this was all training in order to defeat a boss. The end result was, of course, GLaDOS: a loopy AI overlord with a
biting, passive-aggressive wit. She teases you, taunts you, tricks you. And the whole game leads up to a climactic
one-on-one showdown in her central chamber. GLaDOS: "Well, you found me.
Congratulations. Was it worth it? Because despite your violent behaviour, the only
thing you've managed to break so far is my heart." Thanks to sharp writing and killer voice acting, she has become one of the most iconic
video game villains of all time. But, says Valve's Robin Walker, "her
genesis begins with a straightforward process of us trying to solve the
core gameplay problem in Portal". Now if you ask me, the fact that
GLaDOS may never have existed if it wasn't for this feedback goes
to show the value of playtesting. Not to be confused with quality
assurance - that's for rooting out bugs. Or focus testing - that's for market research. No: playtesting is simply watching
people play through a chunk of the game - sometimes with a questionnaire
or interview at the end - and then using what you see and hear
to drive changes in the game's design. For example, if Valve sees that
players are routinely getting themselves killed when trying
to redirect these energy balls - perhaps they should make a change - like having it so you can only place portals on
walls that are way above the player's height. And so on Portal, this approach was used to
touch up almost every aspect of the game. Valve used playtesting to make sure
the learning curve was perfect, they used it to remove moments of frustration, to make sure players noticed key objects, to improve the pacing, to tweak the difficulty, and to ensure the storyline was coherent. Playtesting even led to the game's iconic visual
style, with those sterile white walls and floor. Originally, the game featured
cluttered and grungy environments, but playtesters found it hard to
identify the key puzzle elements. In one test, a player spent half an hour trying to push a shelf onto a button - while
completely ignoring a nearby box. Kim Swift - one of the key developers
on the project - calls playtesting "the most important thing that we
learned since coming to Valve". You see, if you didn't know, Portal
started life as Narbacular Drop: a student project made at
Digipen University in Washington. hey were invited to show the game off at
Valve's headquarters and about halfway through the demo, CEO Gabe Newell stopped the
presentation to offer the entire team a job. Their goal was to remake the game in the source
engine, and within the Half-Life universe. And to help, Valve introduced the small
student team to its game development process. Here's how it works. You start with a goal: perhaps, in this case, to make a puzzle that is clearly
readable, and satisfying to solve. You then take a stab at reaching that goal, by designing something - in
this case, a test chamber. Next, you evaluate whether your design
reached that goal by doing a playtest. And if it doesn't meet the grade,
change the design and repeat. Valve keeps going with this
iterative process until it is "no longer excruciatingly painful to watch
the playtests", says developer David Speyrer. GLaDOS: "Fantastic! You remained
resolute and resourceful in an atmosphere of extreme pessimism." Perhaps, given the subjects
of its most famous games, its apt that Valve sees game development as
an engineering problem or a scientific study. Valve's former in-house psychologist,
Mike Ambinder, says "we see our game designs as hypotheses and our playtests as
experiments to validate these hypotheses". Now, it should be said that playtesting
is by no means exclusive to Valve. And indeed, every game developer sources
player feedback at some point in the process. But the difference is that Valve is obsessed
with it, to an almost religious degree. Ambinder calls playtesting "the most important
part of the game development process". Newell calls it "our secret weapon". And a designer who joined the studio in 2018 said “I always heard that Valve does a lot of
playtesting before I came to work here, and I was not prepared for the amount
of playtesting this company does". Valve does playtesting early. Swift and her team started running playtests of
Portal after just one week of starting to build the game at Valve, even though they only
had one half-finished room to show people. And then do it often. Portal was playtested practically every week. They'd do a playtest on Friday,
discuss the results on Monday, apply the lessons the rest of the
week, and test again on Friday. It sounds extreme. But I think this devotion to playtesting
makes a lot more sense when you learn about the disastrous development of
Valve's very first game, Half-Life. About two months before
Half-Life was supposed to ship, Valve realised that the game... well, sucked. The developer said "you couldn't play the
game all the way through, none of the levels tied together well, and there were serious
technical problems with most of the game. As a whole the game just wasn't working." The only real option on the table was to
scrap the whole thing and start from scratch. But this time, they would do things
differently - which lead to two game development philosophies that are
still present in the studio today. One is the idea of the cabal: where
the staff is split up into tiny, multi-disciplined teams who each take complete
ownership of a small chunk of the game. But the other is frequent playtesting, from
the very earliest point of development. This time, if the game sucked,
they wanted to know right away. So, about three months into development of the
restarted Half-Life, they started playtesting. They found random gamers in video game shops, and contacted people who filled in those little
registration cards you got in old PC boxes. They'd sit them down in front of
the game and just watch them play. Each test would result in dozens
of things that needed to be fixed, changed, added, or deleted from the game. Like, after watching some playtesters break
every crate in a level, Valve realised it probably needed to stuff some of those boxes
with goodies like ammo and health packs. And this new process obviously worked: this
version of Half-Life was a huge success, and is now seen as one of the most
important and influential games ever made. And, as such, Valve would use playtesting
extensively in all of its future games. On Half-Life 2, Valve planned to give out
the gravity gun towards the end of the game, but players loved using it so much, the team
decided to make it available much sooner. On Left 4 Dead, playtesters found it hard
to find teammates who were in trouble, which led to the x-ray outlines
of your fellow survivors. And on Portal 2, a paint type that
let you walk on walls was scrapped, when it made multiple playtesters feel queasy. The arrival of Steam allowed Valve
to go further - converting millions of online players into post-launch playtesters. When Steam's data showed that lots of
players were getting stuck in Episode One, they released a patch to reduce the
difficulty of a tricky siege battle. Plus, playtesting was instrumental
when exploring an all-new technology, virtual reality, for Half-Life: Alyx. For instance, Valve learned that a player's
tolerance for standing around watching people talk was significantly lower in virtual
reality, so had to speed up the game's pace. "Player behaviour helped us navigate VR
development," says Valve's Christine Phelan. "You could almost think of the player as another
designer. There is barely a moment in the game that wasn't improved by what playtesters told
us and showed us through their play behaviour". So, from Half-Life 1 to Half-Life: Alyx,
playtesting has been an invaluable tool for Valve. So I want to share some tips from the developers,
to show you how to playtest more effectively. Though it should be stated: Valve
is not like other companies. From its unorthodox organisational
structure to its near infinite resources, there are very few companies on
Earth who can copy what Valve does. And also: this approach may work best on super
linear, extremely hand-crafted story games - and other techniques, like heat maps,
may work better for other types of game. But, with those caveats out of the way, perhaps their experience can still
provide some useful guidance. So, tip number one is test early. Valve says "playtesting is where we make the vast majority of our most
important changes to our game. So we try to do it as early as possible on
the project, to get the most value out of it." When you identify a problem early
on, you have time to go back to the drawing board and develop an
effective solution to the problem. But do it too late, and sometimes the only way to
fix a problem is with a flimsy band-aid solution. Like, uh, I dunno, having the characters just
tell you the solution to a crummy puzzle. Valve may start testing within a few days of
prototyping a mechanic or designing a level. Even if it looks hideously ugly, with programmer
art or bright orange textures on all the walls. But this actually helps them avoid
wasting time on other aspects: there's no point investing heavily
in art or audio if the game mechanic is gonna be completely changed
- or cut from the game entirely. Number two is test often. Valve typically tests every single week, to make sure that they are constantly
iterating on player feedback. This also leads to a huge amount of data. Each Half-Life 2 chapter had about
100 playtesters, for example. And that mass of information can be invaluable. By having a huge number of people look
at the game, Valve can look for common trends in the feedback, and avoid tweaking
the game based on a few weird outliers. And the more experience you have seeing
people interact with your designs, the better you'll get at preempting
this feedback in the future. Gabe Newell says "after you've watched a couple
hundred playtests, you start to develop a much better sense of what are successful
and unsuccessful design strategies". So, some famous lessons learnt
at Valve include "players don't learn when stressed" and "players don't look up". Number three is shut up and watch. A playtest should try to simulate the
actual experience of playing the game. Which means the observers need to stay
quiet for the duration of the test: no hints, no guidance, no
answers to burning questions. "Nothing is quite so humbling as being forced
to watch in silence as some poor playtester stumbles around your level for 20 minutes,
unable to figure out the 'obvious' answer that you now realise is completely arbitrary and
impossible to figure out", the studio has said. Post-game interviews and
questionnaires can certainly happen - those were instrumental in solving
the GLaDOS problem after all - but you'll often learn more by watching
players than by talking to them. Swift says "they may tell you later
that they like the game but you'll really tell by their body language whether
or not they actually enjoyed themselves". It's also a common game design trope that playtesters like to offer potential
solutions to the problems they face - but as they don't have insider
knowledge of your vision or constraints, those solutions are usually better left ignored. Number four is designers should run playtests. Valve does not outsource testing to a special
department or leave it with a publisher: playtesting is done by the people
who are actually responsible for the design and execution of the
level or mechanic in question. The cabal, from earlier. This shows the developers
exactly what needs fixing, and can give much-needed
motivation to improve the game. And player behaviour might inspire new puzzle
solutions or ideas for the designers watching. Like, in Half-Life Alyx, players would
instinctively cover their real-life mouth to stop Alyx from coughing, and alerting
a gigantic blind zombie named Jeff. So Valve turned covering your
mouth into an actual game mechanic. Number five is get the right people. Valve gets feedback from a huge
variety of different playtesters, from fellow staff members to
little kids to pro gamers. But its learned to always have a target
audience in mind when making changes. Like, when Valve was figuring out how to
make a climactic boss fight against GLaDOS, they initially got feedback
from hardcore FPS players, who said the level needed more action,
more challenge, and more skill. But that idea was a dud: when facing this tricky boss
fight, says Erik Wolpaw, "the vast majority of playtesters who
had gotten used to the slower-paced, cerebral nature of Portal were just
frustrated, confused, and dissatisfied". They still wanted to satisfy
those hardcore FPS players, but did so through optional content, like
Portal's advanced chambers and challenge maps. Number six is to challenge your assumptions. Okay, so maybe the GLaDOS boss fight
didn't need to be a test of skill. So, perhaps it needed to be the most
complex puzzle in the entire game? Surely that would be a suitable climax. But, here's the thing - playtesters
thought the game's mid-point escape, that bit where you use a portal to
avoid falling into a pit of fire - players thought that was
incredibly climactic and satisfying. Even though it's basically
the easiest puzzle in Portal. But Valve realised the time
pressure, the visual impact, and the high drama all made it
way more exciting to players. So maybe they were overthinking
the final boss, too. "We've been holding on to this idea that
we need a complex puzzle at the end and it simply wasn't true", says Swift, and so
they settled on a pretty simple sequence. It's very easy for game designers to make
wonky assumptions about how players will act - I recently made a whole video about
doing this in my own puzzle game. I'll drop a link to that one in
the end screen of this video. Perhaps the most important tip, though,
is that playtesting feedback is just data, and it's up to the designer how that data
is interpreted, filtered, and applied. For example, Half-Life 2 originally
had a very short introduction before Gordon Freeman grabbed a gun and started shooting. Playtesters loved it. Who wouldn't be excited when
jumping into the action? And so, given this good feedback, it would be
easy to keep the game exactly like that. But, despite the positive sentiment, writer
Marc Laidlaw says Valve decided to keep working. They thought it would be better to delay combat. "We wanted you to witness the cops doing
something horrible, and feel like what you were doing was a response to that - not that
you were just a killing machine," says Laidlaw. And they thought it would lead to
a better emotional pay-off if you had to wait a much longer time before
finally getting Gordon's iconic crowbar. BARNEY: "Oh, and before I forget. I think
you dropped this back in Black Mesa. Good luck out there, buddy." Here's the thing: playtesting
is as useful as you make it. If you just bend the game to the whims and
desires of every playtester who comes through, you'll end up with a dumbed-down game,
or bland, design-by-committee sludge. But if you go in with a clear goal. A specific game you want to make, for a
specific audience you want to entertain. And then use playtesting to validate
whether you are hitting that goal, you can use feedback to unlock the
incredible game you are trying to make. Oh, don't click off. I've got one last Portal story
while the Patreon credits roll. So, the invention of GlaDOS is actually not
the most dramatic change that Valve has made, in response to player feedback. During an internal game jam, held
shortly after the release of Portal 1, a bunch of Valve developers came up with
an experimental puzzle game called F-Stop. In this game, you use a camera
to take photos of objects. Then, you can spawn that object elsewhere in the world
- perhaps at a completely different scale. Gabe Newell loved the concept, and said it
should be developed as a follow-up to Portal - meaning each game in the series would feature a different piece of technology
developed at Aperture Science. But, after nearly a year of development, playtesters were very vocal in their feedback:
Portal without portals just didn't work. And so Valve decided to scrap the game
and start Portal 2 all over again. In recent years, we've been able to
see what F-Stop might have looked like, as fans have used leaked assets and code
to make various recreations of the game. But Valve never returned to the concept. If you want to play something similar, check out Superliminal, or wait for the
upcoming photography puzzler, Viewfinder. Thanks for watching.
