(audience applause) - Hi everyone. So I know this is probably a huge cliche since probably every single speaker says this, but it's a huge honor to be here. I was sitting in your seats about 10 years ago which seems like not that long and yet quite a long time. My class, the class of '06 just celebrated our 10 year reunion here like a couple months which was a lot of fun. So it's really really great to be here. I remember when I was sitting in your seats and you know I had just come from Texas, that's where I was raised and I had done some web design in the past, but I didn't really know anything about Silicon Valley, didn't know anything about this kind of tech hub and what it means to have a career here and startups and all of that prior to coming to Stanford. And I remember sitting in the audience and listening to the stories of people who had come before me and in particular Marissa Moyer and Jeff Fitzpatrick. They'd stand up here and they would take about their adventures after leaving Stanford and all of the great things they got to go and build and work on. And I was just like wow, this is super super exciting. And you know, now I'm excited to be here talking to all of you guys and hopefully sharing a couple of my stories along the way. So as you guys heard, I was here, I did my computer science degree, my bachelors and my masters and I was also lucky enough to be part of the Mayfield Fellows Program. So how many of you guys know about the Mayfield Fellows Program? Okay so for those of you who don't know, this is an awesome program that is really about diving into entrepreneurship and so one part of the program is that between your junior and your senior year, you go and get an internship with a startup somewhere in the Valley. And along the way as we're interning at our respective companies we're learning about entrepreneurship, we're taking classes we're doing case studies. And it's fabulous, it's awesome, it's taught by Tina. I had a lot of fun doing it. So at the time the start up that I chose was actually Facebook and you know this is a little bit of a little cheating because in 2006 Facebook was already about 100 people, so it was a startup but on the larger end of the startup. But it was something I was really excited to work on. It was a product that I and all of my classmates at the time used very religiously. So I went there and I remember my first day at Facebook in fact, I was signed up to be an engineer and work on some of their photos features and I had a mentor and her name was Rudgy and on the first day she said, okay there's been a change of plans I'm switching jobs, I'm not going to be an engineer anymore, I'm going to be a PM, I don't really know what that means for you, but let me introduce you to this pod of people over here. This is our design team, why don't you sit with them and you know, chat with them. I've got to go run. And so she left me in this pod with this group of designers and before that, I'm not a designer by trade, I didn't really know anything about design, but I got to talking to these people and I sat at that pod. And low and behold, fast forward 10 years and now my career's in design and when I look back on it it sort of seems really quite random how it happened. But for me what was so exciting about design was the chance to work at the forefront and thinking about what are the people who are going through the experience that we're building, what are they feeling, what are they thinking, how are they able to understand the products that we've built. And so to me that's what was super super fascinating about design and I also wasn't that crazy because at the time the designers that we hired were also our front end engineering team, so it was a little bit of both designing and a little bit of coding. This is back in the world where we were still operating with just websites, so you just had to do some CSS in JavaScript and front end PHP and it wasn't nearly as complicated of an ecosystem as it is today. But we got to building and one of the first things I worked on was photos and photo products, I also remember one of my first big launches a couple months later, which was Newsfeed back in September of 2006. Over the years I've worked on things like the Facebook platform and ushering a bunch of applications built on top of Facebook including a lot of games, back in the era of 2008 and 2009. I worked on Profile and Timeline and I worked on Newsfeed. And today the team that I lead, we work on design for all of the core features of the Facebook application. So when you go on Facebook and you want to catch up with your friends, you want to share, you want to watch videos or join a group, those are a lot of the things that my team works on. So all that said, that's a little bit of an intro into kind of how I got here. The thing that I think has been the most fascinating for me as I look back on all the things I've worked on is the moment when you're starting a project and you're thinking about here's this awesome new idea that I have and I can imagine how it's going to work and how people are going to use it and I want to build it right now and you rally a bunch of other people together with you and you guys are all excited about this idea. That's how everything happens. That's how any idea ever sees the light of day. But the question that I found super fascinating, is how do we know at that point in time whether this is an idea that's going to be successful? How do we know after it's all said and done and we've worked our nights and weekends and we've gotten it out to the world, will it actually be something that people find valuable and that they'll find easy to use and that they'll find well-crafted? And being at Facebook, we've had our share of features, some of them that have gone on to be great success, and some of them that haven't. And a lot of times at the beginning, you know, it feels like there's really no way to tell. It feels like you're kind of rolling the dice a little bit. And this is the thing that I wanted to really study and reflect on over all of the different products that we've built. Is what were the patterns for the things that were successful, are there ways that we can tell as we are building whether this is something that's going to work or not. And what came out of that, and a lot of discussions and a lot of postmortems and a lot of just looking at the things that we had built, was a framework of just three simple questions at Facebook that we now use to ask ourselves whether the things that we're building feel like that they're on track. And they're three very simple questions because you know it wasn't going to be a manual that everybody was going to memorize and understand, but it was three questions that we wanted everyone at Facebook, no matter what their role was or what they worked on, whether they were engineers or designers or product managers, to keep in mind when they're having a team meeting or whether we're reviewing the product or whether they're actually just talking with a colleague about an idea to just think of these three questions and to ask them. And to make sure that we have really really good answers to these questions. So that's why I'm going to talk with you guys about today. So the first question is the most basic and the question is, what people problem are we trying to solve? And the key word here is really the word people. Because of course whenever we build anything we're trying to solve the problem. But what tends to happen is that you start to think in the mentality of your team or your company and you start to say things like, the problem we need to solve is that we need to optimize the click through rate of our page. And you know, you'll hear things like this all the time, boil down in small ways and in large ways. And that's not a people problem. A people problem as we define it, is if you go out and you talk to someone on the street and they were to articulate a problem that they were having, that's how they would say it, that is the people problem statement. So there's a couple things that we look at to make sure that this is a valid people problem statement. The first is that it needs to be human and straightforward. So we're not using words like CTR, we're not using words like optimize or integrate, like these are not words that people on the street would use, these are not words that people who are outside of the tech community are going to use to talk about their problems. The second thing is we want to make sure that it's solutions-agnostic. A lot of times we start problems by saying I am going to build an app that blank, or I'm going to design a website that blank. And already in that statement is an inkling of what the solution is, but what if you know the app's not the right way or what if it's not supposed to be a website? So a good people problem statement gets away from trying to already constrain it into a particular solution. The third thing is that it shouldn't be about you know Facebook or your company or your team or whatever winning. And I think a classic example is if you say, our service is going to be the best at blah blah blah. A person on the street doesn't care if your service is the one that is the best at that, they just want to know hey for this problem that I have, what is the best solution? So stating any problem as we, our team, our company, whatever wants to win, is not getting close to the things that people actually want. The fourth thing is that it gets at the why, so sometimes you might have a problem like people aren't discovering this page and that is a problem that we need to solve. But it doesn't go one layer deeper, it doesn't get at well why, what's actually the root cause of why people aren't discovering this page. Is it too hidden for them, is it too confusing for them to find, it's gotta get to not just what is happening, but what is the reason why it's happening. And finally a good people problem statement can not just solve functional problems like a confusing flow, but could also get at emotional or social problems. Sometimes people just want to feel like they belong, sometimes people just want to feel like they're validated. Those are things that also constitute something that when you ask people they might say. So let's look at some examples. So earlier this year I had the privilege of attending the Mom 2.0 Summit and this is a gathering of a lot of influential bloggers and journalists who are mothers and who talk about parenthood and those topics. And so I got a chance to meet a lot of the attendees and be on a panel with some of them. And one of the things that I heard over and over again, was basically this, and this is an example to me of a people problem statement. I want to talk about an interest with other people who are also interested, but I don't know where to find them. And this is especially relevant at this conference because a lot of times what would happen is you're a new mom and a baby, in your frame group, nobody else is having babies around the same time as you. But at the same time there's so many things that goes through your mind you want to know is this normal, my baby just did this, like should I be worried, what are some resources? You want to have a community of other new moms to be able to talk to and so a lot of times women will not know exactly where to find that community and if they're lucky friends or other people will say hey there's a local group here or there's a neighborhood mom's group, but this is something that a lot of people in the audience were talking about, is something that was a problem to them. So this is an example of a people problem statement. Here's another example. This one we've heard over and over again, ever since we introduced the Like button on Facebook. Everybody would ask why don't you guys have a Dislike button and that would be one of the topmost requests that we've had for years and years. But you know that's not really a people problem statement, that's sort of just a suggestion. If we dig deeper, the reason why people wanted a Dislike button is frankly because not everything in Newsfeed is likable. People write about hard times that they're going through, tragedies that are happening in their lives. Recently we just went through this election which was very very charged and people would read things or say things that made them feel a lot of different emotions, not just happy ones. And so that is basically the people problem statement. Not everything in Feed is likable and I want to be able to easily express other things. And finally one more example, so sometimes I call these people problem statements, sometimes they're not just problems in the way of like hey is someone going to bring this up as like a hardship in their life that they want solved, sometimes it's just more of an opportunity or a thing that if you ask people if they want it, they would say yes. An example here is the desire to share spontaneously and authentically. A lot of times we're going through a pretty cool experience and we might capture it at that moment and then an hour later go and upload it on Facebook but how cool it would be to actually have that experience live with our friends and our family. So that is an example of another people problem statement. So those are basically the first thing that we always ask, is okay do we have a statement that is really about what problem that we're trying to solve for people. That's the first question. So now the second question is how do we know this is a real problem? And by know, I mean what evidence do we have? Is there a qualitative evidence that it is, is there quantitative evidence that we can look at? And I highlight the word real, not because you know I think that there are fake problems, but just because I think the question to ask, is is this a problem that's worth solving? All of us have limited time, energy, resources, money, whatever to be able to devote our being to trying to solve problems, what makes this one the one that we should pick out of the thousands or millions of problems that are out there. And so this question's really about just making sure that the opportunity is something that is worth tackling and that we aren't just solving problems for ourselves individually but that we actually are very aware of the problems that the audience that we're building for is facing. So for the example of groups, I want to talk about, I want to find other people who are interested in discussing the same things as I am. The way that we try to validate whether or not this was a big enough use case was of course we went and we talked to a lot of people but we also looked at some of the data that we had. So when we designed the group's product to start with we imagined that most of the time you would get invited to the group. So I might start a book club and I'm going to invite some of my friends who like reading the same books as me and that's how we're all going to know about the group. So in our initial formulation of groups it was really all about being invited by somebody else who was already a part of this group. But one of the things actually that some of the engineers on the group's team had hacked over the years was this little unit called groups you should join. And so some of you guys might have seen it sometimes you are scrolling through your feed and notice a little story will pop up. And based on what your friends have joined and groups in your community and groups that are similar to things we think you might be interested in, you'll see some recommendations and you can go and explore them and decide to join them if you want. And we thought that this was just a small feature, but really most of the inviting was going to happen through the normal invite process. But when we took a look at the data what we found was that actually one third of group memberships were starting to happen via this Groups You Should Join unit. Which meant that there actually was a need and a desire already using the tools that we currently had that people wanted to find things that they weren't explicitly being invited to. That there were examples of all sorts of things that they wanted to be able to participate in and see that they just weren't getting connected to any other way. So this was I think good evidence for us that if we wanted to devote something, a larger solution to helping people discover groups, that that was a worthwhile endeavor. We also talked to a lot of people in a lot of different groups, one of the folks we talked to referenced this group called the Physician Moms group, this is actually one of the most active groups on Facebook. And I think something like one out of four doctors who are mothers in America are part of this group. And one of my friends actually happens to be a doctor who just had a baby and so she was lucky enough to have one of her colleagues add her to this group. But she talked about how meaningful it was to her and she would spend hours on this group every single week because these doctor moms would be sharing tips on how to balance their work and really tackle a lot of the issues that come from being in that profession. And this is an example of a group that everyone who's a part of it finds it hugely meaningful, but not everyone knows about it because how would you know that this thing even existed out in the world unless somebody told you. Another example is you know I've recently attended our 10 year reunion and at the reunion, a lot of people were using groups to coordinate events and meetups and the biggest barrier is that half of the people I talked to didn't know that these groups existed that there was a class of 2006 group and that there was a reunion group and that there were groups for different dorms who were planning their get-togethers. And finally there are a lot of groups, I'm a designer and one of the top groups that a lot of designers in the community join is this group called Designer's Guild and a lot of designers share tools and articles that they've read and other resources. And every time I happen to add someone 'cause I thought about them, that they might enjoy it, they're always like wow I didn't know about this, that's the first thing that everyone always says, I didn't know about this. So all of this when we did our focus groups in our research that was more evidence that in fact we should be doing more to help people discover groups. So what we ended up doing was actually building a pretty simple discover feature. If you go to your Groups tab right next to it, there will be a tab called Discover and we did our research on what were the topics that people were most interested in. So you can scroll through. You'll see suggestions based on things that you are already a part of but then you'll also be able to go and browse by category and this is where you can find parenting groups, if you're a photographer, people to go on photo trips with on the weekends or play sports with, or joining a writing group. And these are all groups that are public and that actually want to welcome new members. For the people problem of not everything that I see in Newsfeed is likable, and I want to be able to express other emotions. We wanted to make sure that this was actually a problem that many people faced and that the solution that we had would directly address it. So we talked to a lot of users, we had them go through their feeds and describe to us for each story what was their reaction, what were they feeling, sort of like free flow tell us what's going through their minds as they're going through their Feeds. And you know a lot of times people would say, well there should be more ways for me to just say something because what I like about the Like button is that it's so simple. I don't have to go and comment and the keyboard comes up and I have to two handedly type something I like the fact that I can just in one gesture scroll through things and say that I like it, but the only thing I can do is Like. And there's gotta be other ways for me to express other emotions. We also looked at how are people expressing that they don't like something today and we looked at the stickers that people were using, leaving us comments as well as the emoji and what were kind of the most popular ones, how frequently were people just leaving a simple sticker or a simple emoji. And finally we looked at also short comments. So we looked at how many comments were just one or two words and all they expressed was like awesome or that sucks or a very very short phrase that we felt we could encapsulate. So with all that, we decided we'd looked at all that data we'd looked at what were the most common emotions that people wanted to express and we designed something we felt fulfilled that criteria of being really really lightweight so you can still do it with one hand and one gesture, it's not multiple taps and we also, taking all the data we had gotten from what were the most common things, the common reactions that they had. We built the reactions product and so you just scrub through and we took the top most what we wanted to be universal reactions and that's the options that you see in that tray. And finally for the people problem of I want to share spontaneously and authentically in the moment, this was interesting to us because we had the Facebook Live product out for celebrities and we built it for celebrities and public figures first because when we talked to them that was something that they told us very directly that they wanted. And it was something that they were used to because they lived their lives in the spotlight and they're very used to going on the red carpet and doing interviews and being broadcast live. And it was something they felt they could do to connect with their fans a little bit more. So this was something that we had out that was only available to public figures. But the interesting insight for us was well is this something that people want who aren't public figures? Is this something that people in the audience, you guys, me, all of us, would this be something that non public figures would use and find valuable? And that is the question that we set out to uncover and prove. And actually one of the easiest ways for us to do that was to just build it for ourselves, we already had the code that ran for public figures and we just turned it on and see if anybody at the office uses it and what their reactions were. And we saw actually a lot of really creative ways in which people used it, some people used it to broadcast their team meetings for people who were working from home a lot of times when RVC programs weren't working super well. The other benefit is that you record the video and then it's available afterwards. So for people who couldn't tune in live, they still had a chance to watch the contents of the meeting. We also saw people just take it out for a weekend so they'd go to brunch on Saturday with their family and then they would go live and people would chime in and there would be this conversation and the people who did this were like yeah it was awesome, I had a great time. And then we also saw people taking it for a run. So a team at Facebook was doing this relay race and they wanted other folks to cheer them on and give them support so they went live and this was also a super fun thing for them and for all of the people who participated. So that was some inkling that hey this could actually be something that other people wanted. And so what we did was we launched it to a small percentage of our users as a test to see if there was interest in the market. And that was the Facebook Live product on Android, it launched on Android first. And right away we did see a lot of really really creative uses, one woman who's a tattoo artist started to broadcast live as she was in the process of tattooing somebody. We also saw a woman who broadcasted her wedding live because she had family members who couldn't make it and who couldn't travel the long distance. And it was really really cool to be able to see how she could engage and interact with those audience members and almost have it feel like they were a part of that experience. I had a lot of fun taking it out, I managed to get reservations to one of the best restaurants in San Francisco, Lazy Bear and it's a really cool experience 'cause the chef stands up there and he explains every single dish and he talks about all of the love and care that went into how this dish came together and I just decided to go live and a ton of my friends tuned in and everybody was like I really want to go to this restaurant too, so that was really fun. Okay so that brings us to the last question, so let's say we built this, we validate it's something that people want or we think there's enough reason to believe that it's something that is a real problem and that's something that is worth solving so now we build it and we put it out there. But even actually before the thing is out in the world in fact even before maybe we have a full prototype the question for us to ask is how will we know if we solved this problem? What would be different in the world? If we fast forward and now this thing that we've done is out there, how would we know if we should be happy if we should be excited, if we think it didn't live up to our expectations, what exactly should our expectations be? And I think this is a really important question to ask up front, too often what ends up happening is we have this idea and then we build it and we launch it into the world and then results come in. We're looking at the dashboards, we're looking at how many people are downloading it and what they're saying and there's tons of data points that we're trying to interpret and put together. But it's hard at that point in time to be very objective about did we solve the problem because a lot of our natural inclination is to read into the good things that people are saying and to consider all of that effort to have been worth it. And that there's biases that come from when you're already looking at data and you're trying to interpret whether or not it's good or bad. It's much better before you launch to figure out what constitutes success for you so that you can go into the launch with that understanding and as results come in you can map it to your previous, this was my criteria for whether or not we solved the problem or whether this was successful. And so what we do here is we want to make sure we set measurable goals and metrics and measurable is really key, measurable doesn't always mean like numbers or data. But it does mean there's a criteria where if I did this thing and I got this result I know what to make of that result. So for the example of groups, we wanted to help people solve the problem of helping them find other people to talk about their interests with. We ended up building a Groups Discover dashboard. What does success look like for us? So before we launched we determined that if we were successful at actually helping people solve this problem we would see that more people are then discovering groups that they're interested in and joining them. But not just joining because I could make a giant button flash and make it red and I'm sure more people would click on it and more people would join, that doesn't really count right. What really matters is that they join these groups and these groups are actually meaningful to them. Which means that if we fast forward a couple months, three months let's say and then we look back, these people are still using those groups and they're actively engaging, meaning they're talking with other people, they're reading the content, they're sharing content, they're liking, they're commenting. We want to know that people are joining groups that are actually valuable for them and that they're spending time on. So that was actually the more important metric, was not just joins but meaningful joins. For the example of I want to be able to quickly express my feelings about a story but not everything is likable, we launched Reactions. And prior to launching Reactions, we wanted to make sure we measured a couple of things. If we were successful here then what we should see is well people are using these reactions and not just using them, it's not good enough if all the people that were previously using a sticker or an emoji or a short comment, then converted to using Reactions, 'cause then it's sort of a one for one trade. If Reactions was actually more lightweight, we should see more people using that then the previous set of people who were using all of these other tools, so that was really important to us. And so that was the first thing that we looked at. The second thing we looked at was we wanted to make sure every reaction did get a fair amount of usage across different markets. And this was important to us because in the beginning even when we started to design Reactions, we had considered lots of different options. We considered maybe it would be funny if you were an awesome button or there's tons of different things that we could've put in order to make it more expressive. And the reason why we distilled it back down to six is because we wanted to make sure that it was simple to use and if you were going through and choosing from a grid of 20, suddenly it doesn't feel that lightweight anymore. It feels like you're pulling up an interface and you're choosing something and it's going to take more than a couple of taps to get you what you want. So it was really important that we nailed and got to a really small set, but that we pick the right set. So we wanted to make sure that these were universal and it wasn't like people in certain countries were using certain ones more or people in certain demographics were favoring others because it was a more niche thing. We wanted to make sure that these were actually things that expressed emotions people wanted to express in all of our different countries and all demographics. And then finally we wanted to make sure that the experience was good for the receiver. So if you posted something and then you got a bunch of reactions, did that make you feel bad did that make you feel confused, did you not know what was going on, or did you just feel like it was a negative experience and it made you want to share less in the future, that was also really important and that was actually one of the reasons why we never straight up put a dislike button because dislike in and of itself can be very ambiguous. Am I disliking the content that you posted, am I disliking you as a person, am I just not agreeing with your point of view and so in all of our research and understanding of dislike we knew that it was something that people were going to get confused by and it wasn't going to be a great experience. So when we launched Reactions we similarly wanted to make sure that that wasn't the case that the emotions that were being expressed felt like they were adding to the poster's understanding of what their friends and audience had to say. And finally for the example I want to spontaneously and authentically share in the moment, what we decided to measure and look at was that of course people were broadcasting but again it's not just did they broadcast once because that's something you can make happen by doing a ton of promotion and putting it front and center. People will try a lot of things once because they don't know what it is or they think it sounds exciting. The true test for us was when they went live and they had that experience did they like it enough to go and do it again, now that is the key that tells us we built something that is valuable and that's worthwhile because people will come back and do it again and do it again and they'll incorporate it and make it part of their lives. The second thing we looked at is we wanted to make sure that it'd be fine if broadcasters loved it, everybody wants to go live all the time but is it really a good product if nobody wants to watch what people are broadcasting live and then it's one-sided and over time broadcasters are not going to want to do 'cause nobody ever watches anything. We wanted to make sure that the content was actually interesting and the way that we can measure that is the time spent watching live videos going up proportionally. And when we run surveys and other things are people saying that this is a valuable thing that they want to watch. So to recap I went over the three questions that we always ask in product reviews and you know anytime we're talking about products. We want to make sure what people problem we're trying to solve that it's very clearly defined, that we can picture the audience or a particular person in mind as we're designing the product and that we're not solving problems for ourselves or our company or our team but that we're actually focused on an audience and a problem that matters for them. The second thing is just making sure we know that it's a real problem through looking at whatever quantitative data we have through doing focus groups and talking to different users, understanding that this is something that is of good use of time for us to work on and to solve. And finally being very rigorous about what does success look like and how will we know if we put something out there and it did what we wanted it to do. And if not, then we go back to the drawing board, we learn what didn't work and we keep iterating. But these three questions have helped keep our teams focused on what matters which is ultimately the people that we design for. Thank you guys, thank you very much. (audience applause)
