The Lean Product Playbook with Author Dan Olsen

okay this is good I'm Michael awesome okay great nice to be here with you guys tonight at product school today I'm gonna be sharing some advice from my book the lean product playbook on how to achieve product market fit I'm just curious has anyone heard of this book before anyone read it already alright cool so I'd love to hear any feedback you have on it hopefully it's helpful but this would be like the movie version you don't have to read the whole book you can just watch the movie for 45 minutes right great so just real quick so you know where I'm coming from I start out with a general background I've been coding since I was 10 my parents gave me a computer then which I'm grateful for and then I was electrical engineering major and I actually my first job out of undergrad was doing submarine design for the Navy after doing that for five years I knew I wanted to go to business school and get you know leverage my technical background get more in the business side of things so I moved out here to go to business school at Stanford that's where I learned about product management as this awesome career and I was like that's exactly what I want to do and since I had never done it I asked people where's the best place to learn product management and people told me into it and so I was fortunate to get a job at Intuit I worked on the quicken team and and worked my way up to become a product leader there after into it I was a product later at startups and then I kind of stumbled into being a product consultant to startups like an interim VP of product and I did that for box back in 2007 then I had an idea for my own startup and you know like a lot of people in the valley I wanted to start my own company and try that so I did that too so I've been a CEO and co-founder and then after that started out I just went back to being a consultant again so I've been consulting and helping companies and teams for nine years now and I also am very passionate about you know events like this bringing together speakers and sharing best practices so almost four years ago I started a monthly speaker series group down and I live down in Menlo Park so it's down there in Mountain View actually where we bring in top speakers each month this is my twitter handle and you know i post all my content on dan - Olson comm where have my videos inside so don't try to like you know break your hand taking notes I go pretty fast we're gonna cover a lot of stuff I won't go too fast but the contents all there and I know they're recording it too so and my book is you know the lean product playbook and I've been this this lego guy a ninja guys here because I've been giving this talk for a long time and used to be called how to be a lean product ninja and I used to ask people in the audience who's heard of Lean Startup and over time more and more hands would go up now it's like who hasn't heard of Lean Startup and people like embarrassed to put their hand out so if you had a summarize you know what is Lean Startup all about I think on a single like you know single slide or bullet points this is these are the key points that I would say it's about getting explicit about your hypotheses when you're trying to build a product you're making all kinds of assumptions and hypotheses it's about getting those out of your head getting them down on paper or Google Docs and then figuring out how you can how you can articulate those figuring out the fastest way to test those right the cheapest fastest way that's also where we want to keep our scope small we don't want to over invest before we realize that we're heading in the wrong direction and we need to change direction that's where the concept of MVP Minimum Viable Product comes in that we'll be talking about a bit we want to get out of the building it's great to have your own ideas but you need to get out of the building and actually talk with customers and test your hypotheses with customers and then basically the goal is to learn and iterate as you learn basically hopefully quickly with the ultimate objective of achieving product market fit so usually when I put those bullet points up everyone kind of nods along it says if I get it I get it so it sounds pretty easy right but part of why I have a good consulting practice in part of I wrote the book is even people that get these high-level concepts when they go back to their desk and they try to apply it they run into challenges right they just run into like okay I don't know I get I get at a high level on what what what's important but I don't know exactly the details of what to do and for me there's a lot of overlap between product management lean startup concepts that's why I have the word lean in there I think lean startup has helped create a vocabulary that we can use like MVP and product market fit and you know pivot things like that that that has a high degree of overlap with product management and and just like people struggle with applying Lean Startup principles people struggle with applying best practices and product management and so that's why I wrote the book the lean product playbook we're gonna be you know like we said giving one away today so I have a tool that kind of randomly draws people if you just just tweet to Dan Olson we'll pick one at the end and you know if you wants a slide you like you can take the photo that's great if you want to throw hashtag on it so other people see it that would be great too so but basically my book is a playbook and it's a playbook because it's trying to tell you exactly what to do it has a process in there that I'm going to be sharing with you but it basically is a guide for how to achieve product market fit and anytime you have a big movement like Lean Startup there's all these buzzwords right MVP pivot product market fit MVP is probably the most hotly debated and contested term out there you know people getting on like you know online fights about what it is or what it is in our arguments in there and they're in their company probably Markieff it's a little different people actually talk about it like overly simplistic aliy like if some true or false state of existence like I'll box succeeded because they had product market fit startup back sadly they failed because they did not have product market fit and it's just like a true false condition it doesn't really help you if you're trying to achieve it right and so if you go in Google and look for stuff there's not really a lot of guidance on what it means and how to achieve it and so that's why I wrote the book and as I said my backgrounds in engineering so if I want to try to achieve something I want to have like a rigorous framework that I can use and rely on and so I created a framework called the product market fit pyramid which is the core of the book and basically it breaks product market fit down into five components and basically all five of these things you basically need to get right enough to achieve product market fit let me walk you through it the bottom two layers are the market and it's a pyramid and each layer builds on top of the next right each one depends on the layers beneath it just like a real pyramid it all starts with the target customer right who's like for we're trying to make better whose pain point are we trying to solve who we trying to create value for and if you change your tire customer then you may have to change everything above the pyramid right but it's important to know who's life were trying to make better the next step up is for that customer that we just identified what are their underserved needs right what are the what needs do they have and then the next layer is like which ones are the ones that they're not happy with how they're being met today right if they're being happy if they're happy with how it's being met today we probably want to target those needs so I'll share some ideas and frameworks on how we identify underserved needs but taken together these two layers are the market and if you look in a marketing textbook or an economics textbook it'll tell you the definition of the market is a group of people that share a set of common needs that's basically what a market is now your product I like to represent with three layers those layers are your value proposition and that meant to ideally build on top of the underserved needs it answers the question of which needs are we actually gonna bite off and say that our product delivers on and most most importantly how are we going to do so in a way that's better different than the other products that are out there so that's where product strategy and differentiation comes in the next level up is the feature set ok great for those benefits that we said we're gonna deliver what's the functionality that's going to actually convey those benefits to the user and that's where the concept of MVP comes in we don't want to over build here before we realize we're going in the wrong direction right and then finally is user experience which is actually brings the functionality to life and it's what your user and customer interacts with to bring the product to life so basically you can't really control the market you can pick which one you want to go after you can say I want to go after this market and I want to go after these needs but this is where you really get control to make decisions about which value proper you're gonna go out with and what feature said you're gonna go out with and what use your experience are you gonna go out with right so when you have this view of like all the hypotheses you need to get right product market fit is just hey what are all the all the hypotheses and execution that we're doing up here how do well does it resonate with the actual market that we're going after that's what product market fit is basically so it's a simple framework that helps you kind of walk through the key hypotheses once I had this very migrated realize you know what I can create a process around this to guide people through working through these key hypotheses and I call that the lean product process and that's going to walk through tonight and close out with an end-to-end case study at the end but basically you just starts out by starting at the bottom of the pyramid and working your way up getting clear on who what our assumptions about our target customer what are our hypotheses about which of their needs are underserved you know what do we want our value prop to be as far as which needs are we going to deliver on and how we're gonna do so in a way that's better or different what should our feature set be again so that we don't over build what should our user experience be and then there's just one six step which is once you have once you've worked your way through the pyramid I'm a big fan actually of testing and validating your ideas without doing any coding or building if you can avoid it right so I'm a big fan of using clickable or tappable prototypes here and when you get to that point whether it's a live product or a prototype that prototype represents all the assumptions and decisions you made along the way and then there's a sixth step where we take that prototype or live product and we cool and we test it with the customers and that's how we see where we're at with product market fit so that's the six steps of the lean product process and I'm gonna walk you through those and it's meant to be an iterative process obviously right we're gonna have to there's some sequentiality and how they need to work together but you're not gonna get it all right the first time you need to iterate and I'm sure a lot of you've heard of the build measure learn loop from Lean Startup I have a modified version that I prefer I don't think the thing about build measure learn is it starts with build so some startups get really excited their developers okay it says Bill that's build first I actually think you know you can get a lot faster learn a lot faster without building so for me it's a modified version that starts with hypothesize you get clear on what your hypotheses are you design a way to test your hypotheses you test them and you learn and then you revise your hypotheses so that's how we're iterating as we go through the lean product process basically right cool so let's jump in and we'll go through the process and highlight it with some examples the first step is determine your target customer and I should say by the way you can use this process for a new product obviously you can also use it for a new feature of an existing product you can also use it for like a v2 of a product or a v3 of a product a new version and you can also use it if there's a product where you haven't been applying these techniques you're gonna find low-hanging fruit and easy ways to improve product market fit but let's say you're working on a new v1 product right for the first time I just want to acknowledge that you know you may not be clear on who your chart customers you may not know yet that's fine right all you want to do is have a tentative hypothesis because these two steps are so intertwined that as you learned about customer needs you're going to go back and revise your your definition of to our customer so I acknowledge like if you're staring at a blank piece of paper it's like okay Dan's telling me I got to figure out who might our customers it's okay to have a tentative hypothesis and iterate it later but let me show why it's so important to be clear on your chart customer and what happens a lot of times is I find product teams they kind of stay a superficial layer and at first it sounds possible and it sounds good I remember I was judging a competition at the Stanford Design School and I just went through you know I was a judge and I would just ask my questions the first question I ask each team is who is your chart customer you know what are you doing for them what's your value problem and one team told me I said who's your across they said Millennials and at first I was like okay cool that sounds good I got suspicious it sounds specific but then I thought gosh how many Millennials are there out there there's like a bazillion Millennials and their product had to do with a certain vertical like food preparation at home I'm like there's got to be some other way they could have clarified or made it more distinct right so that kind of that's why I call like the surfer let's think about onion and you peel the onions got multiple layers that's like the outer layer of the onion and it sounds okay at first but that's not how you really create part market fit or value got to peel the onion so let me show you this happens all the time whether you're talking about customers or needs let's talk about a need that a lot of people have you have transportation within 100 miles of your home all right whether it's you get to work to get the school to get to the store whatever it is this is the need that a lot of people have right and we could talking about that left talked about at that level but as soon as you look through that need through the eyes of a particular target customer you'll see that the detailed needs are quite different even though they share that high-low Bunny let's take to target customer personas one a soccer mom and the other a speed demon right they both have that need to get around but if I went and did customer discovery interviews of twenty soccer moms I said hey can you tell me what's important to you when it comes to transportation they might bring up things like well I'm carrying my kids and all their friends and all their gear so the cars got to be big enough to just hold all that stuff and you know I'm driving my kid kids around they're like the most important thing in my life so safety is really important to me and by the way I'm doing a lot of driving on the weekend so I'm you know I'm I'm thinking about how much am i spending on gas a fuel economy is important right so those might be some of the things that we hear that are more detailed needs and preferences related to the macro superficial need if we interviewed 20 speed demons they probably wouldn't bring up any of those things they might bring up things like well what's important to me is that the car goes really fast and that it looks cool and most importantly a look like I look cool when I'm driving down the street in it right and you end up with very different products as a result now both again meet the high level need but they address the detailed needs quite a bit right in the and just think about all the different cars and shapes and sizes of cars that are on the road so the car industry has done a really good job of kind of defining those micro segments and building products that are optimized for each of them and their needs so it's important to do that personas are a great way to capture your assumptions they're often used later in the process and UX design but I think you can use it early on the other thing is personas off get a bad rap it's usually because somebody worked in a company with somebody that misused the tools so I would just say don't blame the tool give-give personas another shot if you have it had a bad experience the second step is great once you got clear on who we think are to our customers what do we think they're underserved needs are and when whenever we talk about me customer needs the framework that I like to use is actually problem space for solution space I'm curious who's heard of problem space here before few people so I've been talking about prong spaces like 2007 you see more and more people talking about it which I'm really glad about let me explain what it is basically so problem space is a customer problem customer need or benefit that the product should address right so it's like a well written product requirement a well written user story let me give you an example if I said hey you know what I want to make it easier for people to share photos with their friends that statement make it easier for people to share photos or the friends would be in the problem space something that the product should do and I'm hoping it will do for people right in contrast the solution space is a specific implementation designed to address that need or requirement if right after I said hey I want to create a way for people to share photos with their friends I said and by the way last weekend i coded an app for it that app would be in the solution space or if i was like a my friend manny actually he's a rockstar designer he designed a great wire room mock-ups for it those mock-ups would be in the solution space right that's the difference between the need and actual the actual implementation and what happens is far too many part teams just go barreling right in the solution space I mean we live in the solution space that's where we live all right so it's really easy it's easy just to start designing or build something and not have to answer all these hard questions about who's it for and what's it gonna do for them and that kind of stuff it's easier right but when people build great products it's because they've actually stopped before they got in a social space and thought hard over here and come up with hypotheses and tested those hypotheses the example I like to use to illustrate this is when when NASA was sending astronauts into space they knew that the Pens that we use here on earth wouldn't work because they rely on gravity right and they knew that hey in space there's no gravity so how these guys gonna you know what are they gonna do in space and it wasn't NASA so if you google this you end up on some snopes.com urban legend and the main point is it wasn't NASA but one and NASA didn't ask the contractor to do it but one of the contractors the head of it Fischer said you know what I think I can invent a space back and he went off on his own spent his own money he spent a million dollars in R&D and he invented a space pen I have a space pen here they're pretty cool they tell me it right to the space I haven't been there to verify it myself I'm hoping I can connect with NASA someday and verify it but anyway so it you know it needs to need it worry rights in zero-gravity faced with the same challenge the Russians gave their astronauts pencils and you can actually get a Russian space pen it's a joke it's a pencil in a red box kind of poking fun at the whole space pen thing right so why do I tell this story I tell the story for a few reasons one is obviously if this is just as good a solution as this to the need to write in zero gravity then the one that didn't take a million dollars and all that time and effort is better because it has a higher ROI right that's like the the most obvious lesson from that but the other lesson is just it's really easy even when you're trying to focus on problem space to have what I call solution pollution in your problem space right you're polluting your requirements with the by embedding some solution artifacts in it so when that head of that company Fischer said hey I think I can invent a pen that writes in space and that was his problem statement he had some solution pollution in there what was that a pen yeah a pen is a solution right so that happens all the time the way it happens on a feature team is like in the gyro ticket I'll say build a drop-down build the man you build a configurator it tells you the solution right and the way to get out of that trap and to back into the plumb space is to use the five whys take one way you can do it is the five why's technique from the Toyota Production system from the in Star where you go well why is that important why do you know we don't why do they need to write in space right and even if he just said hey a way to write in the space that would be better than saying a pen that writes in space just be vague but if he applied five wise and said well why do astronauts need a white in space like why do they need to write in space anybody yeah you don't want to write basically I want to write down information and refer to it later or maybe make calculations like you know there's a couple of things you need to do writing that would be an even better way to articulate the requirement right get away writing things down and just say hey here's what they need to do they need to like document information or refer to later when you get that far abstract then now maybe we can come up with some crazy Siri talking situation right there the solution where you talk to it and it comes back to you right so that's the other thing where you know by getting really clear on the problem you can open up the solution set to a more creative solutions basically so those are some of the reasons that I like to share that example now it turns out that pens don't need to just write in space there's also things about not being flammable and things like that so space pens are the way to go but I just use this example to illustrate some of those concepts now let's talk about more of a software example and the idea is you can map you want to star in the problem space and then map to the solution space okay I used to work it into it on quicken but another one of our big popular products is TurboTax who's used TurboTax here alright cool great this is gonna be good so it's a software product so it's in the solution space right that's not a problem it's a solution it competes with another product in the solution space tax cut so they're in the solution space right and the idea is like you know let's try to map back let me just do this like for the people these TurboTax tell me what's the problem like what's the problem it solves what value does it give to you cheaper than an accountant the instructions for taxes are difficult what else easy and fast all right do it from home in your pajamas or not yeah has a lot of knowledge up you know in a way space in your brain with that yeah what else online filing great it validates or checksum yeah you don't need stamps perfect one more I think you said cheaper than an accountant is that what you say I think you said cheaper yeah convenient right so so that's the thing about the problem space did anybody give me the exact same answer none of you guys gave me the same answer how the heck am I supposed to make sense out of that stuff right screw it I'm just going to design and code a product and not worry about that right so there's a lot of different products the thing about the problem is messy but what you want to do and again we can peel the onion if I had to define the overall if I did five wives with you guys and said well what does it do overall try to get you kind of abstract out before I get something like well it helps me prepare my taxes help me do my taxes that's like the superficial layer of the onion like Millennials right and you guys are bringing up much more detailed things that live below it the other person that helped me appreciate this concept was Scott cook the founder of Intuit really smart guy and he would give trainings and talks to groups of product managers like this at Intuit and he'd say one of the things he'd love to ask at the end he'd be like hey who's turbo tax's biggest competitor and we'd all be like can he pick you and be like it's tax cut he's like no you're wrong it's pen and paper because more Americans are doing their tax returns with pen and paper so that's the other thing about the problem space is it helps you really understand where the true substitutes and competition for your product right and back to the product market fit pyramid the the needs the market needs don't change anywhere nearly as quickly as the solution waves that come and go in the book I talk about the the need to listen to music on the go right it started a long long time ago with small FM transistor radios and then we had Walkman right and then we briefly had the the CD or the portable CD thing and then we had the mp3 players then we had the iPod and now we all have our phones right so five or six different solution waves of technology but that fundamental need to listen to music on the go was the same throughout the whole time right so that's that's why it's important to get clear on the promise base and like I said you want to define the process before you jump institution space so what this is the fun part with your team you get to do divergent thinking and brainstorming you get a suspend judgment say great when it comes to preparing to helping people with their taxes what are all the things we could do not saying we're gonna do them but let's really explore the problem space and come up with all the different things we could do right and you guys brought up a lot of them computers are great at maths like you were saying they can check your taxes right instead of your fourth if you think about filling out forms and making math mistakes they can help you file your taxes a couple people said that you don't need stamps instead of having to go to the post office and stand in line these push a button they can help you maximize your deductions right they can ask you questions and say hey did you do this did you do that did you know you could write this off they can analyze your return and tell you what your level of audit risk might be right these are just four examples but that's what you want to do as a team is really some time exploring all the possibilities you could do in the solution space and the problem space sorry before I'm not saying we're gonna do them right it'd be plenty of time later to say that's dumb or why we shouldn't do that one and things like that's the convergent part of the diversion conversion thinking and you'll notice that they follow a well-written problem statements fall a certain pattern they all start with like an action verb check file maximize reduce that's because it's actually doing something for the user it's creating value like it's an action verb right the second thing is it's written from the user's assignment from the company's perspective or the procs perspectives written from the users perspective like my taxes my deductions my audit risk right so it's kind of like a well written agile user story you know it's kind of like written written along those lines and again you want to peel the onion right now to peel the onion and come up with all the different things that you could do and when you do that it's messy as we saw here right we saw how messy it is you guys gave me like 10 different answers and some people gave me two answers right huh and they're like an onion makes you cry I need the little goggles yeah but but basically why is that well basically one thing is you might be fundamentally talking about two different benefits someone's talking about it's cheaper someone else is saying it's faster there's no way they're ever gonna be just the tooth the same thing right but even people talking about faster they might use different words like quicker faster easier convenient they're like synonyms or kind of closely related words and that's what happens when you run this you'll see that there'll be clusters of related benefits and there's a little bit of structure that you can find so let's take these benefits help me prepare taxes reduce my auto risk check my return if I use that five why's technique on people that said this is why they like and what they hope to get out of TurboTax with the cataracs I said why is that valuable to you why is that important to you what we're trying to do is get them to kind of up level and move up to a higher level story and we might find that all these things have to do with empowerment or confidence a couple people said here hey I don't know the tax code is complicated right the story might go well you know before TurboTax I'd have to fill these things out by hand I'm really bad at math I don't know the tax code it's really intimidating I'm sure I'm throwing this thing out right it's very it gives me a lot of anxiety because I'm probably not doing it right but instead using TurboTax to just ask me a few questions and holds my hands the next thing I know I'm done with my return right that would be an empowerment confidence right we call that a benefit ladder because you basically are getting people to climb up and all three of those are on the the empowerment confidence ladder there could be a completely different benefit ladder that has to do with saving time has nothing to do with confidence right saving time preparing saving time filing there could be a third completely different ladder about saving money rights maximize my tax deductions so this is what I mean by a problem space definition is you want at least you don't even have like three levels or four levels just one level that's fine where it's like yeah we brainstorm the detailed problem space statements and then we just kind of came up with what the clusters are you'll notice this whole time I'm talking about TurboTax I've been talking about your tax I haven't mentioned a single feature I'm talking about the problem space the whole time it just so happens that they do have a feature that's one to one mapped with these problems which is what you want what do you want to avoid is you don't want to have a problem that doesn't have a solution that maps to it you don't want to have a solution that doesn't many problem map to it that's just somebody decided to build that for no good reason really and and a side benefit is basically that if you have this clear one-to-one mapping and your feature is well-named the side benefit is a user just by seeing the name of the feature you can pretty much figure out how it's gonna create value for you right so that's basically kind of what I encourage people to do in the problem say it's actually kind of fun because you get to brainstorm and come up with a lot a lot of ideas the next thing if you take my advice is great game we've brainstormed dozens of potential needs that we could address we obviously can't build them all so the next thing that comes up is how do we prioritize them right and that's where we're getting at the underserved part of their needs and that's where the framework that I like to use that I developed is called importance versus satisfaction let me explain that to you so basically importance is for each of those needs for each need how important is it to our target customer from low to high and you can think about this two ways one is like imagine we did like a formal survey with thousands of people and we said hey please rate it on a scale of one to ten and the average is 8.6 like you can get it really precise that way but you can also just use it as a thinking tool like low medium high within your team hey do we think this is low medium high importance as particular needs so you don't get wrapped around the axle because how would I measure the data it can be used as a thinking tool the other axis is how satisfied are people with how they're getting that need met today right so this is in the problem space this is in the solution space again you low medium high or you can think about it like a formal quanta date of survey right and what we want to do is basically think about where each of these needs are to talk about to try to talk about the space let's just divide it into four quadrants so in the bottom left quadrant we have a low importance of user need we have low satisfaction with the current alternatives the bottom right quadrant we have low importance of user Nate we have high satisfaction alternatives at the end of day neither of these are worth going after right why would you spend your precious time and resources going after a low importance need people don't do this on purpose they do this because they think it's important they think it's gonna solve a problem for the customers or they're not even thinking about customers or and they're not using these techniques and then they launch it and then they realize oh geez what we thought was important customers actually don't think is important right so by using this technique you can avoid going after these two quadrants the upper right quadrant we have high importance of user needs that's better than low importance but we also have high satisfaction with current alternatives it's an important need but people are pretty happy with how they're getting it met today that's the definition of a competitive market right and you hear people say hey you need to be like 10x better than the competition this is where you need to be 10x payers in the competition you can't just roll out some me-too product because people are pretty happy the first thing I think of here is that Google search if I said to people hey how important is it you know scale want to tend to find the information looking for online people would probably say it's pretty important but it's not like people walk around going gosh I can't find anything on this Google it's just not working for me right he's pretty much fine what you want maybe you revised your query once or twice but but it works right and in the book I talk about a start-up to actually try to take Google on internet search and failed and in the last quadrant we have high importance of user needs so that's good and we also have low satisfaction with what's out there today that's where opportunities lie and they may not be there forever because we live in a you know competitive environment everyone's trying to innovate and find problems to solve for customers but if you do the analysis you can find these opportunities one of the one of the market opportunities I think that lives up here is ride-sharing basically right companies like lyft and uber have done a lot to get to the level of success that they've gotten to but I think one of the main reasons is they fundamentally address an upper left quadrant need especially here in the city I remember when I used to try to get a cab all right you'd call okay I'll help you try to call they say yeah it'll be there in 30 minutes it may be there in 30 minutes maybe 40 maybe it won't show up right so you can imagine if you did customer discovery interviews that people say hey I think you're the last five cab rides how satisfied were you with that right well let me start out with importance how important is it to get to the airport to your flight on time at the airport forget how important is it to make it to that job interview it's pretty important that date whatever it is it's pretty important to get where you need to be on time and if you just said hey let me let's talk to you about your last five cab rides how satisfied were you and we can break it down and say you know hey how satisfied you were you with how punctually the person showed up or if they showed up at all how safety did you feel how clean was the car how polite was the driver how convenient was it to pay the person right you can imagine low score not all cab rides obviously but some of them you can imagine those scores that's an example of an upper left quadrant Nate basically and what I like about this framework is it's meant to be a visual framework so what I'm trying to say is if there's a certain product represented by this red dot that addresses a customer need that has that much importance and it does so provides this level of satisfaction what I'm trying to say is when you plot it with its corresponding importance of satisfaction the rectangle that's formed is a proxy for how much customer value that that product creates and if there's another product that addresses a higher importance user need with the higher level satisfaction then it just creates that much more customer value and when you view the importance of as a satisfaction this way you realize that the opportunity to create customer value is just how much room is there to the right of where the market is today all right because that's we can just increase satisfaction and that's why the upper left quadrant offers the biggest opportunity because it's got that most area to the right to add more customer value and if you have an existing product here you can apply these techniques to move move it to the right increase the customer satisfaction increases the customer value that you're creating now if these importance versus satisfaction concept seems abstract and fuzzy or theoretical let me show you some real data so this is more in the quantitative survey use case less the thinking low medium high use case I was a product manager for product we had lots of users so we're able to survey them we had 13 key features that we really cared about and so we asked we surveyed our users and asked them to rate the importance and satisfaction quantitatively so now we actually have instead of just low medium high we have numbers going up to a hundred percents on both importance and satisfaction and each of these points is one of those 13 features plotted with its corresponding importance and satisfaction numbers and the number to the right is just the satisfaction number so you can more easily see what it is so as a product manager for this product the first thing I noticed was this point I'm like hey customers are saying it's 100% importance and we're they're telling us they have 98% satisfaction like great job team my second thought was I'm not gonna spend any of my development resources on that because it's already fine like let's go find something else that's gonna create more value right and we had a cluster of different things in here we had this one down here but this one is the one that was most up and to the left right it was pretty high importance and relatively low satisfaction so we focused on that one and moving that one up to create the most customer value and a couple years after he left into it I was really excited to come across the book what customers want has anyone seen that book what customers want so it's a thin book by tony all work and he also has his own kind of quantitative framework for importance and satisfaction so it gave me a lot of confidence Iwaki i came up with that he came up with that there's there's something here about importance versus satisfaction if that idea about getting rigorous and quantitative appeals you I recommend checking out his book all right so that's how we identify underserved needs the next step is great let's get clear on our value proposition which again is ok out of those potential under survey's which ones are we actually going to tell people our product delivers on and how are we going to be better than the competition and the framework I like to use here to get clear on our value proposition Oh models anyone heard of the Kano model few people aren't great so I was I learned about the Kino model before I moved to Silicon Valley and went the business go actually got a master's in industrial engineering where I studied lean manufacturing and that's where I learned about the Kano model and it's a great tool to get clear on your value prop and thinking about your product so let me explain it real quick on the x-axis it talks about how fully does the product meet the need that we're talking about from not at all it doesn't meet the need at all to it fully meets the need all right that's the x-axis the y-axis is based on how much the product meets that particular need how much customer satisfaction or dissatisfaction is created right and if it seems a little complicated don't worry there's only three different categories within the Kano model so it's really actually easy to get your head around the first is a performance benefit or feature right the more the product meets this need the more satisfaction it creates the less the product needs to need the less satisfied the customer is right so more is better less is worse if we were building like micro processors for computers and ours was like 15 percent faster than the other guys chip speed is you know a performance benefit for micro processors let's stick with cars let's say I was shopping for a car and there were two cars that were pretty similar you know they look similar their specs were similar but all sudden I realized that one of the cars had like twice the miles per gallon as the other car I would probably pick that car because all our things being equal you know the fuel economy is a performance benefit for cars I'd rather pick that one right so so that's performance benefit the second one is must-haves now having a must-have you can see it doesn't when the product fully meets the must-haves it doesn't make anybody happy but not having the must-haves makes them unhappy that's what this curve is saying basically right so sticking with cars say I was buying a new car and I walk into this room and I just see this car and I just love the way it looks and then I read the spec sheet on the window and I'm like wow these specs sound great this is like the perfect car for me and then I take a peek inside the window and I realize hey there's no seatbelts in this car I wouldn't buy it because I'd be afraid of getting hurt or dying right so seatbelts are must-have for a car but if car a has 100 seat belts and car B has 5 seat belts I don't see as 20 times better I want you have one seat belt per person your find out basically right so that's the must-haves and the third category is des lighters so not having the delight or doesn't cause a problem because people don't expect it to be there but having a delight or can create a lot of customer satisfaction we call that like wow features basically right not today but sticking with cars when the first cars came out with GPS navigation it was a delight ER right before that people were printing out Google Maps or asking for directions or getting lost and in the first couple of cars of GPS you just put in the address of where you want to go and it just fundamentally changed how do you got from point A to point B but as we know over time more and more cars got GPS navigation Garmin and TomTom came out with their add-on units and now we all just use our phones right so this is not a static picture so there's needs and features migrated over time right so that yesterday's de lighters become today's performance features become tomorrow's must-haves and the pace with which that happened just depends on the level of competition and innovation within your space but tying back to our value probably want to do is think about these three categories of benefits right it must have performance and blighters and use those to categorize and get clear on our value problem and again our value prop is say out of all the benefits that we we talked about which ones are we actually going to tackle with our product and how we're going to be-better different than our competitors and the way to do this is basically to create a grid and then for your product category I left a generic on purpose list one per row what are the must-have benefits that apply to our product category what are the performance benefits and whether the delight or benefits right that's step one step two is you want to create a column for each of your competitors and create a column for your product that's step two step three you want to basically rate or score your competitors right and it can be again it can be low medium high if there is some quantitative measure you can you can use numbers if it's amenable to it but you can also just use low medium I so like this is our competitive landscape let's say you know we have two competitors they both do the must-haves of course these guys competitor a are the best at performance benefit one competitor B is the best performance benefit - they're both so so at performance benefit three these guys have their own delight ersite that's the kind of competitive analysis backdrop that we want to figure out what our value prop is this is basically the essence of product strategy and I honestly think like less than ten percent of our teams go through an exercise like this to make sure they're clear on how their product is going to be better or different right and one of my favorite definitions it's super easy to go hi-hi-hi we're gonna be high we're gonna beat everybody to everything it's super easy right one of my favorite definitions of strategies that mean saying no and there's a great Steve Jobs quote in the book about you know strategy means saying no like it's you know it's just saying yes yes yes to everything is not strategic so you got to pick your battles you don't have the resources to be to beat everybody on every dimension and even if you could it would not be a well positioned a clearly positioned product right so given this backdrop we might go with a value proposition like this of course we're gonna have them must have we're gonna do some somewhat on performance benefit one but we're not gonna try to beat these guys we're going to say no performance benefit - we're not going to worry about that for now and we're gonna be the best at performance benefit three maybe we've identified to market segments that really values that performance benefit maybe we have some ideas on how our solution and technology can provide better levels of satisfaction and then we have our own delight err what matters the most from this exercise is what's called your unique they're French hairs it's what's the performance benefit where we're gonna be the best and what's our unique delight errs right you don't compete on must ABS alright and what we want to do is we want to get clear on what are these before we go into the next step which is our MVP because if we build an MVP and it doesn't have these things in it what the heck are we testing with people right that's what we want we want to test these things in our MVP to see if it resonates with customers so I thought to bring this more to life as anyone used Instagram here yeah alright a few people so if you think about this must have the lighter benefit like let's just talk about what what what you know when Instagram came out they're already a bunch of other photo sharing apps out there right so what was it for those of you that remember back then like what was it that made them apart like made them better than the other ones yeah filters is one and as I think is one other thing I think the other guys would share too but maybe they did a better job on that so sure yeah I mean the other ones were social too I think but maybe they were more social yeah did anybody use the other ones do you remember whether these guys took the same amount of time or were faster they were actually they were actually faster because they did this hack where they would like to start uploading the picture right away before you even push like uploaded or whatever they kind of did some technical hacks compress the image so part of what they did is like they literally the amount of time it would take to share your photo was less because they were doing these technical hacks so most people say that an Instagram one and was better for two reasons one is the upload or your pictures more quickly and secondly they did filters which is what you say if we go back to our perform must have performance benefit de Leiter's uploading pictures more quickly what would that be performance that's a performance benefit right how about filters what would filters be a delight er right so it's very common you know if you if you look through this lens and you analyze products that are winning in the market very common pattern you see is there's a performance benefit where they're beating everybody and they couple it with the unique delight ER and that's a really powerful comment you can't always have it even if you know if you're just outperforming somebody that's good but if you can pull the one-two punch like Instagram then it's really good so anyway it's just a way to kind of bring that concept to life hopefully for you guys alright cool next step is great we're clear on a value proposition what's let's get clear on what our MVP feature said is you know what's an MVP this again this is probably the most hotly contested topic out there what do you guys think an MVP is go for it it's viable okay that's the V yep viable foundational of one of a feature product cool yeah mm-hmm whatever you can test right right all right cool yeah so I think those are capturing a lot of the points there I see two main mistakes that teams make with MVPs that I like to just talk about one is the whole point of an MVP is to not do it the old way where you had like a team of engineers and they would go off into a cave for 12 to 18 months and code and then come out and then launch their product on the world and then realize nobody wanted what they just build right so the whole idea is how can we like before we invest too many resources how can we like not over scope things and make sure we're going in the right direction and to make whatever course corrections we have before we invest a lot right so even though MVP is intended to keep you from over scoping your v1 I see people over scope their b1 all the time I've even had people read my book say Dandan I read your book totally get MVP but here's why my MVP needs to have a ABCDE and F in it so it's almost like this thing we need objective third-party eyes looking at it and asking you tough questions there's almost always 80/20 at play and it's easy it's human nature to be like oh geez someone may complain because we don't have backs we better put that in the MVP someone may complain you learn otherwise let's put in the MVP so again it takes that willingness and strength to say no to do that the second mistake I see people make I like to explain what this framework so is anyone familiar with MailChimp so I don't know I use MailChimp when it first came out and I remember it just head and shoulders UX wise above all the other products that were out there at the time and Aaron Walters was the head of UX design there and he wrote a great book called designing for emotion and a designer named Jesse pissant and modified his framework that I modified it so I just want to give them props but basically he has another pyramid where he talks about how functional is a product how reliable is a product how usable and how delightful is it right these four attributes so we can kind of score how much how well is our product doing on each of these four levels and I was at a workshop once and someone use the acronym derf so it calls the derf pyramid basically so I was like okay it's dirty they don't want to rewrite it so they just put D URF but basically the idea is you know we can color in how much at any point in time how much is or how reliable functional usable is life is our product and color in right and obviously with an MVP you're not going to color the whole pyramid in one fell swoop it's gonna MVP it can't be right but what I see the mistake people do with MVP is they use them if you use an excuse to do this so they don't build all the functionality which is great because you're not you have to build a subset but they use the MVP to ignore reliability and say you know it's okay if it's buggy it's just an MVP it's okay people you know for the UI will do the UX design later will make it easy to use later it's okay it's just an MVP right so that's frankly people misusing a not understanding MVP and a lot of times this happens the organization's trying to transition from waterfall to agile they just kind of get don't get it and then they do that how do you think those MVPs tests with customers who Nihad and test the customers they don't test well it doesn't matter if you have the functionality if it doesn't work when people want to use it or they can't figure out how to use it you're not going to get credit for the functionality right and those Emmys are gonna fail and then the people guys this whole lean agile thing let's go back to waterfalls not working out with this MVP stuff right so it's true that you want to build a subset of the pyramid but you want to build it more like this right so you're gonna build a subset of functionality but the subset that you build you want to you want it to be reliable enough I'm not saying is gonna be perfect but reliable enough and usable enough and delightful enough so you can truly test your value proposition you really trust your unique differentiators right so that's kind of that's that's the other mistake I see people make and this this kind of diagram helps people explain it to others so I don't have time I have a lot more advice in the book about how to break features down and decide on your MVP have time for that today creating next thing is once we get clear on the feature set now we need to create you know we need to do the UX design we need to do the UX design one because we have to do it for the product anyway but secondly I think it's a great way to create a prototype that you can test with customers so for the first time so we basically one two and three year old in the problem space with four we went into the solution space but usually when you talk about features it's like words it's words in a Google Doc or words and confluence or a PowerPoint it's words or a Giro ticket right so this is where we go from words to a user experience right so they're worth making that transition and when I think of user experience design I think of an iceberg because just like an iceberg it's very misleading because there's only a part that sticks above the water that's obvious but there's a lot more under the water you can't see but that's really going on right that part that's above the water is visual design right we all take in information visually you guys are looking at my slide you looking at me look at the colors the fonts the shapes right that's how we take in most of our information maybe a little bit more through our ears when we're listening but mainly if your eyes right but when you use a product that you know is intuitive and easy to use it's because the product team has done a really good job not only at that layer but at these more foundational layers and these layers build on each other and I don't have time to get into it again but in the book I have a whole chapter on UX design because I think it's so important I don't expect product managers to be designers but I think it's important to know a lot about design I learned that early in my PM career to more effectively partner with designers and developers but it all starts with conceptual design which is like what's the fundamental concept for how we're going to approach the design of this product information architecture is how you structure and organize your your your product if it's difficult for people to find things the labels don't make sense then someone hasn't done a good job here if conversely things are where you expect them to be and it's kind of intuitive to find things they've done a good job interaction design is literally how the user and interacts with your product anything they can tap or click on controls forms buttons navigation that's interaction design when you're doing design again we're going from words to pictures if you will or drawing designs there's a range of different design deliverables or artifacts you can create you know it's probably not best to go straight from words to coding that's a big hop right it's probably not the best also to go from words to high fidelity mock-ups and then code so this is also what I kind of view as the best practice of how do you break up the process of going from words to live product in a way that you can iterate and test so I like to categorize the design deliverables by how interactive are they like compared to the final live product how much can I tip and tap and click on them right and then fidelity which is compared to the final product how closely does resemble how much you know the look and feel and how pixel-perfect it is so in the bottom left lowest interaction low Fidelis fidelity we have the hand sketch whether it's on paper or on a whiteboard right I wouldn't show this to customers but it's a great way for you and your team to kind of make that first step from words to pictures and and then you know iterate until you get to the point where you're happy with it the next design deliverable that I recommend is clickable wireframes so now they're higher fidelity because we're putting it into a digital wire framing tool they're hiring activity because you know in the old days they would actually be static wireframes but nowadays the tools make it so easy to make it clickable or topical wireframes you have no excuse you just have no excuse just suck it up and learn how to do that you just have to as a p.m. when I interview a p.m. and they're like I don't why you're a MIT's just it's not a good not a good way for the interview to go house whose use balsamic before all right cool so balsamic makes it super easy right it makes it really easy to do and you know hey any of the controls are baked in okay when I click this it's going to go here and it's gonna go here so you can like string together a multi page or multi screen experience and that's a great tool for you as a team to work through it's actually good enough to test with customers the cool thing about balsamic is you can only get so high you can't get hi-fi right then for the longest time the lines were jagged and there were so many OCD people said can you just make it straight did they finally put a straight mode I don't does anyone know about the straight mode in balsamic okay so defaults to the jagged line mode which is visually conveying to people hey this is a prototype this is rough but there's a secret is a hidden setting where you can make all those lines straight if you want but anyway you can test with customers and you know you're not going to be testing the visual design or things like that but you can test one of the best things you can test is in that MVP discussion you might have had a tough decision in your team about should we include feature a or not and you might have had the courage to say let's try it without feature a well this is when you're gonna find out if people like have a problem with feature a not being here not they're gonna interact with your wireframe and they may go through it and not say anything about feature and you go cool we got it right or they may say are you crazy I can't use this product without feature a in there so better to learn that then than waiting until you finish all these high fidelity mock-ups and have to redo your whole design right you can also learn about information architecture interaction design here once you you know I recommend talking to customers in ways of five to eight once you record Winston repeat and iterate to the point where people are relatively happy with the wireframe then you can proceed to the next step which is clickable mock-ups right so clickable mock-ups you know they're interactive or tappa below their higher fidelity all right mock-ups now we are worried about we now we are paying attention to that tip of the iceberg in the visual design right so I forgot to mention wireframes are typically grayscale on purpose all right the number of times I've seen a product team you know bringing in their initial designs to some like key stakeholder and the first thing the stakeholder says is I don't like this red it's like that's not what we're trying to talk about right now I was like do we have the right feature set that we have the right layout right so you can't to avoid those kind of people can't help but fixate on visual design right so to avoid that you just don't put any colors and you don't worry about the fonts you don't put any images in right you worry about organization copy it's great to have real copy in there any way has anyone use envision so envision is a great tool usually the way you know mock-ups the designer will say great Dan here's your you know zip file with your 50 mock-ups in it right and it's like what the heck am I supposed to do with that well the cool thing with envision is you can upload those to envision and then you know in balsamic the controls have built in click you know kind of click path so you can specify the way it works in envision is you overlay a rectangle and say ok I'm gonna put a rectangle over the sign up button and when the person clicks on that hotspot rectangle then it's going to go to this other one and go to this other one so it's rated way to string together again a set of high fidelity mock-ups and you're not going to be able to express every path but you can express what we call the happy path where you expect people to go and this is a great way I mean at this level when you've got clickable or topical mock-ups that are high fidelity that's a great place to get customer you backhand is a lot less expensive than doing it with like live code and having to go back to dev and change the code and things like that right so again you want to talk to groups five-date get to the point where they're happy with it and then you can proceed confident into your live product right so this is words about features to live product these are kind of the intermediate steps that I recommend that people go through I do recommend that you test your live product because you know these don't have any browser compatibility issues or database performance issues or anything like that sometimes when you send stuff into dev it doesn't come out exactly the way you wanted it to so it's good to test it but that's the way to de de-risk work your way through the design irritably and and do you risk it just to be super clear just a wireframe or a walk-up it's a wireframe sometimes people mix up the term so why your frame is reserved for lower fidelity this is actually a balsamic wireframe you can see the cool thing about balsamic and all these tools is they have widget and stencil libraries you're not creating things from scratch you should just be like okay I need an iOS phone I need a button I need this I need this I need this right does it have any colors no does it have any fonts no does it have any images no we can't get into stupid disagreements and arguments about that because it's not there we have to focus on what's on the screen are we missing anything on the screening should it be there's at the right size or not right here's a mock-up of the exact same app well now we do have colors we do have fonts we do have pictures right so the whole point is work on this first don't jump to this then I'll just tell you this many of you may know this already is a big difference between visual designers who do this and interaction designers who do this and these days we call them UI designers and UX designers all right and I can tell when I work with the team and the first thing they show me is high fidelity mock-ups that means they have a visual designer and they don't have interaction design skills on their team right and so what usually happens is they haven't thought through the information architecture the interaction design so anyway all right once we have our prototype then or our live product then we can go and test with customers right and just again a tie it back now we've worked our way all the way through here and we've got this UX prototype that UX prototype again embodies all the assumptions and decisions we've made throughout the whole process and now we want to close the loop so it's actually important one other thing too is to make sure you closed-loop with actual target customers not just anybody on the street or random people because you want to kind of close the loop and see you where you're at with product market fit I just want to close out with end and case study of it being applied in the real world people have said this is super helpful to kind of kind of bring it make it make it real for them the project was marketing report calm it was a consulting client of mine it was a start-up the CEO hired me they had an existing product already but he had an idea for a new product basically right and he wanted to test a test out this new product idea and and a very small dev team so he's the one that said hey Dan we can't code anything you have to do as much validation as you can without coding I was like great I love doing that we're gonna use clickable prototypes we're gonna do this and he wanted to see if there was you know a business opportunity worth pursuing there or not the idea was a marketing partner con who gets who gets junk mail here anyway yeah so the nice euphemism for junk mail is direct mail but basically the idea was you know say I come home after this talk and I go to my mailbox and there's a coupon for cat litter for me why did somebody mail me a coupon for calor because in some marketing database in the cloud somewhere it thinks dan olson has a cat that's why right and so the idea was to give people control and transparency the best way to explain it is probably by analogy through the credit report stuff right so today you can get your credit score's you can get your credit report if something's wrong you can you can fix it right but you know 20 25 years ago you'd apply for a loan you put in your social security number you'd go off into the cloud database and it'd be like okay oh no sorry you didn't pay your bills on time no you're denied for that loan and you wouldn't really know why so that was the idea it's like to give you transparency why are you getting the junk mail you're getting and help you kind of you know potentially fix it or make it more relevant that was the idea so I said ok great sounds good by the way the chart customer as you saw was pretty much everybody there's a broad customer offer step one is who's your car customer it was a broad consumer offering anybody they got junk mail in the US right so I said great let's get clear on our problem space and let's talk about what you know some of our our top feature ideas are think key problems face idea was basically learn why I received the junk mail ever see like why am I getting this junk mail and the top MVP feature ideas that they had we're a marketing report which was analogous to a credit report it basically consisted of a marketing profile which like all those different attributes that they have like okay you have a cat you're married you have kids whatever it is and then a marketing score which was meant to be like a credit score some numerical measure and higher is better and that was that was the core idea and then one of the executives said hey in addition to that I'm interested in exploring money-saving offers what if you know I send you a cat coupon and you want to say well you know I don't have a cat I have a dog can I opt in for dog food coupon can I opt in for dog related coupons so can I opt in for relevant money-saving offers he wanted to test that he had hypothesis that people also wanted to compare their shopping habits and how much they spend and things like that with other people so that was that box and then because social networking was hot at the time social networking it's a pure like you know key stakeholder slamming a solution into the prod into the in the family all right cool that's fine we're clear on what exactly why people would want that but sure the other executive didn't care about any of the blue stuff he cared about the green stuff and he was also interesting sporting like hey what about just the override you're just reducing the junk mail like let's just test that idea and then we brainstormed a secondary benefit hey if we're really truly reducing a bunch of junk mail maybe we can make some claim that we're saving trees and being environmental because it's a lot less paper going around right so I ate it to get this to this point and we got to this point I said okay is everyone's idea on the board and they said yes and then I said okay cool now it's time to figure out our MVP and I looked at this and I said this is way too big for one MVP this doesn't weigh too much stuff so what I did is actually created two MVPs I created one again the green stuff is the core idea they just wanted to test the secondary stuff in addition right so I created one MVP that had the core green stuff plus the yellow and we call it the marketing shield because it was kind of shielding you from junk mail right so we that was our MVP feature set for that and then we created a set of prototypes for that the second MVP was the green stuff plus the blue stuff and we that was the MVP feature set and we create a set of prototypes obviously these were the same kind of this was similar between the - but these were different then we proceeded on to doing you know UX design this is an example of the prototype we created the screen we've created I call a medium fidelity you know it's a little design but we didn't really spend a lot of time obsessing about visual design when you clicked in then you got to this main page where there would be like a module for each of those features from the MDP feature list there'd be a module and then you would click down to learn more and that's where we'd go to like a whole screen where you would interact with that feature so that's how we didn't got away with not having two completely different UI designs and we would just mix and match the modules and then if you're in the saver one and have the same our modules within the shield one have the shield modules so how that we had our clickable prototypes we recruited people and I moderated them to the park types what did we learn here's that same diagram now color coded with red yellow green when I ran it through with people there were tons of questions and concerns and comments right they really weren't super excited about it so green doesn't mean like oh yeah they're ready to sign up green just means like well we feel like we have a good handle on the questions and concerns that they raise and that we can address them and that if we address them we think there's you know some underlying value that can be realized there that's what green meant yellow meant you know wasn't really too appealing and red meant like they just didn't want have anything to do they didn't get or didn't want it right so the first thing when you do this is do you have any green so luckily we had some green just and there's no guarantee the first time especially the first time you do it you gonna get green so we had green here and we green here what was the core idea that we were testing it was this do we have any green in the core idea that we were testing no so it's like thank God we tested something else right that happens all the time you get out there and talk to people and say you know I really have you know what tell me about that about this when they're like well that's not really important let me tell you about something else that happens all the time right the second thing is also the red you know the red is just as valuable like I'm not surprising the thing that was bolted on didn't play well right do you guys know what a marketing score is really I don't either right it was ill defined I didn't bother going down that path until we tested but I know what it would have taken it would have taken us going and signing some expensive third probably data licensing agreement to get all this data on people it would have taken hiring and you know some smart engineers to develop these algorithms and figure out this whole score thing it would taking a lot of time and money to educate consumers about what the heck of marketing score was and why they should care about it but guess what because people didn't like it we don't have to do any of that work and waste that resources remember back to the unimportant stuff like to avoid the unimportant stuff so we avoided it and you hear people say you need to pivot well here's what we had a pit we were here I mean I decide are we gonna pivot to that green are we gonna pivot to that green right and this happens all the time just by getting out and talking to people you'll learn about adjacent opportunities that are more important than than what you think you originally go out to talk to them about for example slack I'm sure a lot of you guys use slack did not start out saying we're gonna build a communication platform right they actually started building a game and they were you had a distributed team and they were using slack to communicate and collaborate and they decided that that was a much more worthwhile product right Flickr same thing it didn't start out as a photo so numerous examples of you start out one path and then you realize by getting in market and talking to people that there's a slightly different path that's better so we had a pivot we got a pivot here here we decided to pivot here for three reasons one as I mentioned the company already had an existing product and brand this was more consistent with that secondly money saving offers would have taken a lot of time to go out and find partners and do biz dev deals and sign those contracts we've taken like 12 18 months which would have killed our time to market and also there are already a lot of people doing money saving offers so back to the value prop we weren't really clear on how we could be better or different so for those reasons we pivoted here and because we had done no coding we had no qualms about just tossing those prototypes and starting with the blank sheet of paper right because we hadn't any coding and so we started from scratch and what we did is we took the six pages of notes we had from the shield sessions and made sure the new prototypes addressed all the questions and concerns that we learned so we pivoted to junk-mail freeze so marketing report tom was gone here's a new prototype of the new homepage that we did right and I learned a lot of stuff by talking to customers by exploring the problem space with them I learned a lot of stuff I learned that not all junk meals the same is like peeling the onion right here we said reduce junk mail well it turns out not all junk mails the same some junk mail is more annoying than others to people and what we found is the most annoying junk mail generally was the financial related junk mail like people are getting cash advance to pre-approved cash advance checks pre-approved credit card offers and they basically said I really hate those and a lot of people they lived in houses that just had mailboxes that weren't locked and they said well I'm just worried that somebody's gonna come steal my mail and use my cash advance check and defraud my account or take my identity so we learned that so we put that front and center and the second time around you know people would like home right in on that you could see their temple kind of start going either vain going because they were getting angry just thinking about the possibility that somebody would do that so it was very effective other things I said he came before I showed them the prototype or the cake can you tell me about you know what you do with your mail today like let me tell you what I do Dan I go to my mailbox I grab my stack of mail and I go straight to my paper shredder and I go shred shred shred shred I'm like well how long are you reading like five minutes a day that's 30 minutes a week that's 1500 minutes a year so there's a whole save time benefit that we didn't even have in our promise taste map that we learned about so we put our here spend less time shredding mail right so that's an example of how we took the everything that we learned and we've put it into the new concept silly things we said we're gonna save trees remember people said how many to multiple people like well how many trees you're gonna save we put in 43 trees just like we didn't get that question we didn't get that question anymore right yeah yeah right right you could you could pick your tree right all kinds of sand there was other stuff too about like you know I'm not showing here about like UI for how you configure which mail gets through and Mitch will doesn't it's like hey you got to block the catalog because they clog on my mailbox but my Pottery Barn catalog that's got to get through like all these little things like that right and both times I didn't say this but both times we asked people at the end hey would you pay $10 a month for this service and it's always though if he'd ask people if they would pay if they don't actually have to bust out their wallet and pay but we saw like a night and day difference the first time around nobody was going nobody had any interest in that the second time around what they said as well I'd want a 30-day trial of your product but if it does what it says here I would gladly pay you 10 bucks a month was like a night and day and then the other thing that gave me evidence that we had really kind of improved product market fit by iterating was after the test was done I'd be like great thank you so much for your time here's your check for $100 you know we're done right almost every single person after we gave him the check and the test was completely done in the head every right to leave was like so is this product live now can I go use this like no no we're still building it it's not quite ready yet so can I give you my email can you email me when this thing comes out like nobody was doing that the first time around so I was like yet another kind of indicator that we had achieved more product market fit so I was pretty excited about that so to sum up again lean product process start out by getting clear on who your target customer is use the importance of satisfaction to understand which of their needs are underserved use the Kano model to get clear on your value proposition how you're going to be-better different than the competition figure out what your MVP feature set should be so you don't over build before you course-correct the direction that you're going in go through the UX design and create a prototype again if you can if you can create a clickable tableau prototype without coding then great you'll be able to move faster get out of the building and test with customers and basically iterate go through that hypothesized design test learn loop with the goal of you know improving product market fit as you iterate so as I mentioned I run a group down in Mountain View we have monthly speakers the next one is Tuesday December 12th give biddle friend of mine he's the ex former VP of product for Netflix he's going to give a great talk on product strategy and how to translate it into execution if anybody's interested you can go there and the other thing is I occasionally like two or three times a year I give public workshops that anybody can come to I'm actually giving one on November 30th in the city I've created a special 20% off code for P school for product school if anybody's interested if you have any questions you can ask me about that but basically we go all day with group exercises and really get into it so with that I'd be happy to answer any questions that anybody has yeah mm-hmm maybe B's too hard forget about it yeah so it's funny cuz I get a lot of times I'll get that question and it's sometimes it's worded more strongly people say oh this doesn't apply to b2b this doesn't apply to enterprise you asked it more like what's different and when people say hey it doesn't apply like well don't you have customers don't they have needs don't you have competition so you do so there's more that's high level more that's similar than different what comes up with enterprise is you know certain techniques may not be as relevant like like a beat I didn't real actually mention did I mention a/b testing I didn't this is a very qualitative approach in the book I do talk about a/b testing but that's more later after you launch but a/b testing may not be as amenable right if you've got you know ten important b2b clients you may not be able to do a DBA be testing whereas if you're Facebook it's like hey let's let's take 2% of traffic and a/b test and see what happens and get your results in a few hours right so a/b testing may be harder for you to do the other thing is other techniques you me not to be able to you may not be able to survey as people as much right I didn't mention using surveys either so it doesn't impact that the thing that people tend to complain about is the avail access and availability to customers you know basically when it's b2c it's pretty easy to get people if you're targeting like fortune 500 CMOS and it's like hey can I meet with you to show you my prototype they may be more reluctant right so that's usually one of the material differences and a lot of times you may have blockers and gatekeepers like sales or customer success keeping you from talking to those people so but you got to figure out some way to actually get to some of the tactics that are used there's like their customer advisory counsel or customers opt in to be pestered with questions or put in what the other people would think right they opt-in so once a month once a quarter you have a meeting you have a few hours with them and that's when you get feedback on your prototypes and your ideas the other strategy to go is to go with the private beta and like a pilot you know where you're going deep with one or two clients to really work through the MVP and the UI design before you roll it out more broadly like usually those people if you guys have red crossing the chasm there's like the difference between like the early adopters and later adopters you want to find the subset of your customers that feel that pain so much that they're willing to jump in the foxhole with you and and the care do you have for them is hey you help us co-create this if you're co-creating this new product it's gonna really meet your needs and there's an opportunity there a lot of times they'll also get grandfathered in on lower pricing or free or something like that so those are some of the differences that I see conversely the willingness to pay is usually easier to assess because if you're sawing being a real business need they will they will pay you or is the consumers it can be kind of harder maybe you're monetizing through other means like advertising or something like that you're not sure so those some of the differences that I see yeah that's right that's right yeah yeah yeah yeah so I would actually recast it the question was hey a lot of user testing I've seen is very task focused but how do you assess for delight I would rephrase it as the way I I don't have slides here but in the all-day workshop I talked about the distinction between usability testing and product market fit testing that's how I would call it right and basically like old-school usability testing is literally focused on like task completion rates it's like okay we've got a product to let you book flights we're gonna like put on our white coat and get someone to come in and say please try excuse me sir please try to book a flight from SFO to descend to Dallas on this day and we just see if they can do it and we just see they're you know five out of ten people did we had a 50% task completion rate that's like hardcore usability testing I don't think that's as valuable for what we're talking about so what I'm talking about is just more of a blend yeah you'll get usability feedback and I didn't get into time and to share my advice on how to conduct these sessions but they're much less directed I'm not telling people what to do right marking report calm right they're just going there and doing what they would do right I mean that like junk junk mail freeze calm you know I'm not sitting here going okay try to out to try to sign up try to do this try to do this like if I do I'm kind of pushing them through the wickets it's not a real test right what I want it to be is as realistic as if no you just this is in your browser a friend you said hey go check out junk mail freeze calm so my mean hack for that is it beginning of the interview when I show them at the point of the interview when I show their prototype there used to kind of being mice and a maze and I'll show it and I just shut out I'll just be quiet and it's so awkward but then they'll be like so what do you want me to do and I have a pad answer it's well do whatever you would do if you're home by yourself and I just shut up again and then sometimes almost always would be like so so should I register and what do you think I say do whatever you would do if you're home by yourself and then you realize okay Dan's not gonna hold my hand and push me through the maze they realize it and so then that's the number one trick it's not about giving them a task and seeing if they can do it it's about getting their reaction to it you'll naturally get there are usability issues they'll naturally emerge cuz people won't be able to navigate or figure things out right but it's a better way of assessing that product market fit so that's kind of how I and again in the book and then the workshop I just talk about some of those techniques on how to ask quite a lot of the thing too is how you ask questions it's very important how you ask questions it's really easy to learn how to ask questions the right way any other questions yeah huh yeah I see I see that's good well that's a good it's a good quite let's just pull up the pyramid so we can get clear about what we're talking about I know what you mean so you're saying like hey I'm like the UX designer and they've already pre specified all that other stuff to me or I'm a p.m. and I'm jumping in it so I'm jumping in at Step four or five and some of other people in the or cave decreed one two and three and I'm inheriting that and I may or may not believe that's right or I may have a feeling that it's probably not right how can I influence that well I mean the number one thing is is to test test with customers then the question becomes a kind of a blame game oh they didn't like it well why didn't they like well maybe it's your UI design that you did or maybe it's your feature set that you didn't do so it's hard to attribute that if you think that target customer is wrong then you can say hey guys why don't we actually talk to two sets of customers that's an easy one action because most people again they don't like to say no like yeah yeah you're right in case we fail with customer group want maybe we'll hit with customer group - that sounds like it's gonna increase our odds right so that's one thing you can do on that you can actually get good data on this from your interviews so I recommend like I said starting out with discovery and then going into the prototype in the discovery phase I'll be like hey can you tell me how do you get that need met today and you'll learn it's a great way to get competitive intelligence people say I use this I use this I use this and then you can get into well why did you pick that product what other products did you evaluate what do you like about that one that's better right and then then if later when you test your product you see people making comments like wow this isn't any better than what I use today or it's worse because of this then now you've you can isolate it to this value prop layer that makes sense yeah so so those are some techniques other questions yeah well we have to define quality can mean a lot of different things and you just mentioned reliability so what would it what can you tell me more about yeah right right right right so it's hard because prototypes are always reliable right and they don't go down the server doesn't go down so that's one more where I would say you know if that was your hypothesis that our competitor has low reliability and we're gonna have higher reliability the first thing I would do is I would confirm that the competitors customers actually are saying they have low reliability so I'd go interview 20 over users and I wouldn't say tell me about liability I just see if they brought it up and if they don't bring it up at all then your hypothesis is wrong if they do bring it up then you go okay cool and then you can try to then you can like you don't lead the witness but if they bring it up and you go oh hey what do you if you had to estimate it like what do you think it is 50 percent 75 percent you know like some rough numbers and then you can kind of estimate it right so there's you can validate your I paw this is about the problem that you're gonna do a better job on as far as how you confirm that you're doing a better job that's tough because until you actually get out there and do it you don't know but if you have a reason to a strong reason to believe that you're going to achieve higher reliability as long as you just and people say it's important right they won't bring it up if it's not important but you want to kind of quantify how important is that reliability so you're kind of quantifying importance and satisfaction for that rrally ability and you're hoping you're saying it's in the upper left quadrant so you want to validate that if it is and you know that your reliability is gonna be like this much better you almost don't even need to test it if you know it's important and you know they're here and you know that you're gonna be here or you're confident you're gonna be here you should be in good shape you know what I mean so you could explicitly ask people hey what if our product was you know you just told me Uber's fifty percent reliable what if I our product was ninety percent alive I'm they're just gonna say yeah that sounds good that's what they're gonna say right so yeah yeah yeah yeah so I mean that's how I would go about building confidence yeah that's a tough call and there's a little bit I'm happy to give them share my opinion it's a little bit of it depends like if you're gonna go join a machine-learning startup then maybe you should learn about machine learning right or if you're joining a real techy startup then maybe you need a technical background but with those exceptions in general and I think in general a lot of employers over-index on technical skills and I think the reason they do that they do know a lot of them say hey CS degree required for a p.m. a lot of them they don't maybe fully understand p.m. and they're just using it as a hack because they're saying hey if you have a CS degree then you're gonna be able to work with the engineer as well it's like a lazy shorthand for you'll probably be able to work with engineer as well and it's not necessarily true I mean you cannot have a cs degree and be able to work with an engineer as well certainly yeah that too right you could right but yeah and just gonna be clear like they're they just want to avoid the clueless PM who goes and ask for things that would take years to build like that's you know that so there's like the extremes right there's the extreme there's a programmer who gets into p.m. and they just act like a programmer working on solutions and there's the PM with no technical skills and doesn't even know what they don't know and they ask for silly stuff so but in general with the exception of places where you truly need a specific skill like machine learning or technical skills I personally would I would invest more in not even UI design UX design like wire framing skills and also customer research skills right we just got done saying hey I interviewed these customer I did these sessions like who's gonna do that a lot of times in a customer in a company everyone kind of looks around like who's gonna do it you know and sometimes it's like oh we have a professional UX researcher you often don't have that and so it's gonna be like the designer and the PM looking each other going who wants to do who hates doing this less kind of thing right as a p.m. I would embrace that and jump in and do it I want to be close to customers as I can right so and those new ex research skills like interviewing skills don't take a lot of time it's not a huge investment to learn that learning learning UX skill it's like learning a tool like balsamic if you're not you need to you need to have a go-to wireframing tool that you're fairly proficient in and comfortable with so when somebody does come at you with a product idea you can kind of relatively quickly bang out some basic wireframe and say what do you think you know that's what you want to get to the point of doing yet the other thing as I talk about in the book and my other talks is just like there's often the lack of UX research there's often a design gap especially in startups would be like okay we got a bunch of Engineers then we have a p.m. that's what we got so that's where the wireframes skills come in handy you know no one's expecting you be the world's best wireframe er but when you're in that kind of situation even in an established company you know you think oh you work at Facebook you must have designers falling out of your pockets right no on certain teams certain points in time they're understaffed or the designers aren't hired party things or they're unfilled positions even in big companies there's a design gap at time so it can be good to know that and also just even if you do have a designer you'll be able to partner with them much more effectively so I hope that helps and the other thing I would say is some companies are very analytical in their DNA like a Zynga or something like that so for those companies maybe it's best to invest in analytics right so the certain exceptions like the machine learning data science technical for technical products you know analytics for highly analytical cultures but all other ones I would invest in UX design and wire framing and UX research

Channel: Product School

Views: 52,966

Keywords: Product, Product Manager, Product Management, Product School, Tech, Startups, Data Analytics, Coding for Managers, Managers, How to get a job, product manager salary, product manager resume, product manager jobs, what is product management, what is a product manager, product management training, how to become a product manager, cracking the product manager interview, product management jobs, The Lean Product Playbook, book, product book, products, career, lean startup, lean product

Id: m_c-B_MrxQU

Length: 80min 46sec (4846 seconds)

Published: Mon Dec 04 2017