How to Know Your Customers by Amazon Sr Product Manager

is what I do every day and I like doing it so I like doing these type of things because I want to be able to share the joy of of what I do for a job product management is a very unique job and there's not a lot out there about how to be a product manager there's things like product school but I take every opportunity I can to share what it's like to be a product manager so Who am I my name is will I work at Amazon I work on a product called customer reviews some of you might know about that product a lot of people know about that product I came to this product specifically because it's a it's a relevant product I worked on something previously that I thought wasn't quite as relevant it was really cool but not as relevant so this is what I do now and my main focus is actually promoting trust in customer reviews so has anyone here heard of a fake review unfortunately a lot of people that's my job so I try to build systems strategies and policies that make reviews on Amazon trustworthy and I would say it's the hardest problem that I've ever worked on and I worked on some hard problems so that's what I do in my nine-to-five what I'm gonna do today is tell you how to build the product that your customers want I work at Amazon where people really care about customers but this is something that all product managers do customer the customer is the king always and if you build a product that they want your business will likely be successful so as you leave here today my goal is to give you a crash course so like if I got hit by a bus one day or something or I couldn't show up to work you could come in my place and do part of my job so I'll do my best alright this is a really brief overview of what we're gonna do today I'm gonna try and keep it at a pretty high level I know there's some people here who are already product managers so if they're if you see something and you're really interested and you want me to go a little bit deeper just raise your hand I'm really happy to go into it I'll save the longer questions for the end but this is a really rough structure doing what your customers want is is not an easy problem there's a lot of questions there and I'm really going to break it down and say like a step-by-step and a skills based approach for what you really want to do if you're trying to build a product your customers want all right I'm gonna start with this guy I think he's my boss's boss's boss's boss's boss that's Jeff Bezos and this is a quote from his 20 17 letter to shareholders he has really cool letters to shareholders he writes one every year since 1997 but I'm just gonna read out loud so it sinks in a little bit one thing I love about customers is that they're divinely discontent their expectations are never static they go up it's human nature we didn't ascend from our hunter-gatherer days by being satisfied people have a voracious appetite for a better way and yesterday's Wow quickly becomes today's ordinary you cannot rest on your laurels in this world customers won't have it so the first time I read that quote I got really nervous because I said wow how am I ever going to make my customers happy if you know the second I build the product they want they're going to become bored bored with it and at a first glance this is a really scary quote but if you look at it again this is actually a really great opportunity so because your customers are always looking for something better because they always have something to complain about you can harness the power of your customers to build a great product so that's what we're gonna do today if I had to boil down everything I do into one little chart with respect to customers this would be it so you notice customers are in here as many times as possible this is the product development process with respect to customers so this side over here is what you typically see in a startup you don't have anything yet you're trying to solve some type of customer problem and you need to get something to market and there's this thing here that I'm gonna walk you through in a second working backwards you get from the customer to a MVP or a Minimum Viable Product I'm gonna focus on that with you but I'm also going to spend a lot of time on this part of the graph and that's what I do every single day and if you're really good at this you can spend your entire doing this work with customers find out what they want turn it into a product go back to your customers again find out what they want turn into a better product it's a cycle that you can do over and over and over again to keep your customers really happy with your product keep them coming back keep your business running it this is sometimes called a virtuous cycle so this is the crux of working backwards it's working backwards is an Amazon thing I have to do this in my job there's actually a document called the working backwards document that product managers at Amazon use and there's also a thing called a PR FAQ press release and questions but the idea is that instead of building a product and then trying to make it meet a customer need you start with the customer first you find out exactly what you need and then you build the precise product that they want the the product space right now is littered with products that seems really cool but no one ever wanted to buy them and and that's a mistake that you can make there's a lot of really cool things that I'd like to create but just nobody wants them so this is a structured way to avoid that mistake and let's break it down a little bit who is the customer if you are a marketer you might hear this called segmentation the idea is that you need to deeply understand who your customer is and many different types of customers so we're gonna go into an example with uber right after this but you really have to know who your customer is and sometimes this is really easy maybe you are the customer does that help you a little bit I don't know we'll find out but really start with who the customer is first and the second part is what is the customers problem or what is an opportunity that you can solve with your product and you'll typically understand this by doing the first one as you start to talk to your customer as you understand who they are they're gonna tell you things they're gonna say wow you know I was reading those reviews and it was really hard to sort them by positive and negative at the same time I wanted a pros and cons for you boom there's a product idea right there so you're you're looking for what is the what is the customer need and lastly why does this really matter for them there are some customer needs that really just don't matter quite as much people have annoyances some people also have like really existential product needs and you really have to justify why should you do this and what's the real tangible concrete benefit here customers you don't want to build a product that solves an annoyance you want something that's genuinely useful and meaningful to your customers all right so let's break into BER before we do this who here has a smartphone pretty much everybody awesome does your smartphone have a GPS in it yeah pretty much everybody I mean if you've got one of these you have a GPS in it who has a credit card here okay and who here has a need to travel to get around in a let's say 3 to 10 mile basis alright awesome we have a pretty good idea of who our customers are should also be aware that as this group we likely have similarities among us that not all customers would have but we'll come to that and a bit so here's Ober this is actually their original pitch deck in 2008 when they were called uber cab and I'm not making this up this is actually the slides that they showed investors but let's try and map some of the customer needs to the product here so we just said we all have smartphones what about this right here one click hailing in a past world when you want to grab a taxi in Manhattan you had to go like this and they were hot and sweaty like you have to wave your hand around desperately hope that someone will see you I've seen really aggressive hailing too or you you know walk out halfway into the road and say please pick me up since we all have a smartphone and we know that it was frustrating to hail a taxi this one click hailing is really exciting to us another one here that I really liked is guaranteed pickup if you've ever taken a taxi in New York you know that sometimes if you tell the driver where you want to go they actually won't let you get in the car because you're not going somewhere profitable or there's a lot of traffic there and they know they're not gonna make money so one thing that ever found out talking to passengers is that they really wanted a guaranteed pick up and uber could offer them that so here's a little bit of one of overs customer segments passengers and we didn't go into this earlier but there's actually a few smaller segments beyond just passenger there's the late night partier kid who needs a ride to school and can't drive a car themselves or maybe someone who landed in a new city and needs a ride from the airport those are all different types of customers and uber started with them when they were deciding what product to build so even further let's see how can we map some of these smaller segments to these customer needs well ride to school great drivers there's a rate your trip feature you don't want your kid driving in a car with someone who is untrustworthy and that's a problem that uber is actually dealt with on their own but there's one feature that might matter to this customer segment and the airport pickup you know there's a couple ones here but car services require one two three hours notice at most airports there's a taxi pickup Lane but not always so if you can just hop out of your plane turn your phone on and pail a ride that makes your life a lot easier you also don't want to forget that most products don't just have one customer segment so the product that I work on customer reviews has one really obvious segment the shoppers who read customer reviews and want to know should I buy this product but there's also a bunch of other segments and one of them is sellers so I spend most my time protecting the integrity of customer reviews but I also have to remember that the shopper isn't my only customer I have to do the right thing for sellers I have to do the right thing for reviewers and I don't want to forget those customers who aren't my primary customer segments or those who I thought was or were my primary customer segment so uber really has another customer segment its drivers and you can see how some of the things here map to what drivers would want the medallion system is really frustrating does anyone know what a medallion is if you want to drive a taxi you have to pay you know upwards of a hundred thousand dollars to get a medallion that you stick on your taxi just for the right to drive with uber you can start driving whenever you want or lyft or any service like that so don't forget about your other customer segments and you'll often do this in an exercise where you go and talk to everyone and say alright what type of customer segments do I have is this the only customer segment that I have what are other angles I could look at this with so don't forget about drivers and an interesting thing is uber did forget about drivers so even if you're a great product manager darah you know what used to work at Expedia he's definitely a product manager uber forgot about drivers and drivers went to this other app and I hope a lot of you use because they did they didn't like uber so uber had to admit I'm we're sorry we forgot about this customer segment we're gonna redesign our driver app so it works for you and they realized that if they didn't have drivers their riders would become unhappy because there's no one to pick them up so don't forget about all of your customer segments all right so if you think about that that little diagram that I had an even more simplified version of that diagram is just this there's two great pieces of every business every product every new feature one is deeply understanding the customer need and two is finding an elegant way to meet that need we're not going to spend too much time on this one today I will at the end but understanding your customer is mostly about understanding the customer need you have too deeply and intimately know what does your customer want so you can build the right product for them so we're gonna talk mostly about this this is an awkward word because in the tech industry where I work empathy is not often a skill that people celebrate and if you are a product manager it is a skill that is a must-have and I read a quote in New Yorker that said the one area where Silicon Valley has universally failed is they didn't have enough empathy for the people who their products were affecting every day and I think in a lot of ways that's right and empathy the ability to understand and share the feelings of another is something that is your forte as a product manager you know among engineers among data scientists among managers empathy is a great skill for them to but it has to be your core because you need to understand what your customers want you are that customer voice and if you can't live a day in someone else's shoes you'll never be able to know what they truly need so I want to unpack this word a little bit so you can understand what it truly means what I found in a lot of interviews is I'll ask a candidate what does empathy mean to you and they just say oh it's like living a day and another person shoes or I can understand this person's viewpoint and that's a great surface level understanding but empathy goes quite a bit deeper and it goes deeper because of deep-seated psychological phenomena that we are subject to and that we don't truly understand and if you do you can be much better at this so let's just do a quick little exercise do you think there's anyone else in this room that is exactly the same as you definitely not I don't think there's anyone here that's exactly the same as me nor would I want it that way so you're staring at my face I'm staring at your face how do I know what you want that's pretty hard I can make a lot of assumptions about what you want I can look at you I can see what clothes you're wearing I can see your body language you know maybe look a little bit nervous maybe you're hot something like that but I'm making assumptions and assumptions are the Kryptonite of product management don't make assumptions you'll have to make assumptions but assumptions get you into big big trouble so this is where empathy comes in you really don't want to make assumptions you have to truly live empathy and sometimes empathy will be painful you have to listen to a customer who had a really tough story and you have to feel that customer story and understand exactly how it influenced them sometimes you get a customer who's angry you know you built this beautiful product that you're so proud of and a customer tells you is trash that really hurts and the worst ones to read are when the customer says something about your product and it's fiercely negative and you know it's right but you have to understand why did this customer think that how did this customer see my product and how do I turn this into a better product so empathy is pretty hard a lot of people you know read the different the dictum the dictionary definition and say yeah I know what this is but empathy is not a skill it's a muscle it's something that you have to practice and it will help you not just in product management it'll help you and many other things so here's a guy who I would argue is fairly good at empathy say I know who this is yeah Jack Ma Alibaba so he was an English teacher and now he's a tech tycoon another billionaire arguably Jeff Bezos of a of a different market and he says I'm not a tech guy I'm looking at the technology with the eyes and my customers normal people eyes and like he's really not a normal guy but part of the reason he's been so successful I would argue is that he really is able to look at all of his products as it as a an average person you know he doesn't assume that he's like everyone else will use his product he says he has a principle of treating people well and that's what helped him launch Alibaba in 1999 so you can find examples of empathy all over but here's a pretty good one so let's practice a little bit a time feat when you look at customer feedback it's going to be something like this vegetable soup you can see in that can you know letters a little bit of carrots I think that's a green bean in there all kinds of stuff and it's gonna hit you from every angle here are some examples of where you might get customer feedback I've had some pretty interesting examples where I'm walking in an airport in a different country and I'm casually chatting with someone they asked me what I work on and I say customer reviews and for the next 45 minutes I'm listening to how this person read fake reviews and they were so angry with Amazon and I was on vacation I honestly didn't want to spend my time doing it but that was a real exercise of my empathy you know I could have said hey sorry I'm late for my flight I have to go but if I truly wanted to you know practice what I preach I need to listen to this guy and say I get it tell me more about how this went like what went wrong how did you react to this it was really tough so you'll get product feedback in a lot of different venues depending on what your product is you might have some of these and not others so you know maybe your product has Product Forums where you're most passionate users will go write about what they think about the product maybe you don't maybe you have a really small product and your only presence is on Instagram you know how are you gonna get feedback that way maybe people will send you a message or they'll share it on their own Instagram and say hey I really love this product or Wow is really frustrated like this didn't work out for me and some products also have the luxury of being bombarded with customer feedback if you work on a very visible product like customer reviews or anything big that a lot of people use you're gonna get so much you don't know what to do with it but there's also many products where no one's gonna say anything maybe you just launched your products no one knows what it is who's gonna tell you anything about this product so you might have to look but product feedback will come to you in many different venues all you have to do with that feedback is break it down to themes so depending on how complex your product is there may be a lot of themes there may be few themes but if if you listen to a hundred different people they're gonna tell you 100 different things you know the story is always going to be slightly different there's a different variable that was present in this story but not that one but there will be things that are in common in every story and every customer anecdote and it's your unique ability to find out what those are you know maybe everyone's talking about customer views and you know regardless of the story they all say that it's too hard to sift through them all you know one customer said it you know they read through every review and it took them three hours or another customer who said you know wow I wanted to sort through these and you know just I had to do - way too many clicks to find out what I needed they both told you different stories but the theme is that they had a truck they had a hard time processing all that information so as a product manager you might have someone who's like a US researcher or something like this help you but you need to understand the feedback that you get and break it down into a theme and you might think that this is just boilerplate here but there's a reason for that for every theme that I get I find it's very helpful to have at least two customer anecdotes I'll talk about customer anecdotes later but anecdotes are really great at telling a story and if you say customers were frustrated with reviews because there is too much information and you're trying to communicate that concept to someone else they're gonna say okay I don't really get that can you tell me more so your anecdote is a tool to give color to that story to put a person behind that story to make it more real so as you're collecting themes from this soup of customer feedback make sure to save a few anecdotes so you can really truly tell that story and some customers are great at giving you anecdotes other customers just say I like the product you take what you get basically and as you're doing this with all these themes you don't want a theme like I thought it was good or I was unhappy you want themes that are actionable because when you're collecting those themes you're always thinking about this the incremental a better product this theme that I'm hearing for customers how does that translate into a better product that my customers will love more that they really want that has the features they want that will make me successful as a product manager well another thing I want to talk about here is there are some tools to make this a little bit easier if you work at a big company which many you might you just get overwhelmed by the amount of things coming in and it's it's very tough to categorize it all so there's techniques that you can do to make your feedback make customer feedback more targeted in one of them that I really like is called the five why's has anyone heard of that you just keep asking why until you get to the root cause so like an example I have here is the battery is dead why the alternators not functioning why the alternator belt has broken why the alternator belt was well beyond its useful service life and not replaced why the vehicle was not maintained according to the recommended service schedule so you got from the battery's dead - you didn't follow the maintenance schedule and those are two very different things but the real problem is not that the battery's dead it's that the customer doesn't know how to maintain their vehicle so if you keep asking you'll get more useful feedback you won't get the I liked it you'll get the I liked it because this and if you did this it would be better so as much as you can try and break it down a little bit more another tip that I can give is that when you get customer feedback you can have a an instant reaction to that a classic example is when you get negative customer feedback went up someone tells you that your product sucks and you've spent years of your life working on this product and you made the decisions that made them unhappy it's going to hurt and you might get defensive you might dismiss their feedback you might say this this customer is crazy I know better I spent so much time in this I made all the decisions how could they say that this is where your empathy comes in again if you can truly put yourself in the customers shoes you can be a neutral listener you don't prompt the customer with what you think and you really let them explain it don't be afraid of silence either if don't fill in the customer silence with what you think because then you'll lead them down a different road than they were originally going to go with you so a little bit about customer feedback so here's some real customer feedback this is not a product that I work on but this is reviews are a cool way to get customer feedback I'm going to use this example because the echo team actually does use customer reviews for customer feedback and they do exactly what I just described to you they look at 116,000 customer reviews for just this product and they find themes and they use them to make a better product so let's look at what some customer said and again some of them are gonna be good and some of them are gonna be really harsh so yeah if you read this one right here you're really happy Wow my product is something meaningful I made this person's life better then this guy comes in that's a tough day at work but yeah it's gonna look like this and then here's a here's that thank you I really appreciate that but that didn't really help me build a better product so you have to love these but you have to know that this is actually better than this one yeah so let's do a little exercise with this you know if we got this soup can we make something out of it and I've only given you the title here but you know the echo dot turned Alexa into a bat salesman why why were you angry at your LexA and you can read their comment a little bit and say okay what did this customer say is there something I can take away here is this comment to what other customers are saying was there anyone else here who said the same thing unpack that a little bit or you know this useless one why do you love it what worked well for you what didn't work well or this one the customer who's insulting your product don't be defensive you know I agree with you the product isn't that great you know tell me how I can make it better that's a great way to approach those ones now this one is even meaner the customer through your product in the trash I've seen that happen I used to work on a product where someone made a video of burning the product shooting it with a gun and running it over with the tank and that was pretty hard but if you practice your empathy you can understand why is this customer unhappy what about the product didn't work for them how can I turn this unhappy customer into one of my greatest customers who really appreciates my product because how I listened to them and made improvements that made it a great experience for them all right so we just went through a lot of anecdotes and we talked about that soup and we talked about how that soup could sometimes be a little challenging to navigate and there's a reason for that like the soup is not meant to be really concise like you will never build the best product with just the soup so this is how you build the best product you take the soup and you add some data to it and together you can get the product your customers really want and why you need the soup and the data is that the data is factual information like it you can you know tell stories with statistics but not really it's its objective information there's there's no true story to it it it just is and anecdote is what I like to call the framing value so you can look at numbers all day you can look at your product metrics you can say oh conversion is up 10% but why so this helps you answer the why and if you get both together you can get a really strong story and a great direction to improve your product so maybe that guy said you know the the echo dot was turned into a d-bag salesman and then we have some data that says when customers you know got that notification they used their dot less there's a much better story than just the customer who was angry and gave me that anecdote so whenever you have a chance anecdotes and data and this is hard because you know it's easy to collect antidotes it can be harder to collect data so now I'm gonna tell you how you can get some good data here's an illustration of of why this matters this is your anecdote it's one person the emoji girl and she just told you something about your products and you think you can go somewhere with that but you're not sure how much does it matter if everyone in this room told me something different about the product which one is the most important to me or how one person told me one thing and the second person told me something else that was conflicting who's right or who do I listen to or if I listen to both like which one is most important to focus on now and which one can I solve later so the power of data is that you don't just get this one person you get everybody or at least a larger sample size to make a a stronger case for the Y don't forget this sometimes at a big company this is going to be larger than you could fit in this entire room some of you will be product managers for a product where this is 2 or 3 or 4 so use what you have and when you talk about data people have a variety of backgrounds coming into product management and one of the hardest questions that comes up is oh I'm not as technical how am I get this data and or another one is you know I wasn't a software engineer before how do I understand this tech design you know if I was an engineer and fortunately with data this will make you pretty dangerous if you can take a you know a basic statistics class or even read about stats on Wikipedia maybe you go to product school there's a lot of ways to do this but thankfully it's not that hard if you understand what a median is what an interquartile range like you already have the basic information you need to make good data-driven decisions so it's really not that hard and don't be intimidated the other thing is don't necessarily learn sequel Amazon uses sequel to get data they just throw you a big soup say figure it out when I say sequel I don't necessarily mean sequel I mean a way to get the data so let's say you have a massive database with every customer interaction you might need sequel for that but maybe they're much smaller business and your data is just an Excel spreadsheet it's not just oh really as stats but it's a way to understand and a way to retrieve so if you can piece those two together which neither are very hard to figure out you can be really dangerous with data and yes like at a bigger company you'll have a data scientist you have a research scientist a business analyst you'll have people who can help you with this but if you're able to do it on your own or at least approach the concepts you can make much better decisions zaman recognize this know which product this is Twitter exactly so let's see what's going on here this is an anecdote here's someone who's trying to get under the 140 character tweet limit and they're doing all kinds of strange things like what is this right here trying you are you can you compose it's strange yeah you could say that's okay it's the modern world where you it's shortened everything but maybe you have a business customer that says well I really didn't like that feature because I had to do things like this and it made my tweet not professional here's another one this is what you would want to send but you can't send it because now you're 13 characters over the limit so this is what most people did they took a lot of time and carefully worded every tweet so it fit exactly under the limit so here's our anecdote and look what Twitter found so this was a product manager her name is Eliza Rosen and they called this cramming so shortcut this is the cramming so what's happening here is they looked at many different languages and with each language there is a concept of information density just how much how much how much can you get in a certain amount of characters turns out in Japanese you can get a lot into a little but with English like look what happened here so this is 148 we care 140 character tweet limit and wow a lot of people are coming in right under that people are doing unnatural things just to fit under that 140 character limit and they also found out that the the languages that had this problem more than others had a smaller percentage of users who actually tweeted so they took that example that we talked about before and I said wow this is actually preventing people from tweeting like this is really painful we should do something about it so this is what they tried to do this looks suspiciously like a normal distribution except for this strange spot you know what if we just increase the character limit so these customers who had a really painful experience could just tweet smoothly so they did that they increase it to 280 characters and it was really controversial does anyone here use Twitter a lot any big Twitter users okay you do yes or no for the 280 character limit okay awesome you're a great person to ask but not everyone will say that so why did I highlight that this is controversial because that anecdote that we saw where you know the the business user was trying to cram words into that tweet they're not every customer there are other customers who said you've ruined Twitter by introducing 280 characters because 140 was the ethos of this product like you've you've thrown the product you know down the drain by doing this you made people really unhappy but when you have a tough situation like that anecdotes plus data you can make a much better decision and Twitter did go through with this and they've stuck with it and they're their metrics improved quite a bit they got more users tweeting they got higher engagement on every tweet and it was a happy story so a good example of how you can use anecdotes and data here's another awkward word and it's uncomfortable for product managers because we celebrate success so often the two quotes I've shown you before they're both hugely successful billionaires and this is not something that you practice as a product manager you give a convincing presentation you always know the right thing to do for your product yeah humility doesn't come up a lot but actually humility is a really important part of being a product manager so again Jeff Bezos that billionaire he said people who were write a lot of the time were people who often change their minds so is that good or bad you could say that's a flip-flopper you know this person has no conviction they can't decide but you also have to remember that as a product manager you're not paid to get it right the first time you don't have to come into every meeting and say I know exactly what we'll built you don't have to have the perfect concept every time you go into a pitch it just doesn't work that way and even the most successful people it never works that way so remember to be humble and one way I've heard this set is Voice Europeans with conviction but be quick to change when you encounter better information when conditions change maybe when you were initially thinking of this product thing a was really popular but now it's faded and thing B is now popular you need to change should you go forward and say let your pride take hold of you and say I was right the first time let's do it anyway and then fail no you should be okay to be humble and and just go with it so do not forget humility and with humility I'm going to talk about some really common mistakes that product managers make I've made some of these and I expect every product manager will and some of these some people have made these mistakes at really huge scale it's okay to be humble let's laugh a little bit and and see what that looks like so here's the biggest pitfall that I see product managers make I know what my customers want because I use the product a lot I'll just let you look at the picture for a second it's a cow using a I think it's a fisher-price toy and the cow has their head in the fisher-price toy and as you can see it's not working out so the cow is the product manager if you are this cow you're not going to build the right product because who really uses this fisher-price Tori is it is there market cows no it's kids so you have to remember that you are not always the customer and there's there's actually a real word for this it's called the false consensus effect humans in general have a tendency to think that what they believe is also accepted by the majority and it you can look this up if you're interested it's a like there's research into this this is really dangerous and people do it without knowing when you make a statement you know you you just naturally think that because I believe this other people do when that is not the case at all so be really careful of this this can be more dangerous when you are a passionate user of the product you know let's say you're really into gaming and you you know start a new job at Xbox you want to be really careful not to project everything you know about gaming on to that product because you're not the only customer and you may not be a typical customer either one thing that we constantly have to be aware of about in Seattle or San Francisco or New York in a big tech company is that you are very unlike most people who use their product Amazon actually has a way of doing this they make more senior leaders take customer service calls and on my first customer service call you know I was trying to be super professional and you know use all the right words and everything and I talked to a customer in Kansas City who was really angry that their jelly beans didn't come in the mail and Amazon had delivered them to the wrong location and the biggest thing I learned from that is like I was all hyped up about joining a big tech company you know going to my shiny office and Here I am talking to this guy nothing to do with me I'm telling you about how his jelly beans didn't show up and I just realized that like I made a lot of assumptions about who my customers were that we're not true so do not forget this there are exercises you can do you can go meet your customers you can actually use your product with other customers to see how they use it don't think that you are the customer even if you use it all the time another one is this this one will get controversial too I'm a visionary I know what my customers want more than I do that ones associated with this guy most of the time who's a polarizing figure I would not recommend this one unless you are Steve Jobs and even if you are Steve Jobs I would still not recommend this one and this is why for every one of these there's a thousand of those some more painful than others but for every visionary product that customers love like the market is littered with those that they did it leave your ego at home be humble don't forget to talk to your customers make decisions in a safer way I'm going to talk about that next so that your vision assuredly matches the customer need be careful as what I will say here if you are very very sure test it but don't don't blindly introduce something thinking that it's right because you believe so here's another really visible one I actually I spent a tiny bit of time working on the Windows product through a tangential product later and yeah this was really painful and a lot of people left and it caused a lot of damage for that product Windows 8 and this one does anyone know what juicer row is it's a I think it was originally $700 and then they made it $400 it's a juice machine and some guy in YouTube made a video of putting fruits in like a ziploc bag and he bashed it with a hammer and the juice was tastier than this thing turns out nobody really wanted a $700 juice machine even though it looked really cool and some guy had a vision for this thing so don't fall victim to this so we talked a lot about deeply understanding the customer need the truth is that's not all the story I didn't tell you about this part quite as much you could give a entire other talk about that but I will give you some of the highlights just so you're you know if I did get hit by a bus and you had to go in for work me tomorrow you could still do this what if there is a world where you could just listen to customers and then the product made itself you just listen to exactly what they wanted and then the product was made for you that would be great right turns out it it isn't that way I think we all know that but there there is somewhat of a science into taking that deep understanding of what customers need and making sure it really is the right product so I'll give you a little overview of that remember this little wheel we had that happens so how do you avoid that maybe you have an excellent understanding of what customers need you talk to all your customers you have great data to back it up then you took a little while to build a product and something changed and you launched it and nobody liked it anymore so what are better ways to meet that customer need an elegant way with running into this less and you will run into this even if you do everything right you have the perfect data you deeply understood your customers this will still happen to you part of being a product manager is making this happen less or when this does happen make it hurt a little bit less than it could have does anyone know who these guys are all right Airbnb Joe Gebbia right this guy he gave a really cool talk it's a podcast actually it's called how I built this you can just google how I built this Airbnb and you'll hear this quote so here we are in the living room and they pull out their laptop and in those moments we saw how our perfectly designed interface completely and utterly failed well we thought took two or three clicks to get done people are taking 10 to 12 clicks stumbling around and getting lost so these guys all went to RISD and if you're in the design world RISD is is like the best you can get for design these guys thought they had a beautiful interface they talked to customers everything was great but when they launched it it didn't turn out that way so even the best people even people who are hugely successful can fall victim to this what are some strategies to avoid it this is one I would not recommend this one you can maybe do this for small bets if it's something that your product doesn't truly depend on this might be okay but how about if you're launching a brand new product line and its success or failure is critical to the to whether your business lives are you gonna leave it up to a coin flip probably not so these are some common ways again this could be an entire other talk one that's really popular is this a be testing the theme here is that all these are just safer ways to fail failing is okay but fail when it doesn't matter as much fail when you know the stakes are lower artificially lower the stakes so that when you do fail it'll be okay and you'll learn something from it don't take coin flips when we talked about here in the beginning in is working backwards that critical to understanding your customers is when you think you know your customers need to still let down into the minimum that you think can get you some signal in the market and get it out there and see what your customers say so here's a a good example of that Robin Hood Robin Hood had a million users before they even launched the product and some company has abuse this they'll put up one of these pages and say sign up and then two years later you go hey where'd the product go but this is one way to do that you can get your concept out there and find out without having to build and build an app build all the integrations you need to trade stocks find out if customers really want that this is called a soft launch here's another one I've talked a lot about a lot of techie products product managers don't all work on tech products this is a company called Lola they make tampons tampons are hard for me as a product manager because from just what I know I don't understand a lot about tampons and that's okay but let's say I was the product manager for Lola and I wanted a better way for I wanted a better tampon product for my customers and one thing I knew from anecdotes is that customers were unhappy when they bought some tampons and they ended up with too many of the wrong size you know the box only comes with you know this many of this size and then this size and wow you know three months later I have a huge stash of this size that I don't need so I say okay I listen to that customer I'm going to introduce a feature that allows me to customize what I get in the box one way I can launch this is take a coin flip just put that everywhere and make customers do this and see what happens I would not recommend that so what are some better ways you could do a pilot you could say let's let some customers try this maybe we have a group of very loyal customers who we know it'll be a little a little bit safer with let's let them try it and see how their experience goes or you could do a random like a bee test where you know 50% of visitors get this option let's see who's interested in it you also want to measure whether this actually works like how do we know whether customers actually like this and I came up with a few ways you could guess so a customer roughly uses this product every four weeks so we can measure for 816 week how many people leave did they like this product or were they unhappy and they decided not to continue using my product I could send them a little survey and say how happy were you with Lola you know there and give some some like a 1 to 10 smiley face thing and see what people say you could find out what percent of customers or more of them or what percent of customers switch back to the regular product or how many people adjust this breakdown after having done it once so don't bet your business or your product on one change do it in a safer way measure it and find out how it actually performs so we're coming about to time I've tried to give you a crash course if I get hit by a bus tomorrow you could roughly go into my job and at least have a framework for how to go about building what your customers want you'd have some successes and failures too like you you learn a lot by doing sometimes you think you have the perfect information and then it turns out you don't so then you grow by doing that and that's okay this is my best attempt at a crash course I want to leave it open to you guys for questions was there anything you saw in the presentation that you want to know more about something you showed up today and you wanted to learn that I didn't cover just go for it go ahead yeah so where I work customer needs are almost everything at Amazon you don't hear people in the hallway arguing about making money you hear them in the hallway arguing about what was right for the customer so that helps me a little bit that's a valid concern sometimes you can make more money doing the wrong thing for the customer but what you usually find is that if you you know bleed the customer for more money by not doing the right thing for them in the long term that won't work out so it's a balance usually though if your customers are happy you'll continue making money so I would say customers first but then as you launch something say you know does this work for our business and maybe don't watch things that that work great for customers but have no business relevance one really good example of that is movie pass we heard of movie pass movie pass was I would call it charity for movie tickets they I think it's charged you $9.99 allowed you to go to as many movies as you wanted a month and turns out customers went to a lot of movies and they were giving movie tickets away for free and I don't know whether they've officially gone bankrupt but they're just about bankrupt now so there's a good example prioritize customers first but always check before you launch say does this make sense for our our bottom line and like I think you can get that like let's say you're selling a physical product customers would love if you get away for free but there's a balance there like you can't really give your product away for free because then you don't have a business is that helpful of it good yeah sure yeah so if you work at a big company if you work on a very visible product like you have that luxury of just more feedback than you ever know what to do with most people will not be in that position so what are some great ways to get feedback some popular so they you should use more than one and the the tenant that I would follow is never miss an opportunity for a customer to give you feedback so every time you talk to a customer hey while we're here you know tell me what you thought about this product or as you send a customer an email like a transactional email about their purchase give them an opportunity to weigh in you know emails are one way like a feedback form at the bottom saying like how did we do tell us something and you won't get everyone from that but you'll get some signal you can call your customers sometimes there's like rules for whether you can do that and Amazon we don't just call people randomly but if a customer let's say a customer had a problem with the product and they're they're dealing with you know the support team or customer service you can say hey you know I work on that product would you be willing to talk to me and explain you know what went right here what we can do better and customers are often willing to do that so take every opportunity you can you can send surveys you can do user research you some very painful ways you can stand on the street and do it that's a very humiliating experience but it sometimes works you can also pitch it to your friends like let's say you're in a really early stage and say hey you know I'm thinking about this idea you know tell me our thoughts about it and just leave it open to them and the cognizant that you're not going to get a fully balanced approach like they're your friends there are likely similar to you but it gives you something so depending where you are like there's hustle - you know query my database and I would do all of them if you can does that help cool surely the question is how do you practice empathy and not getting hurt when the customer tells you something unflattering I did this by inviting myself to go into the worst customer conversations and this might not work for everyone but I would find the angriest customer and just listen and like honestly just let him beat me up and that was pretty hard and it hurt my ego a lot but it like I thought just I'm practicing this like I really have to think about like my unhappiest customer and one good thing about that is you can make your your unhappiest customer happy you're doing pretty good so it's not a bad idea to start with those if you don't like being yelled at or you you're not really ready for that level yet like maybe don't start there but the the one thing we'll say is put yourself in more situations where you have some feedback that you're not prepared for and self reflect on how you react to that like when a customer gives you positive feedback how do you react to that when a customer gives you neutral feedback how to react to that and have the customer conversation and then after take a minute and say how did that go how did I respond and and self-reflect there and really the secret is practice like it it will likely not happen the first time especially you know if this product is your child like it is everything to you and you know your first customer interaction they want to throw it in the trash it's gonna be hard but as you get more of those you'll learn to take those in a more mature way to respond with a level head leave your ego at home use it as an opportunity rather than as an attack and like remember that customers are not attacking you it's just your product and every every interaction is an opportunity to do better so that's how i roughly go about it yeah go ahead how often do I release something new it really varies I try to release something new every week obviously there will be larger releases and smaller releases I try to do at least something every week some companies will ship multiple times a day but it's more incremental what I would say there is as frequently as you can even if it's not super high fidelity like ship things at lower fidelity to a smaller audience just to understand how they do if it's something that is really really critical to your business maybe take a little bit more time so frequently but as frequent as is necessary to meet your goals and it'll really vary by what your product is and where you're working maybe you're working in the phones business you know you don't release a new phone every week it just doesn't work that way so physical products will be very different than software products software products are really nice especially like web-based products cuz you can update them whenever you want whereas other products will not be that way do you have a question [Music] yeah yeah sure so the question is you get a lot of customer feedback how do you prioritize it and I'll go back to what I said about anecdotes and data so you're gonna have so many anecdotes you don't know what to do with them even if you don't have a lot of customers like you're gonna get little nuggets in every interaction even just in one interaction you get six different things and you can't do all of them at the same time so how do you know the secret is really data so everything that you hear be skeptical now this person told me they really want that feature but do they really want that feature and if I launch this feature what will happen how many other customers are there that also want this feature and also be wary that the customers who are the noisiest are not necessarily the majority so yeah take your customer feedback try to find some objective data to back it up and rank your things based on some formulation you have of of what I call impact so maybe that's the number of customers versus how much revenue they bring in or something like that some composite of how much they matter to your product and it is one of the hardest things to do like your especially on a like a product like customer reviews that has a lot of different customer needs at a different time like I'm overwhelmed by things that I could do and I spent a lot of time saying like what is the one thing that I should do over all these things and again really it just comes down to the data I spend a probably thirty to forty percent of my job just looking at data to help me make a decision on what I should do versus what I should not do or what I should do before this and then what can wait till later yeah I think you would have a hold another talk on that that's a really tough topic yep okay I would try to get someone who has an ask go ahead is sure yeah okay so so do you mean what do you do when the data doesn't match your anecdote or what how do you prevent bias when collecting data okay yeah yep okay so I'll answer like the the conflict one first what do you do when the data doesn't match the anecdote that's a red flag oh hey guy would stop if that happens either your anecdote is not representative maybe your data is wrong or like you haven't measured the right thing so that's something to be aware of I would reconsider your assumptions if that happens or just maybe it's not the right thing to do like maybe you had an anecdote that said I want this feature and then you test it and performs horribly maybe it's not the right feature and that's a valid conclusion as well the other one is how do you prevent bias and data another mistake that I hear a lot is that numbers aren't biased or like just because it's an algorithm there's no bias to it that's not true so you have to be careful not to accidentally insert bias into your data so one way to do this and one thing that happens to me a lot is in the customer reviews products I represent reviewers all over the world and their behaviors are very different so I could accidentally introduce bias when in making a decision about what feature I should build only include US customers I could build the wrong product for customers in India customers in Japan just because I didn't listen to them and I didn't mean to do the wrong thing for them but I just didn't include them in data so the data told me like yes definitively this but it was biased so one make sure your data is representative of all customers be careful of excluding specific things that might be convenient to your analysis like and could likely give you the data you want but aren't truly objective and lastly resist the temptation to use data to tell the story you want I have done this before and it's convenient but if you really want to make a point a lot of product managers will just go cherry-pick something from the data that agrees at that point don't do that it can be really tempting and sometimes it is warranted like if you're if you're just trying to illustrate something that's fine but if you're making a this or that decision that's not okay so don't do that it will come back to bite you later does that help cool see time is it yeah I think maybe like one or two more questions go ahead yeah yep sure so prelaunch you're not going to have a lot of data how do you understand whether your MVP is the right thing without you know an active product yeah that's a hard one I'll go back to Robin Hood they did it in one way they didn't actually launch the product that's one way to do it another way is to pitch your idea to people you have to be careful not to bias them like you don't want to say hey I'm building this I'm building tinder for dogs don't you think that's a good idea yeah you're not gonna get a good data for that but if you start talking about the products and you get organic reactions that gives you some data it's not gonna give you a big scale another thing is you could test it among a smaller group like maybe your product doesn't have fit in the market yet but you like at least have some people who use it maybe a group of employees a group of friends or so like that just get anyone using it and it might not be perfect data but at least get you something some data even if imprecise or biased is better than no data so yeah and you're gonna have to hustle if you if you don't have a mature product and you don't have a lot of customers using it your data is not going to be as good as a mature product like let's say Facebook which has so much data they don't know what to do with like if you're a product manager they're like you can you have anything at your disposal to make a decision whereas you know maybe Lola tampons a much smaller company they don't quite have that luxury maybe like your company doesn't even have a relational database to store customer data yeah so it's gonna vary yeah as a project manager or product manager what tools do I use let's see I you sequel a lot and I also use Python I probably spend around 30 to 40 percent of my time looking at data I also use this tool a lot my voice big part of being a product manager is communicating concepts I use this law to email for better for so you can get really sucked into email and the big part of your job is communicating what either upward or outward or among your team I use some very specific tools to do what I want so I work with machine learning models a lot so I'm using specific Amazon tools to train a model or define features for that model a lot depends on your job I would say the core skills are understanding your customer which is this your brain and this your empathy I would say your communication ability which is your voice or your your writing ability that will vary by company Amazon as a writing company some others are PowerPoint companies and another one is like your analytical skills so how you can how you can look at data or make good decisions based on what you see and that's something you have to practice it also widely vary by what you're doing so I work on customer reviews I build big systems let's say you're working on pricing like you're gonna have you're going to use different tools and have different skills than I would so the pendulum it you

Channel: Product School

Views: 27,380

Keywords: Product, Product Manager, Product Management, Product School, Tech Startups, Data Analytics, Coding for Managers, How to get a job, product manager salary, product manager resume, product manager jobs, what is product management, what is a product manager, product management training, how to become a product manager, cracking the product manager interview, product management jobs, google, machine learning, ai, Machine Learning Products, Technology, Career

Id: JCmOKM8wZnE

Length: 67min 59sec (4079 seconds)

Published: Wed Dec 05 2018