Product Strategy for Your Best Customers by Amazon Sr PM

if today is the product some of you kind of know what to expect and some of you have no idea right so right off the bat two different customer cohorts right you've got the ones who were kind of like know what to expect and can I don't know baseline that they're what they've had in the past to this one and then you've got those that are just wide open and have no idea what what's coming up right so I came and interned with Amazon music and we're gonna talk a little bit about that but and a product manager internship is very much the way I I kind of like received it is a sentence we need to axe because of why that's it go like figure it out three months so I did that got my full-time offer went back to school and then came back to to Seattle for full-time today I worked for Amazon devices in there for almost two years so Amazon devices the Amazon device is basically all the physical devices that Amazon makes so echo Kindle tablets echo Birds whatever microwaves I specifically work on Kindle so let's start to kind of like break you guys apart ways Google Maps who uses ways who uses Google Maps why do you use ways and not Google Maps for cops all right so that's something you care about what else what's that so it so it's better on ways for you do you guys have any so they're both owned by Google right Googlebot ways used to be an Israeli company still and still and it is in Israel what's the difference between these two like they're both by Google they both tell you how to get from point A to B some of them have more features like you know cops road conditions maybe some of them update faster is there anything inherently different from getting to point a from point A to point B in your eyes no well there is so any idea all right so this is from this is from a product manager that built ways and so ways really will get you from point A to point B the fastest way possible right so that means take a right take a u-turn go on this dirt road if you if you ticked off like yes we're good roads and and just get you there as fast as possible Maps will get you there fast but will probably not take those same routes that ways did if the roads are perfectly fine and they'll both probably take the same one but the idea is that Maps is for people who want a worry-free ride right I don't want to get from point A to point B not too many turns I just want to ride listen to my podcast that's it Waze is for those who if they've learned that there was a way to get from point A to B two minutes faster they would be furious right so two different products both owned by Google two different customers sometimes you don't even know why you're a customer of one or the other but some of you didn't make a decision to choose Waze versus the other because of certain features who leaves voicemail messages who doesn't interesting right it's a product that like all of us encounter but she's not tuned to different customers who eats the Apple sliced and who eats it whole night whole sliced same product different customers who sets their alarm clock with just one and who has like n amount of I got one you got one who's got the N like what are we talking about like ten five nice so this one is not really a product feature but you guys are you it's like one customer cohort is using it or hacked it in a way that's going to serve them better just wake up on the first line it's like air B&B or hotels both right both is there anyone that's like strict I used to be very strict for a route for Airbnb I went and then they came out I was like that's it I'm only going for Airbnb I think when I moved to the u.s. I realized hotels could be better and then like I'm mixing but that's also because now I have kids and my wife doesn't like Airbnb so much I think if it was just me Airbnb all the way who pays to listen to music who doesn't do you still listen to music just not pay for it or not listen to music at all who's using like a free service like nice alright we'll talk about that alright my favorite one who owns a Kindle who does not own a Kindle for those who own a Kindle do you want to refine that do you want to like what's is there something missing in the explanation of what is an e-reader what is a Kindle what's different in the screen versus your phone it looks like print why so so there's something different in the technology right so an e-reader is basically has an e-ink screen right so this is physical ink that goes to the top of the screen that's why when you shut it off the image is still there but the idea is that the experience of reading on an e-reader versus reading on your phone is significantly different because of the technology that goes into an e-reader by the way technology has not changed in the last 14 years on it improved hasn't changed matured so my role for Kindle parts some other stuff but my role I have to customer cohorts I've got the prior owners those who already own a Kindle and I need to sell sell them a new one an upgraded one and then I've got the customer acquisition bucket so these are customers who have never owned an a leader right it's tricky we're selling the same device for a customer that might know what it is knows the value proposition can say oh this is a tank and we're also selling the same device if a customer who's like this is like my phone but it's not right so this is how I like to start thinking about customers as they relate to problems that we're trying to solve one what are these different customer cohorts into what do they know about my product what do they need to know about my product how is it different from each other I'm gonna walk you guys today through really really high level ideas of how to unpack a problem so to speak so usually it's a problem statement so when I came to and and this is what you will encounter I think for any product manager that will come into the the job on day one or will start 2020 with a fresh like like what are we gonna do next year a lot of times it starts with a sentence so when I joined music they said we've got prime music so these are prime customers who can listen to Amazon Prime music which was free if you're a Prime member and had like two million songs and then we've got Amazon Music Unlimited which is basically like your Spotify Premium right we need to get customers from this cohort to migrate to this cohort from free to paid from zero to ten dollars a month that's it like go figure it out and so I want to throughout like today when we talk about different things talk about three different companies and potentially what would be three different problem statements for each we're going to talk about Kindle so that's the one I'm doing right now so I'll have more insights for you about that and then we will talk about Spotify and we'll talk about Airbnb which I've never worked at so I don't know if I'm right so let's all share together our ideas on how to unpack these problems I don't really think this is a framework like I don't see this as a framework like the different supplies that we'll go through they're not mandatory to unpack a problem but this is how I like to think about things at high level like I've got a problem I'm gonna take a shower now I'm gonna think about all these different things and once I've kind of conceptualized what I'm up against then I'm gonna solve it all right so problem statement so we've got Kindle so the first problem we gotta is we got to upgrade existing device owners Spotify this is similar to what I've done with Amazon music so we got upsell premier tier to current free customers we got Airbnb we got upsell experiences so experiences on Airbnb for those who don't know so you booked a room now what they have you can actually just go to Airbnb calm like experiences our front and front and center right they're trying to sell you I don't know you you're in Madrid like a guide like you've got your you booked your house now here's a guide to tour the market whatever right all these different experiences so this is what we're gonna be up against the first question I think that you'll hear in all the PM books and everything and all the PM interviews is be ready to ask why okay cuz when I joined music they told me we got an upsell premium to current free customers by taking the data the metadata of the songs seeing how customers interact with that metadata and predict when will someone upgrade to Amazon music unlimited alright that's pretty specific so you go ahead and you start to solve that and you gather all the data and you start to you know do some regressions but how about asking why like why do we want them to do this because maybe the solution does not even like follow in the data itself maybe it's not about predicting maybe it's something completely different maybe it's marketing based right maybe it's what maybe it's just like discount base maybe it's expansion based who knows so that's why we're asking why are we doing this and when we asked why are we doing this there is the why for the customer that's external like what would the customer benefit if I were to do this and that that really ties to marketing or it really ties to incremental product features that will make and then you got the internal it's like how would we internally benefit from customers moving from point A to B and that's not always very clear a lot of times this is your finance folks right they're like we gotta save the business in some way maybe we should do X or Y or Z you got to know that they're in on it right because maybe it involves investments first so let me ask you why should we upgrade existing device owners on Kindle actual device so so one you're right right so Kindle makes it its money because you buy Kindle than you buy books so you've got your device it's working why and you're gonna make money off of books why should I get you the better one all of that so that's good right so straight-up revenue right but maybe you sell a device that are lost so maybe you maybe the profit aspect goes down you said something about customer experience unpack that a little bit potentially a new device has better experiences why because all of a sudden now there is light on the device or there is additional more better uniform light so maybe they can read better at night so maybe they're gonna read more at night so maybe they're gonna buy more books at night right or not at night but you know me so they would do it if you've got yeah they would do it so that's one thing she's like all right like another incremental feature that you may not have any older ones so if a customer keeps on purchasing they've bought into the idea they made an initial investment that's one way to retain them and that's true so customers who actually upgrade end up sticking around for longer right and then why do we upsell on Airbnb we'll skip the Spotify one why do we upsell for air wouldn't be like what's in it for them why would they want to do this this is a very different product than what they're built their platform on so improving customer experience once you're already traveling okay what about internal internal incentives do this okay so there's revenue there's diversification right so basically if tomorrow like I don't know they're like the you like in New York right so in New York you can no longer book Airbnb for no life if it's less than 30 days I think so all of a sudden like what if it happens in Europe like EU is says no more air B&B in Europe so you got to start diversifying that's one these are kind of different ways of thinking through external and internal travels because this will help you actually understand which cohorts you need to you need to kind of go after right so experiences versus customers who are actually going to book a place could be different so let's assume this is kind of like a small a small example that you guys will unpack so Amazon retail all right if you had to break down Amazon retail customers so basically whoever goes online and shops on Amazon how would you break down the customers to only two cohorts like you're either in this bucket or you're in this bucket in a way that would make sense for Amazon we got to make sure that we measure the right things so we've got our problem we've got go back to the Kindle Spotify and the and the Airbnb aspect what are the ways what are the I would call it columns or buckets that we care about that you would start measuring or do you would you forecast that that would make sense so I'll start with one because you guys mentioned that sales right I want to increase Kindle upgrades one of the ways I'm going to measure this is are the actions that I'm taking increasing sales like that's it's all like we're in here like this is corporate right like money needs to come in sales are are always there right is there any other bucket that makes a lot of sense that may not see initial revenue right away engagement is crucial right how would you measure if something that you're doing is working when there isn't an initial revenue coming in is that customer using my product more right are they listening to more songs per day are they skipping less songs per hour whatever right engagement I put some more up so we've got the sales on all of them engagement we can talk about engagement on all of them what about efficiencies this is more kind of internal so what if my cost structure for Spotify when I've got free customers is higher than if those customers would be would be premium I'm talking about cost I'm not talking about the revenue meaning I gotta pay for a free customer $1.00 and if they're premium I got to pay for them I don't know point five right those are real instances where you would want to move a customer from one cohort to the other simply for internal efficiencies this very much goes into operations right like what if a customer whose non-prime and you've set up all your systems to like ship the item in a day or two in a specific f.c fulfillment center and now you've got a lot of customers who are non prime and they're you know we can't like put them in the in the quick kind of operation mode of ship it a day or two and we got to wait for five to six days even if we might even if it might cost us more to ship in six to seven days versus shipping one or two days right like imagine tomorrow there are no more prime customers right I'm just making this up but everything is like to the extremes so let me use an extreme example there are no more customers they all became non prime and they all make an order tomorrow the FCS are just sitting there they're like alright let's wait for day six seven we can't ship it now because it's a prime benefit right so a lot of times I and III exact like this is a very exaggerated example but a lot of times efficiencies internally would would tie back to the problem and something that you will measure if you got customer service calls if customer service calls go down because of a change you've made in your product pretty good right if none of that happens but over time you're building a brand that's more likeable also good also an important bucket so all these buckets when you come up with a solution they usually all tie to potentially a different group in your organization right so you got the finance people you got the sales people you got the marketing people if you rally them on your product or on your incremental feature and you know exactly how to measure it right they're in on it because if no one if your product has immense efficiencies on the cost side but you haven't told anyone that that's what you predict will happen you may not get enough funding but if you realize that there's gonna be efficiencies you can say hey like operation folks why don't you put in a million bucks help me because it's gonna save you 2 million down the road right so it's all about finding stakeholders that would help you that would help you kind of get behind you to rally the product all right so I chose these as the examples I specifically said we're gonna focus for Kindle we're gonna focus on for upgrading customers were going to focus on the product for Spotify we're gonna focus on marketing to upgrade them from the free premium and then for incentives we're gonna focus on Airbnb right so we want to for every and B we want to diversify and sell more experiences so on the product side what is it it's basically hardware product features right or it could be incremental software features so essentially really like after we've created the strategy what are we actually going to implement right so if we've learned that a lot of our customers right now are saying like guys like you got to have USBC on your Kindles it's just not like all my phones have USBC you guys have microUSB just doesn't work anymore we're gonna make a hardware feature or it could be anything else right so fact that would be one what can you do on the marketing end for Spotify to get from from free to premium so that that ties a little bit to incentives right it's like I'm gonna make you do something and I'm gonna give you something in return so that ties I see that so it's a combination of marketing and an incentive what marketing is straight up we're not making any changes to the product like we're not adding lights we're not adding software features we just want to tell you what's out there right incentives for Airbnb what are you gonna do like how are you gonna get them to use more experiences again no changes in the product so for a customer who has booked the room they finished booking we know they're going to Madrid here's I don't know $10 off for choose your own your own experience or whatever so that's an incentive so everything we've talked about is down here the next thing I want to talk to you guys about is exactly the point that you made 2 million versus tell menu whenever you decide on what are we going to do there's always a downside like right like you're gonna upgrade people that we're gonna be like oh I can't believe there's 10 million I only got two but what about the others right so we always want to think about the actions that we take as product managers how do they affect our non customers right so at the beginning we broke it down we broke Kindle down for like prior owners versus non prime owners if I'm focusing on features that are really much like hard core focused on the prior owners and want to get them to upgrade and I physically change something in the device is there a chance I'm actually hurting customer acquisition right like if I'm gonna have incremental product features and it's going to increase the price of the device by 30 bucks and now I've priced out those who have no idea what this device is and they're never gonna try it at $100 they would have tried it at 70 but now they're not going to try it who are these customers we got to map them out right because when you think about all these incremental features that you're gonna going to implement you're always going to have people within the organization that are gonna be like whoa this is really bad for my customers like it's cool like you work for Spotify and you want to upgrade them to the premium you're in the premium you know the premium tier group but what about us we're but we're here to increase the engagement and the customer satisfaction for the free and you're going to tell the free that they're gonna like that they're two million sucks right it's really really you got to keep as a product manager you got to keep of the wide lens on who are you hurting and which customers are you actually benefiting and which customers are you actually hurting and in my experience there's always a balance there like just don't forget about the customers that are not yours within the organization because as you saw at the beginning some of you slice apples some of you don't some of you use Waze some of you use maps for really good reasons right if someone in Google just says we're gonna axe ways like we're only gonna go for maps and we're gonna put a pop-up on ways that's gonna say hey dude like you've got maps on your phone - you might want to try that one it's pretty bad for those who actually want to stay in ways right so if you're a product manager for maps and you work for Google it's pretty important that you understand the product managers that work for Waze pretty much all roll-up to the same place so these are basically everything that we kind of flushed out this is something I want to focus on a little bit because we haven't touched this we talked about promotions right what's the problem with promotions like I wrote it out here but basically like someone helped me flush this out what's the issue with promoting all the time so devaluing the product so one you train them that's great but then you said something about you attract customers who are what yeah let's spell it out so these are customers who would only go for the only take the incentive or the experience sorry when you discounted by 50% let's take an extreme example right so they took it you've lost money on these customers because I'm gonna tell you that 50% off of the experience as pure caught pure cost on your end you're hoping it's going to increase kind of engagement in the future you're not going to give it to them again but they're never never going to take it up again right so that's one when you talk about apps - so Spotify stuff like that I've probably opened Spotify on five different accounts right and I probably paid one month because I probably forgot to unsubscribe but if they're gonna give out if you're if you're someone who's like this is I like this product but I'm not a hardcore music listener now and then I want to like open the app and like tell it exactly which artist I want to listen to and you don't care about all of you or like playlists and connections and all that you only want to listen to music you're always gonna pick up that offer right so you got to be able to map these customers but the idea is when you promote it's very much correlated to lower engagement now am I saying never promote no right because promoting also gets you to those cheap customers that might not actually be cheap you're just not sure so they buy I don't know they buy a device for 50 bucks instead of a hundred they really like it they start buying content they're increasing their engagement and then they might buy more products at a higher price but there's one thing I really love that you said with the training of customers if you are if your incentive is predictable that sucks right because a customer it's like I would just mentioned for Spotify it's predictable they're always gonna have this like free three months for $0.99 like it's it's pretty and and it's always it's not like they do it at the end of the year it's always there it's just like if it's predictable there's a good chance engagement drops the promotion cycle for Amazon in general is predictable you've got prime day so how if you if you work for Amazon what do you do to flip that to change that but you should have a really good justification to say the amount that I promote to balance it out with the engagement that I'm getting is kind of like what I found found to be the sweet spot all right so potentially Spotify is saying yeah we're gonna promote all the time but that's my sweet spot because you're always gonna get the customers who don't care like they're gonna buy whenever not so price conscious you're always gonna get customers with what low willingness to pay so without promotions you're never gonna get them right so there has to be this balance once you launch we're getting close to the end of what are we gonna do to actually understand once we launch the feature or once we change the product what are we gonna do to understand are we actually right where we right basically track ourselves so you've got to store all their data you got to do the acquisition cost they're gonna fall off you're gonna send them an email later to actually come back to the platform you're gonna try to understand why did they drop off all those things cost so it's not always the case with software where basically it's like one extra plus one customer is zero also you are getting ads like you are getting revenue for from from people or for companies who need for Spotify to have this you know free tier right so there's all these elements I don't work at Spotify but I'm pretty sure they either got their sweet spot with the promotions or they're still trying to figure this out like don't make the assumptions outside that because it's a huge company like Airbnb like they've got it down to the science no they're experimenting all the time they may do something and at the end of the year say that was a bad choice we should have not provided incentives for experience just because no one's picking this up right so short answer is maybe who knows but something to think about so let's go back to these so this is kind of like a simple way of how I like to think about are we like how do we track are we doing the right things so you know for Amazon I'll use Amazon as an example you got the customer reviews you launch a product you watch those reviews like a hawk so I don't know for example experiences if there was a specific experience on on Airbnb you want to see if people are actually liking the experience those who've picked up the offer right picked up the incentive are they giving it five stars versus someone who didn't pick up the incentive and it's given him four stars right if you pay something for less you might give it more stars because it costs you less so you want to see hey if I'm incentivizing customers does it increase my customer reviews that's also a good metric to check then you got customer service that's I mean customer service is very good to basically flag issues right you launch something first day customer service is calling you back as a product manager like we've got this issue this issue they don't know how to set this up you don't know what's going on the promo isn't working this marketing thing they don't understand so they're calling us so you got to watch customer service because they're gonna give you the non quantitative data that might take a few days to flow in then you got the quantitative data this is queries this is like you got to set up the measuring this is like your BI folks right this is like the business intelligence folks this is your tableau this is your sequel this is everything that you've laid out to start to track trends post launch then you've got anecdotes I love this okay to me what this means in practice to me is at 5:00 p.m. every day I log on to read it to the kindle reddit forum and I just read and which is a very good like like there's a ton of people there I like I don't understand what they're doing but they're they're always talking about Kindle I read everything that was written that day I read all the comments and you start to understand stuff I can't tell you how many times I found I found issues with how this product is being sold and very like weird places like they're telling each other go to Target and like tell the tell the representative they want to they give them a coupon code then they go to the register they actually just give one they get a discount on you know like they find ways to like game you so that's one the second thing is the really third loyal customer so they're gonna tell each other what's wrong or what they like about the product so you're gonna say hey like I had an issue with them really not liking a specific next best read that uh that Kindle has suggested and they shared it with they didn't call customer service they just shared it right with each other and so right away I was like oh this is this this could be very very bad we got to reach out to the content team and tell them that Kindle is suggesting this specific book which is a not a book you would want to suggest and so these are anecdotes this is hard to get people like to focus on within the organization because these are very specific small things and they're not these huge data trends and people like data trends so you got to have both because if this is a story this is interesting you got to be able so if it's one customer that says ha this sucks but you also see in the data that fine but like 900 thousand other customers really love this and it increased the engagement you put that aside but if this customer is telling you about something that might break for other customers its flagging something or that could become really risky or maybe just get you thinking about something maybe a customer would say something I would read it and I'm like interesting let me go create a whole different query that I didn't even think about and be like oh this customer is actually behaving like 50000 of our customers every month right so I wouldn't even know how to look for this data without these anecdotes so you got to kind of break those apart and defined it competitive reactions is important you do something what does your competitor do Spotify does something what does Apple music do right if they react maybe you are onto something if they react differently maybe you're off of something right so all these things are also important I'm gonna end with an example that I wrote nothing about I just found this pretty fascinating right so this was Nordstrom return policy I don't know when but at some point in the past and this is the one I've screenshotted yesterday so in the past they say we don't actually have a return policy for purchases made at Nordstrom stores or Norse and calm super ambiguous like what does that even mean like as a customer shopping in North's room so so do I can I return it like do I just go there and I don't know try to return it like what's up now Norse erm is known to be a pretty good customer experience company they're probably going to accept it right but they made a change right this change didn't come from nothing like all the like this is this is probably I don't know how many legal folks are on top of something like this but they made a change as we handle returns on a case by case basis with the ultimate objective of making our customers happy it didn't really change much for you as a customer because you still don't know if you're gonna be able to return this but all of a sudden they do have a return policy it's on a case by case study right why do they make this chance I don't really care about what they wrote but like what do they make this change there was obviously some issue like there's obviously something going on right so my hypothesis is and it's I mean it's pretty common you're a retailer you want to reduce the returns right now that goes head-to-head with I want to increase my basket size right so as a customer let's focus your song online as a customer they want you to put more and more stuff in your basket before you check out right my assumption is as they increase that it's correlated with returns so they've got to find some sweet spot right so we're product managers right now let's assume that they want to reduce their returns okay that's our kind of like that's the sentence I'm giving you on your first day of the job reduce returns from Nordstrom online so why do we do this why would we do this because right because we first we first zoom out and we're like why very internal stuff right so far and it could be that it's only internal they're like we're processing all these returns its operation someone goes to the store potentially if not they got to go to FedEx there's a huge amount of cost on simply operating this thing who is the customer cohort if you were to break down Nordstrom shopping online to two customer cohorts I guess or whatever who would be the cohort for this specific problem I would suggest to break down customer cohorts by the those who are most likely to create this problem for you so if if some customer is putting in their bucket in their bucket and they're in their cart 10 Nikes with like different sizes different things I'm pretty sure they're gonna return right like they are their action is telling your they're going to return for sure so that's one thing maybe I'm going to break down my customers for this problem for n amount of items in a cart just made this up but this is something potential that you can think about an unpacked as as a product manager for Nordstrom online so how will you measure this let's say let's say we've decided that anyone who buys more than one item in a single category so like more than one shoe or more than two dresses or it falls into the high-risk and anyone else is not right so how will you actually measure this are we so once once you implement something how would you what do you care about how would you measure your success what I would do is really see if for the customers that I've identified and took action on have I changed their behavior are they did I now do something that will make the opposite and make them not buy anything maybe there you go they're like all right I can't buy ten shoes because something blocks me like maybe the solution is like a pop-up that says you can't add more than five shoes so they're like I'm gonna go to Nike I'm gonna get this at Nike right so all these things is how would we actually measure but the idea is less returns over a specific month for a specific cohort for a specific category for a specific time of the year for a specific price like maybe I only care about measuring the returns for items I've discounted by more than 15%

Channel: Product School

Views: 12,690

Keywords: Product, Product Manager, Product Management, Product School, Data Analytics, Coding for Managers, How to get a job, product manager salary, product manager resume, what is product management, what is a product manager, product management training, how to become a pm, product manager interview, machine learning, ai, Technology, Career, PM tools, software, metrics, product management basics, growth product management

Id: RGKvCaZkUnM

Length: 40min 49sec (2449 seconds)

Published: Mon Feb 24 2020