"Product is Hard" by Inspired Author Marty Cagan of SVPG at Lean Product Meetup

all right let's get to the main event here tonight speaker Marty Kagan I'm super excited having them back I'm sure like a lot of you you read the first edition of his book when you were trying to figure out how to be a p.m. and and improve how you did product management he's the founder of the Silicon Valley product group before that he was SVP of parked and design at eBay and before that he was at Netscape before that he was HP and this is the new book second edition if you're not familiar with it you might be familiar with the first edition and as I said we're gonna be giving away five copies and Marty's gonna be talking to us about some of the most difficult product decisions and issues that he's come across in his illustrious career so without further ado let's give him a warm welcome and welcome to tell us about park is hard or any of you here when I gave a talk last here few years ago I was kind of surprised security let me in okay but I'll let you ask good people but Scott Cook who is a legendary in the product world asked me to give a sort of a special talkin let's just say I was a little rough on their CEO but it was fun for me so it's great actually Intuit is legendary in the valley it's probably the original serious product company but what I wanted to talk about actually was this is this is not a normal talk actually there are a set of questions that I get occasionally that are just really hard know most questions I've been doing this so long most questions at the end pause like this - you kind of know the question already as soon as they say the first word or - and then you kind of know what the writing cliffs crisp clean answer but there are some questions that are really not black and white and I normally kind of hope those don't get asked but this stuff I wanted to just get them on the table and to be honest I'm sharing with you the topics I'm working on I'm writing articles on all of the things I'm talking about so they're kind of the ones percolating and on it you probably know it helps me when I kind of talk about it to think through these things but they're hard problems and my guess is you've thought or encountered all of them and I'll also say upfront that the hardest one of the hardest questions I get I can't really talk about because it would take easily the entire hour that is some because many of you I'm sure struggle with what is really the best way to split up our product teams and give you know who should what team should be responsible for what and that's that is a really complicated topic many of you know that already it's got about at least 20 identifiable major inputs or factors that you consider and it's one of those things that if we if we got in a room just with your heads of technology and product and a big whiteboard in a few hours we could have something useful I'm not the point where that's crisp I'd like that to be the case but that's a while out but the rest of them I think are reasonable to talk about and that's so I kind of broke it I I like to think of things in the product world as talking about the people which is what everything really arts with and then talking about the product how do you decide what we're gonna work on and then the process how do we actually do product work and then um in the new edition of the book I added another major area in the taxonomy which is culture you kind of realize everything boils down to culture and I've become seems like more and more convinced of this and anyway you already Dan told you about me so let's jump into those topics they're not really in any particular order other than this little little grouping so the first one I want to talk about is there's an easy answer to that of course what's a product manager actually responsible and we talked about the role and what designers doing what engineers do but honestly for a lot of people that's not such an easy question and in fact it's got several dimensions here there is the most common scenario is the product manager may be disagreeing with the rest of the team in particular engineers and designers that is that sort of question of what do you do then it's certainly there's the question the relationship between the product manager and is her boss director of EPA product then of course the one that most people talk about is product manager against that I shouldn't say against right but versus the pride that stakeholders the different stakeholders and that relationship but how do you actually who really gets to decide what and then of course especially in small companies the role of the CEO or the general manager and now those are really that gets tricky now one of the reasons it's tricky is because you'll hear this as a theme this is got a huge cultural impact some company cultures these are a lot you know you'll answer one way and another company cultures you'll answer have been a very different way now what I wanted to put out there is sort of separate from the culture what are really the characteristics of good product managers because ultimately the product manager at least in again the cultures that I would cut I personally would call a strong product culture where teams are really empowered but held accountable the product manager is kind of on the hook for the results so kinda the buck stops there especially if you believe as I do of the model of the CEO of the product not where they're the boss of anybody but when they're really are responsible for the end result and in that case sure you might say but I will also point out I do this to product managers all the time that's a really bad way to make a decision so how how do we really make a good decision in a in a serious product team with some hard problems there's a book that just came out actually that I considered like the best business book of 2017 I strongly recommend everybody read it's called principles by Ray Dalio it was the founder and CEO of Bridgewater and I will also say it's a company I've never even been to but they're a hedge fund company they're basically so it's not my world but I have heard of them because they're famous for running up some very serious cultural experiments and I won't take the time to describe it other than say it's bizarre it is a bizarre culture I I would love to just try to work there you know to see what it's like it is definitely infect a lot most of the people that try leave it's not for everybody and they tell you that they're very upfront but anyway the book talks about that culture and kind of gives the origin story of Bridgewater which is interesting but then he gets into really principles for how people should work just individual work principles and life principles and those were just gold and so I strongly encourage you one of the things he talks a lot about how do you make a decision and you know he's just the way he thinks he sort of turns everything into set of first principles and this is notion that he formalizes and I think most good product people do which is he calls a believability weighted decision and the idea is we it's not a democracy it's not like you you know engineers designers we all just vote on something that yep that's not what II or I is advocating but what are you saying is that if somebody has a competency a demonstrated competency in something that weighs more and sold for example I vote and I've been lucky I've worked with some rock star designers the kind where you always want to work with them you never want to have them leave your team and if the designer has an opinion about something to do with the experience like it's gonna take a lot to get me to not agree like unless there's a really big reason I'm like I'm gonna leave it same with engineers if you've got good engineers to sort of like go against that strong recommendation that's just you know it doesn't make a lot of sense now so those are pretty straightforward with the team this for the product manager to actually have that believability wait waiting they need to know what they're talking about and that's the big area where a lot of product managers struggle with this because they don't do their homework they haven't done the work and so then you have a situation in fact I tell them all the time a lot of times we have a case in a small company where the CEO is meeting with customers every single week and the product manager hardly ever does my view product managers shouldn't be able to decide really anything they can play backlog administrator they can do that they're not really in a position to make any real decisions and they hate when I tell them that of course but it's like do your homework dude you got to go out there you got to be there with customers at least as much as that CEO that CEO needs to and you need to share what you learned so that they know that you know a lot of what you're talking about so you are believable that's key it's similar with stakeholders my favorite relationships are when the product manager has done their homework if it's if you're working with a lawyer the the lawyer has explained to you a lot maybe some of the laws that are relevant here or the constraints and you have studied them you've done your homework and you're only proposing things you believe work within those constraints we do that with part with contracts with bizdev we do that with financials we do that with marketing we so you've got to do your homework and then this is a lot easier to do it's not even so much a me versus the stakeholder it's like here's what I think we should do and here's why based on what we talked about the floor it's a different dialogue and same with the CEO so I guess what I'm really saying is do your homework and then be smart about who you look to to make a decision find the people with a proven you know record with that one of the unusual things about the Bridgewater is they literally have people carry around what they call baseball cards you know like baseball cards it has the stats oh they actually this is gonna sound a little out there did to me stats like what is your track record on making decisions about that so yeah I'm not advocating company you just use that but it's really gonna be interesting to watch them okay this is huge common problem and this is one of those messy messy topics right so many product people now this is gonna be correlated with those of you that said you work in a medium to large company all right it's one of the nice things about a start-up I mean you credibly busy to start up but at least it's doing work and a big company you know you are you know often wall-to-wall meetings sometimes double booked meetings many times double booked meetings wha from beginning the day the end of the day and you know and this is the worst part is the worst part for me at eBay all these people working it was a seven-day-a-week place and I were I was up all the time and you'd get people who were working all the time and you'd have to tell them I know you're working like crazy but you're getting nothing done really I mean that's reality if is honest radical candor is what we would call that now right but that's true yeah and it's there at meetings all the time they're not getting work done I try to be very honest with product people product managers in particular this is what I'm about to say is absolutely true with engineers or designers but they if they have a trouble with this you have a big because there they are makers right they are maker but product managers are kind of caught between and there's a lot of making in and a lot of managing loosely turned and that's some and I tell them I do not know how to do this job if you can't spend for quality hours a day or real hours a day so that's with your slack channels not driving you nuts that's with your notifications not driving you nuts with not on email you and this is harder than ever right it's always been hard honestly but I think it's harder than ever the technologies unfortunately many of us many of you have done are really good and they they succeed in interrupting us and it really makes it hard to get work done honestly I've got to the point where if I don't turn on airplane mode on my phone I just can't get through anything hard it's because the things we work on take thought and and I don't mean like going off on a walk in the woods although that's not a bad thing either you'd be amazed how much you can think about but I mean literally if you're gonna sit down with your designer and engineers and customer you need to focus and I don't want to sound like I'm a parent lecturing I don't mean that I just I just honestly don't know how to do this job if you can't find four hours a day and there's only two ways I see teams do that or in product managers do it realistically one a lot of them especially in here in Silicon Valley those four hours are from 6:00 at night to 10:00 at night really a lot of them and and then the other way though which and I don't want to make it say I was guilty of that too you were guilty of that too it's that's but the better way I think is you stop going to all these meetings yeah used to do and I know that sounds harsh but and you know because most product people feel like they have to be at every meeting where their product is even hinted at and that is just you gotta get over that you do you have to let go cuz there's really not gonna be anything interesting to talk about until you get a good product anyway so one way or another but you got to figure that out alright yeah a four hour per day reality alright we can't find enough designers and engineers locally I put that up there because I I pushed very hard for co-located teams just the the results especially when we measure innovation of a team that's truly co-located that's product design engineering really sitting together not just say building like building nine I've been do it but in the same office right around the same tables the results there are just dramatically different than the teams that are distributed so that's just the reality now you know we can't always do that even in companies have tried very very hard they can't always do that and so we need to talk about what do we do in that case cuz you're not just gonna roll it up and go home we're gonna first of all yeah you will probably do remote employees that's our first choice many of you work in companies that have remote employed employees I will say that can work way better if the product manager Dan has been made a product manager designer and a tech lead how is our tech lead if those three can sit together and ha as tech lead can talk to the engineers that are elsewhere that actually works pretty well but we still get the collaboration that we need to figure out the products if we can't keep those three together things drop way down contractors again you know if you've not heard the phrase missionaries versus merchants mercenaries we need so much of what I do with teams and I know a lot of you care about is building missionary teams so much of that well contractor is pretty much the definition of a mercenary right so it's really hard to have that passion about the problem and about the customers if you're rented if that's what you know you're just basically a contractor and so I'm that's not our first choice if you do do contractors do it as a long-term engagement and definitely not statement of work stuff we are talking time and then finally agencies that is our third choice now some agencies are much better than others as far as working the way we need to work you're looking for people that understand that you boy you need somebody you're gonna invest in and then you want to leverage for a long time so messy but a lot of us have to deal with this all right we want diversity on our teams this is personal very timely I get this a lot most companies feel like they're either waking up or they feel guilt it into it I don't know all I know is most companies are finally starting to like you know when they all had some program in place pretty much every company but it was doing nothing right and so now there are actually under a lot more pressure to show some results and and their biggest complaint is there's no there's no funnel there's a sources to write there's nothing and and they're like how can we do anything we get literally three percent of our resumes or women or other underrepresented groups and so I and I I personally feel very strongly about this topic because we've seen for a long time the performance of diverse teams versus non it's just it's again it's all about great product teams and it really is true I mean a long long time ago learn a lot of people think you put a team of rockstars together it's gonna be awesome it's not true actually and the diversity of opinion diversity of mindset diversity of experience I don't want to sound politically correct but that is true and so then there's great data behind this so then the question is all right they're convinced or whatever reason they're convinced how do we get some people in here and I try to tell him it doesn't happen on its own you the higher managers have to get off their ass and go find these people it's true they have to that's the only way it happens you've got to go out there and find them and the thing is they are out there and if you're in a medium to large company you probably don't even have to look beyond your company I see this all the time there are so many people in the company that are so much better than the people that are often in the you know product organizations and you know there's all kinds of dynamics going on there in particular one thing that frustrates me about women is so many of them don't think they're qualified that's just their natural reaction on that qualified and I'm like let's talk honestly about the people that are in this job and you [Laughter] and a lot of times I have to like push them to trust me just like you are more than qualified in fact partly the fact that they don't think they're qualified to me is a good sign because that's the humility that's so important in product last thing we want is somebody that thinks they're just Eve jobs Junior and you know they know everything that's pretty much gonna be a bad product person so the talent is out there the other thing is it is so much easier when you start early as a startup because imagine you're you know you're a hundred person company you've got like hardly any women engineers maybe you probably have a few female designers maybe you've got to feel a product person so you find a great candidate engineer this happened to me just recently the company work I brought to them an awesome female tech leave Canada and she interviewed and she and I asked her what she sign after what she said there's no way I'm going there I would be the only woman in the engineering work here and there was maybe some QA but there was she was the only end she's like you know would you she was threw it back at me would you want to be the only like it's like this and she you know I can really make sense it does it's not what you want you want to start early get on healthy balance in your organization from the vegan makes it a lot easier as you grow the other thing that I just saw that I'll mention is Atlassian hired ahead of whatever diversity but that some it was interesting cuz she she wasn't uh normal she wasn't from HR thought that was so cool and but they and supposedly feel like increase the diversity hires eighty percent in the first year first round capital one of the cool blogs out there good blogs wrote about it and I and I and she so she shared her tips and I loved them I thought they were very useful so you might look that up alright let's get into process and I will also sort of warn you this is a bit wonky like you know policy wonks where process walks here if you came to this I'm that was true last year so I figured it'd be true this year you're kind of into the way we do products so some people use this term dual track it's it's it's Jeff Patton is sort of where I got it from you got it from one of the clients but the idea is this is basically the way I draw how modern teams work that there's objectives usually okay ours that drives discovery work where we figure out the right product to build and then we deliver it in delivery now um there is I used to in fact refer to this as dual Trek it's been a while I've been for a long time just calling it discovery and delivery but some a lot of times I'll get that question but but you know they'll google it and they'll see a handful of references and they're like what what's going on where where is this and that what's really going on is that so many companies use their own term for this and these are the main ones that sort of the original one was build the right product build the product right fake it before you make it that was for ten years at Google the way they referred to this at move fast and don't break things you probably recognized Facebook use it as well build things that don't scale and then build things that do scale Airbnb although it really came from Y Combinator and that is awesome I love that personally it doesn't really roll off the tongue but that is that's a great way to explain it because it really makes it clear what you're doing in Discovery you're not trying to have a scalable solution in the lean startup world customer development runs in parallel apparel a product development that's same concept in the Design Thinking world design sprints or discovery Sprint's run in parallel to delivery sprints and largely I just call it continuous discovery continuous delivery but the point is it's all talking about the same thing now this does cause another discussion that's really important I think it's a really important discussion it's like wait a minute there's like in one fact one person told me that looks like water so there's not like a waterfall so but that's an important thing and I don't blame people that asked that because they they don't know and they're really asking how is it not and this is this is really I think critical for product for really all product people to understand so let's talk about what really makes something waterfall I mean today it's just sort of a pejorative you're using waterfall you're lame all right that's what that synonymous with but really what is bad and I want to be clear most agile teams as far as I'm concerned they are waterfall so I want to be clear but what really defines waterfall there's really three things first and this is the defining characteristics the requirements somebody documents requirements even if it's just user stories used to be in product requirements documents and then that drives design designers do their work and then that's all handed to the engineers usually it's sprint planning and the engineers do their implementation if that's how you were which by the way most conventional agile teams that is how they were that is I would absolutely call that waterfall and that's that is bad second result yeah that is I'll try to make clear why in just the second part of this the result is all output that's another defining characteristic of waterfall meaning there's this process whatever is and at the end of the process you get a feature or you get a project and its output as opposed to a business result so that's the second defining characteristic and the third which is usually cited as the most serious consequence of using waterfall is that all the risk is at the end we don't actually know anything about what we have whether the customers gonna buy it whether it's usable whether we can really deliver it on time and the money we need and whether or not it works for the different parts of the business until we've already built the thing if that's how you work and even find what most people we think a waterfall like three months or six months it can be a two-week sprint that's still if those three things are happening I argued that is that's the badness of water for at least it's two weeks instead of three months okay but that is that is not what we're talking about so what does when we look at this when we look at this way of working you can see the reason we work this way is to avoid those three characteristics literally those three first principle of this way of working is discovery is all about tackling the risks upfront we are so number the maybe you don't remember but the four main risks in product or value risk would people buy it usability risk and they figure out how to use it feasibility risk can we actually build this thing and viability risk will it work for the different stakeholders in the business in product our obligation is to make sure the stuff we actually have our engineers bill is we've addressed those risk it's gonna be worth building and so in Discovery that is exactly what we're doing tackling those risks up front second we don't have a product manager that writes requirements or stories and throws it over to designer and that gives it to engineers instead we literally define collaboratively what that means is product manager designer tech lead sit this close and they work it out together and instead of implementing features from a roadmap they're solving business problems usually from an okay arm that's that's literally what we mean by collaboration and the third point in this way of working is if we have to do ten different features in order to get a problem solved that's what we do actually just was I just earlier today I was with Google cloud team and somebody there throughout the number that right now they're running it about they have to try ten different approaches for every one that works is that and I've been using the number more like you have to try to for everyone that we're kind of a but 10 that Sun you know they've got a pretty high bar there but that is the point is that's all out we're looking for results if we are measured by results instead of shipping features then it rarely takes just one try usually many tries but we want to do those tries here not here so that's where that comes from the one more related to this that this conversation usually brings up the some of you may recognize this diagram it was done by Henrik Newberg he's like very famous agile coach because he early on helped Spotify and I love Spotify actually and I think they're very good but he drew this diagram if you haven't seen it it's it's an important one but he was trying to tell a jille team's look some teams will say we're gonna have four Sprint's we're gonna bill like in first sprint we're gonna build some infrastructure second sprint we're gonna build another layer of the platform third we're gonna you know start building some services the force we'll put a user interface on it and he's like that's not really the point of agile and by the way I think he's absolutely right there but what he says what we do is we every sprint you start with something useful something useful and a lot of people get hung up on his analogy because it's pretty obvious that you wouldn't start with a skateboard if you think you need a car right but Grant loosen up on that stuff and just say that every week you know we've got something you can ride and something you can get from here to there now if I meet a custom software organization usually I do or an IT internal you know the kind where you don't have product managers you don't have designers I point them at this and say as best you can do in a product company this is weak and I want to make sure you can see why the way I like to phrase it is if you've ever heard the old saying if the only tool you have is a hammer everything looks like a nail well if the only tool you have are engineers this is your view of the world you are using the engineers to do all of your learning the funny thing is this diagram is the antithesis of Lean Startup absolutely it's a complete waste now let's talk about what we would do in a modern product team here in discovery we would do these iterations let's just pretend again that we believe those are the right kind of iterations but we would do them but instead of taking whatever six months or it was really a car probably years but whatever so instead of five significant things maybe it's ten weeks let's just say instead of that we would do this in two weeks and we would do it fast because these are just prototypes that's what discovery is is we're using prototypes there's four main kinds of prototypes but we'll use whatever and we would figure out very quickly whether we needed an engine or not or whether or not it had to stake luggage or whatever we would figure that stuff out because our job remember this is what I was saying before in Discovery is to test value usability feasibility and viability and not to progress to using our engineers that are really there this is people and by the way this diagram I find even though this was my original intention but it really makes clear why so many teams are so confused over the concept of MVP because MVP from the product manager point of view they're using it for this from the engineer's point of view they're it's being done through this they're like nobody wins because the engineers think it's really crappy stuff we're about to launch product people are saying it's taking way too long so that's what happens when you use the same tool for the wrong job so the engineers are responsible for scalable reliable performance Hannibal Europe may be an early start up it might not be so much of an issue but for most of us we've got customers running their business or their lives on our products and you can't you will you need to let our engineers focus on this this is their highest order use and so we want to make sure they are focused on that but we don't want to do this for discovery as way overkill is what takes so long till we use one of them several kinds of prototypes in order to test to validate these things and as we realize that there are pieces that are ready to be done commercial production quality then we put it on the backlog now I did I took one other Liberty with this diagram that I probably didn't need to do because what I could have just shown is let's say you decide you know ultimately in this hidden Henrichs scenario they do want the car I could have shown well the car being delivered but what I wanted to show is that you know what if the engineer once you're sure you need a car so no more games about skateboards or motorcycles if you're sure you need a car the engineer should be able to build that car the way they think appropriate I can pretty much guarantee you they are not going to build like four different power systems they're gonna decide the one that's right for the car and they're gonna build it and this diagram is actually closer so they might make some architectural decisions then they might actually build out some services then they might build out some interface they can do it in the order they think appropriate the other thing I would point out though is if you work this way this amount of time is generally significantly less only because of it all you've probably heard that old saying a problem well-defined is a problem half solved one of the frustrations and typical agile and why sprint planning is such a painful meeting is because nobody knows what they're talking about their story says so little product manager doesn't know yet what they're trying to describe so there's a lot of unknowns and so things drag on if however the product has been validated with prototypes and I have to emphasize in Discovery the engineers everyday are along with us as we learn it just doesn't take much time generally takes about half hour for the engineers to participate in the learning and discovery so that when it comes to build they know what it needs to do and in fact if there were feasible feasibility risks they would have explored those here like let's say they're not sure like literally Tesla wasn't sure that a battery would carry a car 200 miles they explored that here and then of course they built out car so that's a really important concept for us this comes up a lot okay hopefully that makes sense yeah then sometimes we get kind of the other my engineers just want to build and they don't want to participate I've been saying for a while that if you just use your engineers to code you're only getting about half their value now most engineers I know love when I say that especially to the CEOs because they know and unfortunately often they're right they think the C CEO thinks it's just they're there to cover and a lot of product managers unfortunately think they're just there to code fact they think their job is to shelter the engineers from the ugly of the world they don't want him to see customers in pain they don't want them to have to hear about business analytics analytics anything they don't want him to see RIT they don't want them to stress their you know stress over it so they and but and sometimes though the engineers say they just want to build and first point I would make is that they should be building most of the time roughly ninety percent of their time they should be building just like a designer should be designing and we want to make sure they are makers for that but we also want to make sure that they are spending some amount of time on discovery all the time they need to be part of and that's because one of the higher-order principles in product work is shared learning it's you've all seen the problem when the only one that understands why something is necessary is the product manager and then they throw it over the wall the engineers and they're totally not into it they're either outright skeptical or they're gonna just basically go at their pace and it's our job and product to motivate evangelize and the main way we do that is you've got to bring them along for the learning makes all the difference and and by the same thing why product managers get so frustrated when a CEO just tells the product manager just do it because I said so the same thing all right also yet don't shelter them in fact it's counterintuitive to a lot of product managers but one of the most motivating things you can do to engineers is to show them the customer in paint show them the customer in pain it is most engineers are very personally embarrassed by that by though even if they didn't write the offensive code it is they feel like this is our company because this is bad I'm gonna make sure this is and it's very motivating for development and then remember just even if it's 15 minutes a day to keep them apprised of the learnings of that day what what you learned at a customer what you learned by talking to the lawyer just keep them up to date yeah we're in a regulated industry I hear this we're in a you know we're a big conservative company how can we actually work this way and my view of course is you don't have a choice if you either I should include there's a great read is it Reed Hastings or Reed Hoffman I think it's a Reed Hoffman closed both that actually says if you don't take these risks it's only a matter of time but the idea here is what do you do well first of all we use opt-in technique now opt-in techniques are not our first choice because they're not as predictive as blind techniques like an a/b test but you know what there are really good evidence and we eliminate that issue we lawyers are fine with this as long as the customer opts in so we go to certain customers say we've got an experimental version would you be willing to run it or even ten of your people willing to run it or whatever it is and those are the ones we include and collect the data on it could be qualitative learning or quantitative learning this is mostly talking quantitative or qualitative we can talk we can have them sign an NDA also we lower our expectations of the degree of confidence when we do quantitative testing because we're not gonna with an a/b test you can get large amounts of traffic but in a an opt-in technique for example or an invite-only technique we are getting smaller numbers so you can either wait really long before you have a statistically significant results or you can just lower your confidence and say we're gonna make this decision not based on proof but based on evidence and that's what we do and then you have to lean or you can and should lean more on the qualitative technique we have great set of qualitative value testing techniques as well as quantitative techniques so you can absolutely do that again it's not as predictive but it's incredibly insightful so we can definitely do that all right product topics all right product topics our execs are addicted to roadmaps yeah maybe the single most common one I hear and this is a problem for sure I hate to say it but there's a lot of product people litter exceed the roadmaps - I always though try because to me that's disappointing because they seem to like understand the principles behind things like Lean Startup but they don't understand that that's the same problem the same problem in it's just as ridiculous as an MBA doing a 50-page business plan with a bunch of assumptions product manager having a bunch of assumptions about a bunch of features on a roadmap actually working if you believe Google's number that they have to try 10 different approaches before one works then what does that tell you about the likelihood of your roadmap producing results you can count on about 10 percent of them so it's time for product people to wake up to this but anyway I'm talking about execs and that is the more general case and also I I understand where this is coming from I'm not blaming they basically there first of all they're trying to run a business and so they really need to know when things are gonna happen if we're gonna they don't care about little things but if we're they don't put we don't put little things on the roadmaps either right we put the bigger things on there and if we're gonna have something and then marketing it's gonna have a big promotion or spending or a partner's got to plan their time they're trying to run a business and a roadmap even though they're skeptical at this point on whether that dates are accurate they at least get something and so that's important and second they want to know we're working on the most important things and not just going off and having fun and so those are the two needs that roadmaps give for them and I and that's fair we need to acknowledge that so one of the easiest ways that I advocate teams to wean themselves off of their CEOs addiction is to our is what's called an outcome-based roadmap also known as transition roadmap an outcome-based roadmap is basically you just first of all you have to plan about a year doesn't happen fast cuz cult this kind of thing doesn't change the CEOs don't change their mind instantly on stuff like this but if you plan about a year keep doing roadmaps just like you're doing it today the only difference is immediately go in for every item on your roadmap if you're gonna integrate some payment system or whatever it is on your roadmap figure out or ask whoever you need to ask but figure out what the business result is that would hopefully get better once you do this so if you're integrating a payment system to increase conversion rate then find out who's asking for this and ask them what conversion rate they're expecting so you're telling us to integrate PayPal today we're at four percent what are you expecting once we integrate you hopefully it's somewhere it's probably on some business case somewhere but they should know and if they don't know you make an intelligent choice of what it probably is and just put that there say okay integrate PayPal is the feature on the roadmap and the the KPI is conversion rate and our target is up to six percent and here so that's and then you keep it there and every time you talk about that roadmap item you make sure to talk about the KPI and then the second part of this which is really the most important as soon as this PayPal integration goes live we start reporting on that business result though that will make very clear if you're like ten times smarter than Google and everything works awesome you're heroes otherwise you can start to see is like what we launched the PayPal integration we verified it is working correctly unfortunately it has not moved converting right and so but we do have other ideas to improve the conversion rate and it's this process of it is a bit ironic because basically this is what we're talking about here is how the Board of Directors works they care about the business results and the level of features and stuff is really introduced by that layer of management and again we understand why but it's important for us to try to if we really if you want an empowered product team you've got to give them the problem to solve not the solution you can't blame them for conversion rate not getting better if you don't if you tell them it's not negotiable you got any great paypal that makes sense to everybody basic but you can see a big disconnect there every team use and roadmaps and supposedly use it okay ours most okay our implementations are silly yeah they're not doing but I am I am bullish on okay ours and I'm going to talk about it alright I thought this was a funny one recently I started this happened like nine months ago or something and immediately I started getting emails and questions when I visited a team mostly designers were really bent out of shape about this I don't know how many of you saw that but it caused a lot of swirl um now in fairness he works at an agency and at an agency you don't have product managers really as your client you have a bunch of what we would call business owners usually in marketing and they change their minds they give requirements like crazy many of which probably most of which drive the designers nuts and they know that when you know if they're given those requirements it's gonna be a bad design my guess is he was having in one of those bad days where is were really frustrated with these crazy clients and you can't like that's your business but unfortunately this is the most common complaint I hear from designers at agency that they kind of they're hired guns they have to build what the client is telling him to build of course I and he was giving the example yeah an engineer built something some way and it was now seven seconds response time and it just ruined our design kind of thing so his a odd social the engineer is a designer they might not know it but they're a designer because they just you know boot up the design and that is um and my point is he's confusing two things I'm giving him as a benefit out my guess is he knows this but I still think it was a very ill-advised way to phrase it because there's no question these people are not designer they're not they the engineer does not a Dyer do they do things that can impact the design absolutely so Ken oh by the way so Ken what lawyers people doing pricing changes the competitors can mess with our approach sales case sales people and their capabilities mess with our stuff government compliance changes laws mess with our stuff privacy brand issues financial constraints security well these are all real contracts these are all real um this is our world and this is your world right we're talking about this is what product managers deal with the point is the reason product managers are so important is because requirements come from all of these areas I hate that word I'll be honest I and I go there's like in the new book especially I don't even think you can find the word except maybe this is you know there's not requirements are not really requirement the job of product manager is to figure out the underlying constraint what is really going on remember every customer is gonna describe things different all of it what we need to do is identify constraints now and of course just to be clear what designers do is solve curve floor constraints that's what they do well I love them engineers solve for constraints too but they're doing different they're solving for implementation and designers are solving for experience but they're solving for constraint they can deal with a nine-second response time as long as they design the experience around this delay and so we need to we need to be clear on that and I'll saw it we need to make sure the product managers understand you've all seen the Dilbert cartoons making fun of product managers and the idea they're the ones that just gather up requirements and pass them on they are not helping because all they're doing is passing along their requirements they're not doing the work to understand the underlying constraints this can radically improve your designers ability to do a great design and your engineers ability to come up with a great solution all right there's one but everybody has struggled with this one well unless the early startups but it doesn't take long before tech debt is a big issue and in bigger companies a lot of them are crippled with tech debt and so um this is just you know this has always been a thing even when like early eBay lost a third of its market cap through major tech debt issues because people lost confidence and their ability to keep the site running but luckily they survived I was at Netscape as you saw the lead died because that's why we lost the browser wars actually so tech debt has always been a thing the internet combined with agile combined with misguided lean startup usage has made it even worse and so today I see more of it than ever and so we need to talk about tech debt first of all the company needs to treat it as it is which is a business continuity issue this can literally kill companies now don't get me wrong I've never been one to say you should have no tech debt or anything like that no there's a reasonable amount of ongoing tech debt and always working on that tech debt is healthy but way too many companies let it get way out of control and basically when it gets it doesn't even have to get to the point where you have major outages if you get to the point where you cannot respond to the market and where you can't take care of your customers because it takes your engineers so long to build anything and by the way your engineers will typically say at my last company I could have done this in a week here with these systems it's maybe a month a lot of unknowns because every time we touch this we have a huge problem so ultimately it is the engineers well I should say this be careful it's the CTO ultimately it is the CTOs job to make sure you your systems your architecture can meet the needs of the business now a lot of the CTO is complain to me because they're head of product doesn't let them work on tech debt and my answer to them is you're wrong for asking the head of product as you know and to be clear there's lots of examples of this I'm not talking they're the one that's gonna get fired they're the one though it is not up to the head of product this is the CTO or VP of engineering's responsibility but first foremost you've got to treat it as a continuity issue it's a serious thing if you're in serious trouble you need to make a hard call this is never fun but I always tell me I'll up your engineering capacity dialed down product and design product and designing gonna be able to do much anyway in fact you can usually see a sign of tech debt it's not definitive but it's a good indicator of tech debt if the backlogs are too long if there's like more than a month or so of work queued up you're out of balance the engineers are not able to keep up not for sure but very likely due to this so you got to get more balance we got to put as hard as we can to get that architecture to where we need it to be and which means for a product people you have to pick your battles you cannot do everything you want to do for your product yeah which we never really can but you can't even do the what we're used to settling for you will probably get only about half of that which means you don't get much in a quarter which means you better pick wisely better make sure you're picking things that move the needle so that's a hard problem how many of you are using okay ours no okay not half yeah I am a big advocate of okay ours but and this drives me nuts because the truth is I am hoping for a better system to come in the next few years I do not think okay ours I certainly hope they're not the last word because too many teams have way too much grief when they move to okay ours now even that I even then I still advocate them I've been using them myself even that was actually taught at HP it was the predecessor version which is not that different but it's called MBOs but it's um I'm sold but again I think I realize that why it's so much grief you know it's conceptually these are not hard alright okay ours are not hard but in practice the reason I think they're so difficult is because they force the cultural issues to the table so the most obvious one is okay ours are supposed to be chaos are supposed to be key results but that our results business results most companies fail on that the main principle and they just put the features from the roadmap it's all it is is a cosmetic reference from the roadmap they take integrate PayPal from the roadmap it said q1 in a great PayPal they move it to their okay are for Qi the integrate PayPal that is missing point of okay ours the ideas you have on a business you have an objective or the business like increased conversion rate and you have a measurable business result get that conversion rate to 6% integrate PayPal has no place on an ache okay art that is one of our I don't get me wrong I love pickles great sometimes it depends right it really depends it's good for some companies in it I use that as an example because so many companies I know have integrated PayPal and got nothing and that's not really PayPal follow it's not they sound like it broke it's just wasn't the problems that needed to really be fixed they already had ways to pay usually the issue is people don't want to pay I mean that sounds you know flip but it's true it's really value right they don't really want this and so it's like oh would I got this but I didn't have paper so anyway um that's what okay ours are about but you can see if you really do that that's gonna force some cultural change like execs and their addictions to road maps again so um it brings to the surface these issues the other thing I want to say though I also admit to you I do not recommend okay ours to everyone because okay to really do ok ours to really give a team business objectives and empower them to solve that requires a pretty high competency of that team if you've got let's just say novice team you know not really with the experience that's necessary they're gonna be like a deer in the headlights they're gonna be waiting around until somebody tells them to integrate PayPal billion so you have to have a pretty good so really ok ours located sorry predicated on good level of competence of the team and the organization so you've got it want to truly empower the team and be willing to do that but to me that I mean those are the only kind of teams I want to work with I mean I'm just to me life is too short those are the teams that are gonna really do great things those are the teams I want to help but you know so I see a skew of the of the industry but that's really I think why okay ours or okay ours are so difficult because they are forcing this issue it's not a conceptual thing it's cultural which leads me to the last topic which is culture so see this all the time as we grow we're getting more risk-averse I bet many of you have heard that um now first of all you got to realize that there's a lot more to lose that's one of the nice things about startups nothing to lose you don't have to worry about stuff look at remember Airbnb just having fun did you notice their listings in San Francisco this week are half what they were last week why because now there are a company that people care about like Marriott and they don't like them and so they lobby against them and sure enough Airbnb has to follow the law now um don't get me wrong I love our Airbnb but they they now have to deal with business viability issues like Marriott does or like other big companies do and so it's as you get bigger there is a lot more to lose that doesn't mean that we don't need to move fast and take risk but we have to be sensitive to that this is where when again when I talk about CEO of the product I am talking about that product manager understands all elements of the business I recently started doing something with the product managers I coach which is when they're just starting out on a new product I'm having them fill out a lean canvas for their product it's already done but I mean the products already there but it seems like it's a great way for them to realize this is not just about giving a backlog and grooming it for developed this is if you're gonna make good choices you have to understand how sales works how marketing works how the financials work on this what are all the contracts that impact us what are all the business constraints that impact us ah if you don't know that then you probably are gonna have to escalate some all these decisions up to some either CEO or some other executive in which case again you're just a backlog administrator not a product manager so yeah its product managers this was part of that do your homework you've got to understand these elements the business ok my engineers just aren't excited about what we need to do this is they all you know most product managers know they need their job in fact it's their responsibility more than anyone else to make sure their team is a team of missionaries so what can you do you can bring them out to see users and customers in person preferably in pain yeah which is not that hard a lot of the times were asked to go to customers it's because they're having trouble but they're all it's always good although it was in the latest Jeff Bezos shareholder letter and their gold by the way if you don't read that every year you absolutely should it is his literally gift to the industry every year and one of the things he says he had this great line in there too which is not no customer ever asked for Amazon Prime which is true right that any good innovation is really it's not coming from the customers that's a whole other talk but and he he said you know one of the great things about going to customers is they are always unhappy or always really and you know he's trying to say there is always more they want always more they need it's like that's a good thing that's not a bad thing it's a good thing so yeah get out there and then share the full business context with your team everything share with them your cost per customer share with them your lifetime value share with them how much money you're losing on every customer if it is share with them the economics of your business whatever any part of that context that helps them and it motivates them especially when it's difficult which it usually is because that is motivating to developers these are these are as a general rule very smart people you know one of the things that's always so I did spend most of my career I had one exception to that which was eBay everywhere else in my career including and by the way just hit 35 years so long career but has always been doing products or now services you could say I coached teams with teams developers mostly but not in consumers it was only at eBay that I was really doing consumer services but they and enterprise it was developers and one of the things most people realize there is one thing that's been a constant about our industry is there has been a constant stream of great innovations for developers I mean you have to look further than AWS for that but I just hit 17 billion dollars amazing anyway why is that my belief is because developers have constant access to the customer you could say well they're the customer but it's beyond that it's that their friends are the customer the people sitting right next to them are the customer and developers when they see somebody that has a real problem they want to fix it they want to come up with a better solution that's my belief for why there's so much innovation in the developer platform space and people say well they're just not motivated to solve problems for the enterprise or for the finance side it's totally not true because the companies that actually bring those developers to those finance people in these other get just as motivated and just the same level of innovation though what's really going on to me is it's about connecting them to the customers in pain not only the engineers but there's a big one engineers are they're consistently our best source of innovation remember they know the technology better than anyone they can see what's just now possible before most of us and so you really want that's when I say the magic happen all right my CEO tells me what to build it just trust him a lot of people product people complain about that and yeah that's an issue because well first of all very often the product manager has not done their homework and so you know you kind of can't blame the CEO for that but even when they have done the homework a lot of times they haven't shared what they've learned so the CEO doesn't really know that they've been doing this every week and learning about the customers also so you know one thing that really irritates CEOs is they've they know what has to be done and I'm saying let's now for a minute grant him the benefit of the doubt there right this has to be done we do need the integrate PayPal let's say and they say just integrate PayPal don't argue and the product owners like well we have to do validation we've got to go and prototype this and validate it make sure and that's really frustrating for them it's super frustrating for them it's like you know really we're gonna lose you know if the team's good maybe it's only gonna take a few days or a week but it could be like two or three weeks and like really you know especially if your startup and you're sort of trying to race for survival a lot of CEOs will won't tolerate it and even if they do they're really bitter about it I try to tell the product managers be smart about this pick your battles you can be learning things as you go and go out to customer you say yes let's go do that and and as you go forward you can do some validation as you're going forward oh maybe this isn't the big one so worst case you've got a like a lot of other ecommerce sites you had a PayPal button that nobody use it's who cares it's not gonna kill your business all right and then be very transparent about what you learn share it that's critical and and finally especially with executives two things prototypes especially I know I actually already told the boss salmon guys I love boss salmon but in this case I want high fidelity prototypes with executives I want high fidelity because it's in executives care I've learned not to show my balsamic prototypes to executives most of them cannot get past a wireframe now I kind of finally figured out why they they care about the brand it's their baby and wireframe is sort of intentionally not showing that now some of them they just really have no idea about design and they're really uncomfortable but most of them they want to see what it's really gonna look like that that's the bottom line so you either have to die decide that now you're gonna deal with it now or you're gonna have to deal with it later but especially with these executives high fidelity prototypes are fantastic and there are many tools that will help you do that not as fast as you can deal with all Sammy prototype but fast okay and then data its Amit this is really the big difference in product between how it was done years ago and how it's done today but data is amazing it's sort of the big equalizer you're you know it's not like it guarantees you that you will win the argument because it's just part but it's so different because when it was my opinion versus the CEOs opinion you know I lose if it's who knows but let's look at this data it's a very different discussion so one of the things I learned and my go-to answer for things where there is a disagreement it's just run a test and kind of one thing I specialize in is lots of fast ways to run a test that don't take forever that we can get some evidence quickly all right almost there Dan this is the last question but um I actually think one of the hardest questions of all which is is it even really possible to change culture because a lot of a lot of what we're talking about is culture hopefully you'd agree it's culture and so that is no question it's super difficult to change the question is is it possible first thing I want to emphasize is that product culture what we mean by product culture this sort of willingness to run lots of tests to make decisions based on data to do lots of validation to be humble that you're probably wrong on a lot of these things in our job is to figure things out to know what you can know these are all what we mean by parts of what we mean by product culture that's a subset of broader company culture so for example Amazon is probably the best company I know for product culture is not a company culture for most people anybody who tells your otherwise just not being honest and now some people I I just learned the kind of people to introduced Amazon because they have to be the right personality they have to like not have significant others or children or anything it has to be like I want to work constantly and learn more than I could anywhere else I want to do that it's so great for that it's a rough culture but oh man are they awesome at product and and other companies are you know Oliver those are different dimensions so they can be all over the place of course my favorite places would be graded both I would argue Netflix is pretty awesome at both there's other companies too Etsy is pretty awesome at both and there's different cultures that's a whole dialogue but here's the thing with I mean we let's talk about uber because uber first of all most people I I certainly think they have an awesome product service is awesome the company nuts has not been awesome and it's kind of been the poster child for the worst of Silicon Valley and I am personally thrilled even though I am normally I tried very hard to help the founders stay in place not in this case and so I'm thrilled that they made a change the challenge for them is thus thousands of middle managers that were hired under travel that's the challenge and by the way all the people they hired so it's you have a large company that has some people in there that are just thrilled that can finally be a place they're proud of again and but they have a lot of places you know this just this is just the way culture works it's stuck in these middle managers and it can take many years to kind of evolve a culture really to change so that's what makes it hard and I definitely hope they do it and I know quite a few mm people that would never consider going to over before under Travis that are now joining many of you probably do there's a lot of good people that are joining that's like the best thing but there'll be sort of a struggle which they know or at least ones I thought prepare yourself because you're gonna find the people that even though they won't say it out loud they are a reflection of the old culture and there's this desire for the new against the entrenched old and you're gonna deal with that that's why it's so hard to change culture I do think it needs to start from the top that's why that needed to happen with Travis but on the other hand it doesn't actually change for the people or most people what really impacts the culture is their manager so they got a deal with that and that that's kind of where a lot of the culture discussion ends up going but the nice part about that is that just focusing on getting better at product is usually a subset that we can manage the hardest part of that is usually the okay ours and the roadmap all right those were pretty random set of topics but hopefully you know hopefully you found that useful you can see the things I'm thinking about working on and I'd love to if anybody has thoughts on them there's my email right there and I always love to get question comment anything I'm happy damn do we have time for any question you definitely have time for questions the way this works though is you have to raise your hand and well someone with the mic will give it to if you don't have a mic don't ask that question that's how it works all right this guy's raised his hand good job there you go thanks Martin let's get more TV grounded that's a great talk [Applause] my spouse she's also in profit management my question for you is you know you've earlier this I saw your bio which was you're in eBay you didn't platform then you were and design at Netscape can you give a sense of the differences between platform product management and the application level product management of the user facing product management and how are the challenges in the sure yeah yeah I I kind of did that well first of all most product managers are not platform product managers they are either they're they're what we which is called product manager might be an enterprise product or a consumer product or a device I am most a lot of my time was at the platform level both engineering and and product and there yeah there is something called ppm platform product manager the other thing I would well first of all the most obvious difference is it's very technical product management you know most companies you're not a candidate for a platform product manager unless you're a current or former developer just because there's you that's who you're doing things for and there's just a high learning curve otherwise and in fact it's also true that in a lot of good companies that are and we're talking about platform services mostly api's is the interface mostly that sometimes the product manager is the tech lead they it's a dual job it's a lot of work but it's a dual job remember at the platform level you're really meeting the you're there to help all the other teams do their thing any go back to over they have a huge platform services team and they're there to help the Rider teams and the driver teams deliver great experiences for riders and drivers which is also one of the differences in normally if you're in a regular product manager you're okay ours are pure business results grow the number of drivers make them happier these are real KPIs for drivers on the other hand if you're in the platform team you're really there to help those teams so it's we kind of bend the rule on business results because you're gonna really be they're all some companies they kind of do in between and and just measure the satisfaction of the other teams that are using their stuff but yeah we loosen that it's it's I think the hardest kind of product management is platform because you have so many constituents then you have very demanding requirements because everything above is sort of rolls down a lot of hard core stuff that's where a lot of the machine learning is going on right now that's where Big Data stuff goes on that's where things like authentication goes on and so yeah it's it's hard it's I think it's great it's hard I'm trying to think if there's any other relevant difference of course most API or micro service teams for example don't have a designer on the team most of the time that's okay because you're doing api's that don't necessarily manifest in an interface sometimes you wish you had a designer because you are doing things that eventually show up but most don't have them because there's barely enough designers to go around for the for the user facing team that's a difference - there's also the ratios are usually a little different there's a more engineers per product manager sure this question right here excuse oh sorry Thank You Marty you know as I think about a lot of the topics that you mentioned from and most relevant to about KPIs and the lack of the the fact that you get a lot of business stakeholders telling you about features as opposed to the KPIs and the business results that they want drive the one problem that I did not see or that one of the things that makes it hard for product managers to create that focus on what problem we want to solve for is prioritization and despite all the number of ways that we have to prioritize and Dan tell teaches us many of those ways in his program if we don't get ever have that culture of prioritization and how do we evaluate and stack rank and compare things and to know what are we gonna do first second third and maybe not even the rest you know what are your thoughts on that okay good question but um so first I should be very clear there's prioritization occurs in many levels of a company there's business strategy which has priorities implied like when are we gonna go international and things like that and I don't think we're talking about that and there's product strategy which is like what target markets are we gonna go after in what order I don't think we're talking about that you're probably talking about how do you prioritize your roadmap and maybe this or hopefully this won't surprise you but I think this is that's you're already up the wrong tree that is remember what I said let's use that Google example is it fresh in my mind from a few hours ago it's gonna take ten different ideas to solve a problem it's not about prioritization it's about trying lots of ideas much faster the way we do so there is a priority which comes in in the okrs and not everybody does this but a lot view most people in do OPR is it's like one two three objectives and one two three key results for each objective I push the leaders to prioritize those objectives and key results because that will be important to us but once you know let's say you're the CEO I'm the product manager you give me my three prioritized objectives I know what I have to work on and it's not about a roadmap it's about trying out 40 ideas this week and figuring out at least a few that moves the needle so it's not really about that then of course we have to prioritize what goes in the backlog the build but that's really much more for our engineers to decide the best order to do this it's more to me a symptom of a problem thank you good question and it's monkey puzzle I can repeat it if smarty for the presentation so my question here is on the product discovery part so we hydrate things and we finally are about a product then we pass it on to the engineering people to scale it and building such a data object so my question here is so when do to decide how much data is too much data to stop while doing the titration so that I can next move to the scaling bar I think I understand are you referring to that graphic I had yeah so you're asking so let's be clear and analytics is a huge top okay it is a great topic - its but we collect most people think of the analytics we collect here these are production analytics this is what Optimizely runs off of stuff like that but we collect analytics here - in the differences this is only sometimes statistically significant this is usually statistically significant so most a lot of times it's qualitative here quantitative here so every time we forget the car thing let's get past the car thing let's just say we've got a hundred ideas we've got a big objective like reduce the or increase the conversion rate there's an objective or increase engagement you guys all from hopefully familiar this is what good product teams are actually doing rather than chasing features on a road map so these are our objectives and well we're we've got lots of ideas and IDs ideations great lots of ideas we don't stress over sizing those ideas because we've got techniques to let us try out these ideas really of many sizes very quickly and anyway we will evaluate those ideas sometimes quantitatively sometimes qualitatively data comes in if it's quantitative and then as soon as we have sufficient confidence that this is worth building though never proof but reasonable evidence that it's worth building then it goes on the backlog then the engineers do their thing and as soon as it's live we're collecting production analytics now we're getting 100% of the traffic and one that's why I Optimizely comes in here because once you get a hundred percent of the traffic we can run lots and lots of split tests and we can optimize the funnel how do we avoid bias in this it might be a team a same type of which may not even sync with the users designed by us but which kind of bias are you worried about it may be someone who is not so exciting might be the use of maybe oh I see I'm sorry yeah I don't usually actually try to rephrase the question was how do we make sure now it would just build in what we want to build really it might be the engineer or the designer or the product manager and so when we actually are in discovery now I want to be clear you don't have to do this on everything this would be on the risky things big risky things usually yeah you don't have to do it in this order either but normally we will test on our users and customers first the reason is is because if it doesn't work with them nothing else really matters and when we test with users and customers were really testing usability and then value kind of have to do it in that order I can explain why if you're interested but we're doing that we're trying to see could they use it would they use it or would they buy it if it's already a product they've bought it's about would they choose to use it they haven't bought it would they buy it so our most important biases we are that's why we do that because we pretty much know we're going to be wrong in fact the principle that stuck with me from Marc Andreessen which was the co-founder and Netscape I learned I learned a lot from this guy but what really stuck with me said the most important thing in product knowing what you can't know and it kind of makes you humble and it is really true and so we test user with users and customers like the buyer the users if it's b2b consumer just with me and then once we've done that let's say and we iterate ten times I actually want to step back one minute because before we even show it to users and customers I want to give a plug for why I love balsamic I that is why because we often will iterate many times just us you me the designer we will 20 iterations what's great about this balsamic we can do that in a half a day literally 20 iterations trying many different approaches because I say that because until we like it we we don't want to go waste the time and go try to see if customers will buy it and all that so we'll use balsamic there often again for a hard thing we'll get it to the point we feel good and then we'll go do a value test and usability test and again if this is a complicated thing that probably is going to be a high fidelity user prototype there are other cases too depending on what we're building but and then let's say we do ten iterations and we're good we've now made it so they have no trouble figuring out how to do what they need to do the tests and they convince us they would buy it either qualitatively or quantitatively we get convinced that's another discussion I could explain how we test that or you could buy the book and then once we're good then we want to double back with the engineers because they have hopefully seen these iterations if we've been doing our job and follow it along with us but we haven't really asked them to like essentially what you would call scoping we haven't asked them to really tell us if this is for sure feasible and so now we let the engineers play with that prototype you know go through each of them QA or test automation whoever we let them play with it because we are trying to make sure they feel like yes we can deliver it when we need it with the skill sets we have with the component tree we have and then once we're and we might need some iteration but once we're good there the product manager will typically take the high fidelity prototype and identify they'll look at all the different business stakeholders remember the CEO and the product and maybe it'll say for this thing we're working on it needs FDA approval so I'm gonna have to go to the compliance officer and I'm gonna have to go to the lawyer but I think everybody else is gonna be okay so I'm gonna go take the high fidelity prototype to them and show them what we want to do that's basically and when we get good answers there then it goes on the backlog that piece it might be a feature it might be a whatever that makes sense I know that sound I picked an example where like they were all risks in mostly in practice there's only a couple risks or one risks or sometimes no risk and you just say put it on the backlog alright hi uh yes um I've been involved in a some accelerators and there's a big debate in accelerators is uh well you know when is a product manager relevant in early-stage startup let's say there's ten hello hi it's there's ten people in the startup when is a product manager relevant or early-stage startup is it a CTO the product manager the CEO or a great question and so of course there's lots of opinions on this but for me I'm a strong believer that in a start-up one of the co-founders should be the product manager they don't have to have that they they wouldn't normally have the title product manager over there they have a title like CEO or they have a title like CTO or CPO but whatever their title is doesn't matter it's usually just co-founder that's the one that's focused on product and I like that to stay I actually like to stay quite a while a certain point that will become too much they will have some choices to make like if it's a CEO they're gonna have to decide are they gonna do you like Zuckerberg and stay with the product or are they gonna and hire like somebody else to help the other stuff or are they gonna hire a product person and then they'll focus on the fundraising and stuff and managing the board and let that product person do that so yeah yeah bring in I'll even go a little further if you've got an early-stage startup with a co-founder or coach let's say two co-founders typical and one of them is really it's their baby the product that product manager is probably not going to be well-received even if the prod the CEO thinks they should hire usually they think that because some board member tells them they should but that is not usually case because now it's like it's their baby and by the way they've just done like a hundred customer interactions they have all this knowledge and they see this person come in that has none of that and basically that person is forced to be asking them for every decision anyway they're just let's see I have a question for you I know you started off with the kind of the scenario saying that you know when you're working with your subject matter experts you give them all the all the kind of room to make that call right as you respect their their insight you know perhaps it's not always ideal like that and can you talk about maybe some successful techniques you experienced where it's about decision making is use of a hierarchical weighted voting type of technique is it you know dots is it you know someone's just gonna say follow my lead you know that sort of thing well let me let me think make sure I understand because I think in your scenario and let's just pretend that you are a product manager and you have a designer but it's a really weak designer two per week designer and and that's your real issue you are trying to figure out because you want to defer in principle to a designer but like that's crazy that's like an abdication of your responsibility because that person does not have the experience to make that call is that our scenario that could be a valid scenario yeah all right so assuming you are doing a customer facing product my view is get a designer not really get a good designer that this is way too important way too poor I should I keep thinking of balsamic one of the things that drives me nuts is product managers playing designer and doing all the by ceramic wireframes and handing it to their quote designer to me if the designer will take those wireframes happily I'm nervous okay how about adding on to that I know though onto that and saying okay well I'm the product owner and off be happy between the product owner and the you know the UX lead or what have you yeah well again if the product manager doesn't feel like they had that experience on the team first of all you could go to the manager of that designer and say we have a big decision here I'm nervous about this I would love you to weigh it very easy and so you would go to an expert on that or maybe it said an agency you have a relationship with god that's you might there's a technique that's not expensive called a heuristic review have one of their experienced designers take a look at it and basically say no I think this is fine or no we have to redesign this yeah and we pick designer could be anything it is very awkward though I mean this is a tough situation especially a common one is you have an engineer that becomes a product manager I was in this boat and now I you know you as a product manager but former engineer you might not think you might think you know a lot more about the engineering than the engineers do very quick way to get a unmotivated team so I try to coach the product managers to like bite your tongue they've got to learn they have to learn so I'm you know you you really got us be careful they're similar with the designers give your designer as much latitude one of my favorite designers told me the guidelines she does is if he can fit on a napkin it's fair game for you the product manager to describe a design as long as it fits on a napkin if it's like more than that no yeah you mentioned a framework earlier for trying to figure out whether or not you have too much tech debt you said if you if what's on the backlog is gonna take over a month then perhaps that's too much when it comes to experimentation versus meeting commitments that you have to your customers already you know we do a lot of experimentation I think we do a great job but we do have existing customers that pay us money to deliver our product you is there any is there a an analogous framework or evaluating when to make a decision to do more experimentation on things that are new ideas that you don't know versus things that the customer may be actually needing asking for that may be in line with the product strategy okay you are kind of combining a few important issues there one is just to be clear when you say experimentation I'm I'm thinking discovery this is what discoveries about we're running experiments we're doing a lot of stuff we're also going to customers we're done but we're validating prototyping worth we're thinking through the solution this goes on in parallel with this all the time it's not a phase neither of these is a phase just like your developers don't like develop and then you know go off for a month it's like always they just keep releasing keep developing and in discovery we're keep discovering we will just work on our objectives until we hit the key results sometimes if it's going well we'll go beyond the key results if an idea doesn't work we usually toss it aside and try another idea again that's why those roadmaps lock us in it's a great quote from Bezos again which is stubborn on your vision but flexible on your details and those roadmaps are detailed there you don't want to lock yourself in now you're talking so the short answer if we just talk about our normal work that is your work is to figure out the solution that needs to be built but I think you're also suggesting a scenario which what about working on some bigger goals in parallel you know in addition to the some people call it keep the lights on right just the stuff you got to do as part of our job and your you might be asking like where does that fit in and first of all I'm that I want you to do that I want you to think beyond and I want to make sure your focus on customers and not just you know doing low-risk optimizations discovery is where we would normally do that one way that you can get some big chunks of time is because a lot of times product and designer just really struggling to keep up and I always tell them what perfect opportunity to let the engineers focus on technical debt seriously in fact I always tell the engineers if that backlog starts running dry running out don't even ask just switch to tech death take the pressure off now because you you know the developers probably desperately need a month to go do that now in that month you can do a lot a discovery sprint a design sprint only takes a week and you could do huge things in a week which I'd love to see you pursue that so and you know I'm not you wouldn't do three weeks or four weeks of discovery sprints in a row that would be brutal but one in that month would be awesome yeah hi I'm Marty thank you again um have wondered ice more a philosophical question about the role of VM in a company where we're saying embrace failures at ethers a go big go for the moonshot and my PM mentality of you have to tackle the big risk fast and early where does that mentality of yeah embrace failure fit in our roles good actually my partner Chris Jones just wrote an article about this that I I really did enjoy that as I want to encouraged him to publish it was so he this topic of embrace failure fail fast he he was really referring to the situation because there's really two kinds of failure the good kind and the bad kind right failure and discovery is the good kind that's in fact one of my favorite discovery coach is guy named Tom Chi who ran Google X's discovery team and he he likes to say he doesn't even like to call it fail he calls it learn learn fast so the point is you don't succeed or fail with a test with a prototype you just learned and in fact he likes if it doesn't work well you usually learn a lot more so so that's what this is now I do want to be clear this was the point of his article if you go forward and use your engineers to build releasable product and it fales that's not good and that's what we have to be careful that is not what men buy fail fast that is don't don't fool yourself that is bad I'm not saying that people should be terminated with this because it it we try hard though to avoid that that is not what we embrace we do post mortems and figure out how could we have figured that out here one of the companies I really love is Etsy and they are quite good this is a discipline and they screwed up on something and they actually launched something that didn't work literally didn't achieve an objective and they were surprised because they thought this was clear you know sometimes you make a judgement I even I even encourage teams it's ok once in a while you're gonna be wrong you just think it's not risky but it really is anyway they launched something and it didn't work and they had a post mortem like that would have been a five minute test and discovery why didn't we do that well they even think it's rude because they didn't think it was a risk this was the infinite scroll stuff if you're interested but anyway that's yeah just be clear what's the good kind of fail and the bad kind of fail is we should even I don't think it's good that I used the term I've said fail fast a lot up here and I don't want people to think this is a ok fail this is what we try hard not to do because that really is a waste of our engineers and even the opportunity cost you're never gonna get that time back if you're a startup if you're a startup you usually got like 18 months 18 months of time and that goes super fast if you're a big company it's not usually that you run out of a you know time it's more like management loses patience in about six months and so the clock is going against that or lose confidence oh we do like two more questions and then we'll so thanks Marty for this awesome talk a question on who which are some of your favorite products and who are some of your favorite product managers I put it in my book highlighted sticks of my favorite product managers and six of my favorite products and I'd encourage you to read it but if you don't want to spend the money for the book look at the article behind every great product I'll let you I hope you do read it actually because there's some amazing stories in there about to give away okay last question hey when you were talking about the definition of the waterfall we kind of met that we kind of meet the definition of the two-week waterfall okay so how do we go about changing that and what is a good elaborative team look like well good that is what I'm advocating here so um yeah I don't want to sound flip on this because honestly the answer that question takes me a solid day because that's that is I what I just literally did that a couple days ago with a group of people I went through how we do that just a solid day that some because there are a lot of techniques want to set up teams and let's see the three big things though I think I had it let me somewhere else let me show you so yeah as long as you figure out a way to do these three and again you I don't want to sound like I'm Hawking the book but that is what the book talks about but it's though if you do those three things first of all you're not waterfall but more importantly I think you'll make way faster progress and have real innovation that's what's important and so first you got to tackle the risks upfront value usability feasibility viability don't write a line of code until you've tackled those risks second you have to you want to put that product manager designer and engineer together and even if you could tell I don't I really get frustrated with those teams that are given those roadmaps or because you know that's taken well that's the violation and number three is what it is so I don't like it but even if you were to say all right we got to integrate PayPal do that with the designer and the engineer side-by-side because there's still a hundred ways to integrate PayPal I mean not a hundred but there's like half a dozen ways to integrate PayPal I mean legitimate ways that they tell you you can do depending on what you so at least together and that's just define collaboratively that's the requirements the design and the technology and then finally don't celebrate just by releasing something only celebrate when it actually it's the result you're looking for those I mean first of all we don't really do release parties so much anymore because you release everyday what's the feel non-stop parties like yeah but we don't do that we celebrate not we use a release is kind of a non-event pushing something live happens all the time well we what matters for a good team is you accomplish some business result like Facebook fixes fake news we should all celebrate when that happens and maybe we should boycott them until they do I don't know I'm thinking about that so all right thank you very much [Applause] [Music] [Applause]

Channel: Dan Olsen

Views: 61,119

Rating: 4.9325604 out of 5

Keywords: marty cagan, inspired: how to create tech products customers love, build products customers love, product is hard, cagan, lean product meetup, inspired, product management, meetup, lean startup, svpg, intuit, mountain view, inspired v2, by, marty, at, lean, product, product school, product manager, what is product management, how to become a product manager, what is a product manager

Id: gCYFmrvPI8Q

Length: 106min 7sec (6367 seconds)

Published: Fri Feb 23 2018