Product Interviews: Execution & Metrics

hi guys i'm diana and i'm a product manager at a big tech company in silicon valley california and i bring you the best tips to help you get into product management and teach you how to succeed once you've made it if you're interviewing for a product manager role soon you're probably going to get a debugging question what's a debugging question you ask here are some example questions from actual interviews you're the pm for our dating product and users have dropped by 10 what would you do how would you diagnose a 15 decline in user engagement or events usage has dropped by 20 overnight what would you do as the product manager these are actual situation that us as product managers are put into on a weekly basis hence why interviewers like to ask this question the goal as i've said in past interviews is the interviewer wants to understand how you think it's not to get the exact answer but to understand a how you think logically and structurally b that you know how to use data and see that you can come up with hypotheses that are data driven so today i'm going to show you what 80 percent of candidates answer when given this question and then i'm going to show you what 20 of the best candidates say when given this type of question so let's get started this is what 80 of candidates say when given this type of question so say you're the pm for the events product we notice our metrics drop by 10 how would you figure out what happened all right i have some clarifying questions is this in a specific country or is this global and was there a major holiday that happened this was global and no major holiday as i know okay no major holidays okay this is how i want to approach it i want to think through internal and external factors that might be causing this 10 drop so let's start with external factors has something happened with competitors did they launch something and what about pr have we seen the news talking about our product negatively no we haven't had any negative pr scandals in the news and with competition i'm not sure but that's for you to figure out okay i see so maybe not external factors but let's look at internal factors have we shipped any products recently that might have impacted this maybe we should check if there's a bug in the code and that could be causing the decline in numbers we launch features every week so yes where would you go now you might have noticed that felt like a weak response not because it was wrong it felt like the interviewee was guessing off of a checklist and we're not looking to hire product managers that are just going through a checklist we actually want them to think now that we've gotten a weak answer out of the way let's talk about what a great answer looks like so as always you know me i'm gonna start with a framework here's a framework on how to tackle this question in a logical structured and data-driven approach and this is actually how i think as a product manager in my day-to-day work so let's start with the structure number one you want to understand the product and this metric in relation to this product number two you want to break down this metric to understand what's affecting it and what it may affect three this is where you want to start asking for data as you would if you were a product manager working with a data scientist or a data analyst number four based off of the data you want to brainstorm some data-driven hypotheses that could be causing this decline and number five you want to figure out based off of these hypotheses how would you actually go about validating or invalidating these hypotheses so now let's dive into an actual question so i can show you how this framework can be used the question we're going to go through today is how would you figure out what caused the 15 drop in user engagement on instagram so before you dive into any question always feel free to ask clarifying questions in this specific one i would ask the interviewer was this 15 drop a drastic drop or one that happened over time let's say the interviewer told us this happened drastically over the last two weeks now let's use the framework we just talked about to get towards some hypotheses so the first part of our framework is understanding the product and this is key because you want to understand how this product is bringing value and where this metric falls and the big picture of the product in this case instagram is a social media product that allows your friends content creators influencers to share photos videos products so that other users can engage with it people engage with this type of content by liking it commenting resharing buying this part is important because you want to make sure that your understanding of the product is aligned with how the interviewer understands it so that you're not making any assumptions that will get you stuck later down the road second now you want to understand the metric in this question the interviewer told you that engagement went down by 15 but what does engagement mean that's not a metric so in this case let's break down what engagement could mean when we talk about instagram how do people engage users can be broken up into two kinds the content creators slash producers and the viewers of course one person can be doing both but let's simplistically break it down like that a simplistic representation of the content creator's side is they post and they reshare now what type of content can they post they can post a normal photo post a video nowadays are real which is like short form tick tock type of content that launched this year now let's move over to the viewer side how can viewers engage with a post from a producer they can like it denoted by the heart comment re-share it with friends they can buy something if it's a product post or they can send a dm a direct message to the poster so those are just some high-level ways how we define engagement on the instagram platform and this takes us to the third part asking for data to investigate so we just talked about a couple ways you can engage on the platform and now my first question is number one is this drop-in engagement coming from a specific type of engager was it the content creator where we're seeing a drop in the engagement or was it the user's side or was it the viewer's side who are not at engaging with the post as often let's assume our interviewer told us when we talk about engagement we're talking about engagement with a post so in this case i'm then asking are there specific types of engagements that are dropping for example maybe we're seeing a reduction in likes or maybe a reduction in comments which of the five that we named is actually seeing the reduction or maybe it's all of them but this is the type of data you want to understand to help you then narrow down what might be causing this problem and yes here is also where you want to understand segmentation of let's say countries is this happening in just one market or is this happening across all markets so this is a stage where you'd be digging for data and asking the right questions to help you propose some hypotheses of what could be the problem let's say the interviewer gave us some information that the decrease in engagement of 15 percent were being contributed by a reduction in the total number of posts which went down by 15 so that piece of information i'm asking myself was there a drop in the total number of content creators maybe leading to this or maybe there was a drop in the production by the top content creators contribute to it enormous share of the total content produced perhaps there was friction in the content creation flow whether in the entry point was misplaced or there was a bug for example a couple months ago when reels came out instagram decided to replace the create post button with the reels button which probably led to regular photo posts creation for being severely reduced the interviewer didn't tell us whether this was country specific but if it was we can imagine depending on the country the issue could range from an internet shutdown from particular countries to maybe a national event these are just some hypotheses and what's most important is that your hypotheses are data driven and not randomly selected that's the difference between eighty percent of candidates who will randomly guess versus the twenty percent will ask for data up front and then present some logical hypotheses to follow the data step number five you want to figure out how you would validate or invalidate your hypotheses so what would you tactically do in this case i might check the trend of content creators over the last few months and see when the drop happened exactly i might also test if there's been any changes in the creation flow by playing around and dog fooding it myself and to rule out any country-specific changes i'd work with our country managers to check if there's been some large national event that's happened across certain regions or countries i hope you were able to see the difference between the 80 of candidates versus the best 20 and how they answer these type of questions what next go try out some debugging questions using this framework and see if it actually helps you think more logically and structurally and try it with people that you're mock interviewing with if you've got follow-up questions make sure to comment below and also like and subscribe to let me know you want more videos like this thanks guys you

Channel: Dianna Yau

Views: 4,772

Keywords: startups, technology, emerging markets

Id: lgtcCBGZgcE

Length: 13min 9sec (789 seconds)

Published: Mon May 03 2021