Facebook Product Manager Interview - Flawless Execution Interview Response by FB PM: Picking Metrics

hi welcome to product alliance today we're going to show you a few secret interview techniques that you'll need to crack facebook's infamous execution questions to do this we're interviewing a facebook pm and then we're analyzing his answer to draw out the tactics that'll help you get the facebook offer pay attention to how this pm draws facebook's strategy and mission into a metrics question then finds metrics more nuanced than monthly active users and brings a structured approach to the metrics drop question without further ado i'll hand it over to our host audi to kick off the interview take it away hey steven thanks so much for joining me here today how's your day been hey adi things are going well how are you doing thanks for having me of course yeah i'm doing well too so today we're going to do a execution question one of the common questions you see in execution is defining success metrics but before we dive into that how about you tell everyone a little bit about yourself yeah of course so uh my name is steven i've been a product manager for a couple of years now i've worked at places such as facebook instagram and shopify and currently i'm a product manager at a startup called grail which is a peer-to-peer marketplace for style enthusiasts to buy and sell luxury fashion items that's awesome sounds like you have experience at small startups that are hot and upcoming and also large tech companies so it'll be awesome to do this mock with you so uh for the question today uh i was thinking that we could talk about how you would measure the success of facebook dating so don't mind me i'm gonna be taking a few notes throughout the call so i can give feedback afterwards but uh let's get started okay great that sounds good um i'm just gonna take a moment and gather my thoughts and think about how i want to approach this okay so i just want to start out with some clarifying questions uh first i just want to make sure that we're on the same page about facebook dating as a product uh so my understanding of facebook dating is that it's something that's very new that facebook is currently testing in a couple of markets and it's basically going to be like you know a competitor to some of the dating apps out there today like hinge maybe not so much like tinder is that right yes i think you have a good understanding of the positioning uh and i agree with that yep okay great um one thing i want to ask those can you tell me more about how we're viewing the current state of the product so i can get a better understanding of you know is this uh something that we're looking for understanding the product market fit from a success metric standpoint or are we looking to grow it uh you know and increase the engagement yeah that's a really good clarifying question so let's assume that facebook dating is in the same stage that it's in right now we've done uh small-scale beta testing before with it and now we've launched widely across the united states and we've been seeing users engage with it and adopting the feature and so we're looking to take it to the next stage okay great i'm just going to take a moment and jot down some notes but at a high level i'm going to start out by thinking about what are some of the goals that facebook dating might be trying to solve and then go into some of the potential success metrics of how we might actually go about tracking those goals that's a good framework i like that okay so at a high level facebook mission is to uh you know give people the power to build community and connect people all over the world and where facebook dating can come in and play towards this mission is being able to really help users build relationships at a much deeper level and uh so i think some of the problems that i might solve are things like adult loneliness for example uh or you know just helping people build like lasting relationships i think you know facebook probably has heard a lot of examples where users found their lifelong partner on facebook and you know this is one of the most meaningful types of uh connection and human connection that it could be driving so i can see that's how it really aligns with the top one company strategy and so i think for us you know in terms of dating team there's two paths that they probably started on which is maybe facebook dating could be helping people go on dates but i would hypothesize that the primary goal at a high level is figuring out how can we as facebook help people build relationships uh especially like long-term and meaningful ones does that make sense yeah i follow that great uh and in addition to that i think there are some other strategic goals that i just want to call out some of them might be related to growth and you know helping increase the number of users who might find value from facebook uh monetization could be one although i think that for this specific product it might not necessarily be a direct monetization play whereas someone like tinder might view this as a monetization opportunity but i think a bigger one is actually engagement and retention and giving people more value and something that they can use facebook for and increase the stickiness of facebook steven sets the stage really effectively here before diving into the nitty gritty of the question he takes a minute to explain why facebook dating exists in the first place by connecting dating to facebook's broader mission and showing how it adds value to users lives then he emphasizes facebook's strategic goals by showing how dating could help with each by doing these things steven shows his cultural fit to facebook along with the strategic chops he's not limiting himself to showing that he's good at execution or metrics he's taking every opportunity to showcase all of his strengths no matter what the interview is nominally about i'm just going to take a moment and sort of like jot down what some of the key success metrics are that tie back to our top line goal of helping people build relationships awesome okay so i've just jotted down a couple of ideas of potential metrics one thing i'll call out right off the bat is that uh you know before i go into all these metrics it's really hard to actually measure the uh you know a successful relationship being created all the time because you never really know uh until like much later down the line if someone's gonna end up getting married for your product so i think for here what i'm really trying to figure out is what are some of the sort of like leading indicators or metrics that we can actually measure with product that tie into our top line goal um to actually get you know as close a tie to the outcome that we're trying to drive as possible so um you know just going through my list that i brainstormed the first one is related to just active users and specifically uh active users of the dating products so i would maybe say like weekly active facebook dating users or monthly active facebook dating users and this just gives us a general understanding of like how many people find value from this product and it ties back really well into sort of like the current state of where we are we're no longer in this beta stage we've launched this widely and so we want to know how many people are we actually able to impact on this and this is also really important because you know it's a metric that impacts the dating pool that we actually have available now what's interesting about this metric though in terms of some of the potential trade-offs is that it might not necessarily capture the outcome that we're wanting to drive technically speaking uh you know one way to game this is that we might have a lot of active dating users but we're not actually helping anyone build a relationship if they're all active because it means that you know if they're continuously active then they obviously haven't found success from the product so um those are some tradeoffs you might want to think about with that one steven definitely avoids the common trap of assuming that mao monthly active users is the be all and all of metrics mao is usually a good starting point but it can be easily gamed and doesn't necessarily correlate to user satisfaction note that steven doesn't dismiss mao outright he acknowledges that is valuable but calls out some very particular flaws that it has in the context of dating and it's a great idea to mention mal first like steven did and then dive into three to five alternatives um the second one that i have is you know just straight up uh matches so the facebook dating product works similarly to a lot of competitors out there where you put your profile up and you can talk to people and get matches and so i think that uh tracking how many people actually end up matching with each other uh is you know potentially a great leading indicator of how well we're doing there the second one i actually have or the third one that i have is uh something around conversations and this can actually be broken down into two ways you know some people might say off the top like conversations in terms of uh in terms of like messages sent but i should think a better way of measuring conversations is actually uh like a back and forth conversation between two people and so what i mean by this is if we just measure conversations by someone sending a message it's not really an indicator of building towards a relationship if the person on the receiving end never replied and i think this is something that uh we want to avoid especially in the dating world where you know we could be driving a lot of matches and we could be driving a lot of messages sent but if they're coming from accounts that don't want to talk to each other then we're not really achieving our outcome and so i think a great way of breaking this conversations metric down could be something like defining a conversation as a message sent and a message uh or a message sent from one user as well as a message sent back from another user or maybe even something called like a two by two conversation where um someone sends a message to someone else they send it back and then you know there's another exchange of it back and forth uh some additional ones that i think could be good might be something like a status change so facebook uh i think a big opportunity is the fact that it has the uh data to know like you know what's your marital or like your relationship status so seeing if we can actually impact that as well as maybe if people end up friending each other and take this off the platform steven's ordering of his five metrics was really clever here he arranged them into a sort of funnel first people match then they start a conversation then they friend each other or change their relationship status so first they hit their number two metric then number three then number four and five metrics focused companies like facebook love the concept of a funnel both in marketing and in metrics so notice how steven savvily brought in the funnel without being asked really well done here so i just went through uh you know a good list of metrics that i think could all be potentially pretty good for this i'm just going to take a moment and think about which one i want to prioritize sounds good i think you listed a lot of good metrics and i liked how you showed me the trade-off between the two so yeah go ahead and do that okay great so i think if i were to prioritize out of all of the metrics here um you know understanding that our goal is to really build relationships as well as this product has launched uh worldwide but really hasn't gotten to a big skill yet i would probably focus us on the uh active dating users maybe as like a good top line metric of just like measuring the health but leading indicators of like north star metric would probably be the conversations with at least two messages sent per user and the reason why i picked this one is because i think that this is a very actionable metric for us to be able to like make product changes and understand if it's been impacting the impacting the sort of uh like behavior of our users as well as uh you know the outcomes here should end up driving towards some of the like top lane goals so if if we're getting more conversations between two users uh this should actually lead to a growth in the number of like active um facebook dating users without necessarily being like you know very gamey towards like how we actually uh acquire those active users because uh you know like i mentioned before uh getting a lot more active dating users doesn't necessarily tie back into our ultimate outcome which is helping people build stronger relationships and i think that the first step that a dating product can really uh make towards people building stronger relationships is facilitating those conversations stephen brings an excellent insight here it's usually difficult to directly measure your product's impact in one big broad north star metric and if you made a product change and facebook's total active user account went up you have no way of knowing if that increase was caused by your feature or by any of the thousands of other product changes that were being made at the same time instead you want to create a proxy metric that is correlated with your north star metric like steven's choice of number of substance conversations in dating it's easier to measure your impact on these simpler metrics and you can still demonstrate that you've moved all the top line metrics audi never asked about proxy metrics but steven went above and beyond by calling him out on his own makes sense so let's dive into that i like how you like acknowledge that trade-off and that's a very common trade-off that a lot of uh dating apps see let's say like one day the metrics for uh monthly actives or weekly actives drops off ten percent uh would you what would you do would you immediately how would you figure out what's actually going on uh in a deeper level like you mentioned a good leading indicator that you would keep track of which is uh the two by two conversations making sure that people are messaging back and forth but are there any other indicators like that that measure behavior that you could look into that would give you additional insights yeah definitely let me take a moment and sort of gather my thoughts uh actually before i do that one quick clarifying question are we talking about um uh a drop in a specific metric like the one that i just mentioned or uh is it like another one yeah let's say the one you just mentioned uh in weekly active users uh that drops ten percent and what is the time frame that this drop happened over is it something that you know we just came in and noticed today that it dropped by 10 or yeah uh we hadn't checked this since last week and when we ran this week's numbers uh we realized that it dropped um i'm just going to take a moment now to put together my framework okay so um there's a couple of steps that i would take here but at a very high level there's like two ways of breaking this down one is i would first think about ways of breaking down the metric itself to try to identify um potential hints towards like where in the product uh this total drop is actually coming from so what this means is thinking about you know uh a percentage drop or 10 drop in weekly active users did this come specifically from a certain area of the world so segmenting by geography did this come from a specific type of user maybe we see that there's a 10 drop from users who uh normally come in via notifications you know and so that can help us understand okay maybe there's a problem with notifications and it wasn't something across the board or is there something we can break down the metric by for example via platform uh maybe this was a drop that we saw on android users or maybe like mobile web users and so we can try to see if there's like a bug in any of those platforms so that's sort of like one area that i would just always look into just to really understand where the problem is coming from now on the flip side let's say that we have a little bit of understanding there some of the steps that i would take to really build hypotheses around why these metrics might drop is first just looking into things like seasonality so you know oftentimes uh we'll see these big drops let's say during holidays for example like maybe it was just christmas season last week and no one was using their phones and trying to date and so this makes a lot of sense and it's not something we need to worry as much about usually that's not the case or if it is it's something that you'll know and it's very obvious most candidates would automatically assume that this metric drop was caused by a product change but stephen avoids jumping into conclusions and correctly calls out that the drop could have been due to external factors like his example of people not dating during christmas week this shows precise open-minded thinking on his part but more importantly it shows that he takes time to analyze the question before rushing into an answer the second big step is really trying to then pair up some of that data with things that we can match to like internally so looking at like was there a product change by our team was there a product changed by another team that may be affecting this or maybe there was a bug right going back into some of the data we can see okay there could have been a big drop in weekly active users coming from android and this was caused by an internal change we made or this was caused by a like facebook wide bug on android and then you know if you don't find anything there the third step that i would really do is then look into uh if there's some like external factors uh that could have explained this uh maybe this was something that you know is just like happening across the world so we look at the competitors and like try to understand is our competitors experiencing this problem as well or uh maybe this is something due to a launch made by a competitor so maybe like hinge or tinder just came out with a brand new feature that like drastically changed how people view facebook dating and so that's something that we end up needing to react to um so yeah that's sort of the framework that i would take to go about debugging this awesome i think that's really thorough and like regardless of where what area you find that issue in you would totally be able to identify that root cause to take immediate action to figuring out whether it's something you need to resolve or something that's temporary and will change later so i like that you went into all that detail awesome i think you did a great job of breaking this down in the interest of time i just wanted to see before moving on uh before i give my feedback if you uh have any thoughts on uh because self feedback is important so if you have any uh thoughts on like maybe things that you could have improved and then i'll share my thoughts as well yeah i think some areas where i could have maybe improved a little bit is sort of uh along the lines of like really getting into the like tactics of or it's not tactics or maybe asking some more questions around like the metrics breakdown and sort of like asking more i think i just kind of went into a high level framework there but i probably could have asked more questions and try to see if like you as the interviewer had some answers to some of the hypotheses i had or maybe actually added some of my own hypotheses to try to just like make that even stronger and really like take this to the next level in terms of something that's like very concrete something else that i could have maybe improved on i think i spent a little long talking about all of the different metrics where maybe some of them weren't too necessary and like went pretty in-depth i think i could have been a little bit more concise and talking about all of them really quickly and then prioritizing just a couple of them to go deeper into and why i might have picked those overall this was a 10 out of 10 answer from steven showing that he really earned his pm job at facebook he showed a strong understanding of why facebook built the dating app in the first place and how dating could contribute to facebook's metrics and mission which means he got some points for product sense in an execution interview that's the mark of a truly exceptional facebook interview the meat of the interview was excellent he methodically brainstormed a long list of metrics by thinking about the funnel assigning one or two metrics to each subsequent stage this way he ensured he didn't miss any important sub-metrics and yet he was still able to prioritize and ultimately pick one top-line metric to go for which is something that facebook really values the only thing i would have done differently is being a bit more explicit about the funnel perhaps by sketching out the funnel on a whiteboard and drawing lines from each metric to a stage in the funnel it isn't strictly necessary but it would have made a great visual aid all in all a phenomenal answer from stephen makes sense and i like how you started off like building on what you're saying with like the mission tying this back to why people are using the product and when you define those metrics it gave me a good understanding that you understand the product what we're building the goals and you had a lot of different ideas so i thought it was good that you did that and that you explicitly chose one to prioritize one thing that would have been helpful that i caught was if we when defining those metrics had kept track of a funnel so you had a lot of good uh metrics on how to measure like deeper level leading indicators like the two by two conversations but seeing at a high level that maybe at the top you're measuring weekly actives but then afterwards you're measuring the number of matches and then from those you're measuring the two by two and then from that you're measuring in person meetups uh from which point a relationship status change if you were tracking that in a funnel and you had some structure there that would have helped guide the conversation but overall i think uh your approach to debugging like what would go wrong if you had to drop off in in usage uh you gave a lot of ideas and and just want to say thank you for for sharing all of that yeah of course thanks adi for that eye-opening interview the facebook pm you saw steven gave a really impressive answer he used a couple of step-by-step strategies that the product alliance developed to ace metric sections and metric drop questions in the full flagship facebook pm interview course you'll learn these two strategies plus almost a dozen more to help you ace every type of execution product sense and leadership and drive question that facebook throws at you you'll also get over 40 mock interviews just like this one including expert commentary and whiteboard graphics that you just saw you'll get the inside scoop on videos that tear down facebook's 10-year roadmap and the strategy for whatsapp reels watch dating instagram messenger and more we'll even throw a list of interview questions that our team members and past customers have actually gotten during their recent facebook pm interviews we update our list monthly so you can study with the questions that you might actually get asked word for word the flagship facebook course was built with input from over a dozen current and former facebook pms including hiring managers and product directors as a result we're confident that the course will put you a step ahead of hundreds of other candidates looking for the same facebook pm role and puts you in a great position to land your dream job at facebook thanks for watching and we'll see you next time remember that this lesson was just the tip of the iceberg so make sure you click below to check out our full flagship facebook pm course

Channel: Product Alliance

Views: 24,340

Keywords: facebook pm interview, facebook execution interview, facebook product manager interview, facebook mock interview, fb pm interview, product management, product manager, product manager interview, pm interview, product management interview

Id: dt7OrazE6SQ

Length: 22min 24sec (1344 seconds)

Published: Tue May 11 2021