Facebook Product Manager Execution Interview: YouTube Goals & Decline

how would you set the goals for youtube [Music] hey everyone i'm here today to do another product management mock interview with selena we're super excited to have you back on the show to do another interview selena but before we jump into it could you tell us a little bit more about yourself absolutely nice to meet you viewers my name is selena i'm a product manager at facebook and actually knows steven from back during our time together at stanford and i also previously did a mock interview with exponent about facebook movies which is really popular stephen mentioned so happy to be back yeah um your video just reached a hundred thousand views by the time people are watching this one it might be much more than that but we're super excited to have you back uh today and at the request a lot of people to talk about an execution type question these are more analytical metrics driven questions that are asked by top-tier tech companies um and so without further ado i'll jump into asking you the question and there'll be a couple follow-on questions that we'll do to investigate the issue further um and so first off we're gonna be talking about youtube today um which is the platform that we're gonna be showing this video on um and an awesome like big high growth channel right now especially during the chronovirus pandemic um so the question that i have for you selena is how would you set the goals for youtube yeah absolutely just give me one second to collect my thoughts take your time all right so i think the way that i'd like to go about this is first we'll talk a little bit about youtube youtube's mission and what their overarching goals would be and like a high-level metric bucket that we would want to set as success and then we'll dive into user actions that could best support that overall metric so we can hone in a little bit closer on what we'd actually like to track and then from there i'll propose some tactical metrics as well as a top line goal metric and then probably some counter metrics that we want to watch out for and then i'll close it up with an evaluation of any trade-offs sounds great thanks for laying it out awesome yeah so youtube's mission overall i would say is to first of all entertain its users you know billions of people all over the world love to go to youtube to watch videos every day but also to inform i think a lot of people also use youtube with that use case to educate themselves and learn something new so in terms of an overall metric i think we could either go for you know an engagement bucket metric or potentially retention or monetization um so youtube monetizes with ads and they actually also have a subscription product a couple subscription products um youtube premium youtube tv so i think we could consider a monetization as well but i really think that it's important to focus first on engagement and then monetization will follow so if you have users who are loving your product who are coming back to your product and spending time in your product that's how you're able to monetize and you know either insert ad breaks or upsell people to the subscription product so i think i would really like to focus on an overall goal of monetization we'll we'll drill i mean sorry of uh engagement and we'll drill into that a little bit further as we go in that sounds great um that makes sense before we jump into that perspective i'm curious selena if you would have a perspective on when we wouldn't prioritize it that way like would we um is there a case you can imagine where we might prioritize monetization over engagement um yeah i think you know it youtube is a mature product so we we could focus on monetization at this stage i think you just want to be careful to not focus only on monetization and forget why uh you're building the product in the first place which is to entertain people i think once you have a really stable sturdy base like definitely you can add teams that really focus on monetizing and really growing your your books but i think you know overall engagement is the way to win and you know there's competitors encroaching in the space like uh facebook uh videos for example on facebook watch so i think engagement is the way to go in this particular case totally cool yeah go ahead with the engagement and and we'd love to drill into that more okay awesome so i think next is um thinking about user actions that would help to lead to engagement so youtube is all about watching videos so i would say that's really the number one action that we want to drive is you know how many videos users are watching per day how long they're staying to watch videos um you could probably also look at the diversity of videos they're watching or people deep diving into one particular area or are they really you know exploring the full variety of content from different content producers as well so i think that's on the watch time side and then on the other engagement activities that you can do i know you can like you can comment you can subscribe so i think those are all really important actions that help fuel this ecosystem because you know youtube it's not just viewers there's also creators which is really important for the overall amount of content that's on the platform so i would want to make sure that there was um you know engagement in terms of users uh following them and commenting and liking so i think those are the two buckets that i'm going to start with for now so we have watch time which is you know number of videos watched overall you know length of [Music] session and we can also look at the average per user as well as the overall top line numbers there and then kind of other engagement actions which was liking commenting subscribing i'm trying to think are there other actions that i want to highlight in terms of engagement you know i think you could also consider on the creator side uploads might be a way to measure engagement i think i do want to focus on the viewer in this particular case so i'll probably focus more on there but i did want to highlight that since this is a two-sided market you'd want to think about the creator side as well but i think the overall you know north star is really about people watching on youtube totally okay so now that we have this like large list of metrics that we could look at i'm going to prioritize a couple so i'm going to pick some tactical metrics that we would want to you know it's more of a gut check to make sure we're on the right track and then a top line metric which is our overall north star this is the one we want to be driving to accomplish our overall mission of um entertaining and informing people um and then i'll also suggest uh tracking metric for um things to watch out for to make sure we're not you know running into too many trade-offs along the way okay sounds great so i'm just gonna take one second go for it okay so i have my three categories i have my tracking metrics i have my north star metric and i have my counter metrics so i'll start first with tracking so this is really what's telling me if this was early on in the launch of the product that things are going in the right direction these are early indicators leading indicators that things are going well so i would want to track average watch time per user and this is letting me know for a given user who's watching how long are they watching i'd also want to track the average number of videos watched per session this is giving me a good understanding of you know the number of videos that are being shown as well as potentially some distribution across creators and then the last one is a likes comments shares i'm sorry not shares um subscribes per videos viewed so i'd want to see that's almost like a like through rate um a kind of blended metric so i'd want to do a weight to figure out you know what what particular weight of these leads to users being more likely to come back and retain a week from now and to engage more with videos and i would want to see a healthy score there got it and on that last one do you have a hypothesis on what blend or how you'd prioritize or wait some of those likes comments and shares or subs i think yeah yeah i think here i would actually be inclined towards subscribers because i think that's what really powers the ecosystem that then would lead creators to want to upload more videos so i think that would be really important but also this is also a social product so i would want to make sure that there's still you know discussion going on in the threads as well got it um okay so then this brings me to my north star which is really watch time at the end of the day that's how you know you're gonna get your monetization down the line and how you're also going to prove that users are enjoying the content that's there um and this can vary with length of video so there's different lengths of content on youtube like this video will be a certain length so i think we also want to pay attention to you know how watch time is distributed as the length of content varies right because maybe you'd have someone watching a one hour video and bailing out and they only watched one video versus people watching a lot and there's a lot of different um content format links that are coming out so i think that could help to inform the strategy moving forward totally um that's something a great north star metric like um any concerns with it as we've come to this or kind of circling around this metric yeah so i think that's a great uh transition to my counter metrics and one of them would be you know uh wanting to make sure that people spend time it's like time well spent um i know that's the facebook metric but basically we don't want people spending so much time on youtube that they forget to live so i would want to make sure that people are you know healthy and enjoying the internet in a positive way um i think there's also a distribution concern right if people are just really rabbit holing on one particular length you know they're not expanding to the full ecosystem of of creators so i'd want to make sure that you know all the different creators had at least some share of that distribution um and then i i guess there's other counter metrics that aren't related to watch time do you do you want to talk more about these inverse of watch time metrics or am i am i good to cover the other counter metrics um you can cover the other kind of metrics okay yeah so the other two that i wanted to watch out for was churn so this is all about engagement um and retention's not quite the same but if i was noticing you know a drop-off in users who weren't coming back to me this would show me that the quality of the videos we were showing wasn't matching what they expected so i would be looking out for that and another one i would look out for is the amount of reported videos so maybe watch time is up that is really high but these videos are low integrity which is a brand risk so you want to make sure that you know the content is up to a certain standard especially there's you know a lot of kids that watch youtube so i'd want to make sure that you know our community was a positive and safe place for all of our users to be totally that makes a lot of sense um any other comments about the metrics and sort of the ones that you've the kind of metrics as well before i ask a hypothetical question about the metrics yeah i think you could maybe also consider the counter metric of creator distribution you know i kind of mentioned that as something you'd want to track along watch time but i could see that also being a counter metric to look out for if you just saw a really high concentration on specific videos um i know this is also something that youtube changed the way that videos could go viral to help ensure kind of more distribution for all different types of videos so you wouldn't want too much watch time concentrating on any one particular producer or video type so you might want to track that not just as a north star but as a counter metric as well totally um okay awesome well thanks for the thoughtful discussion around watch time i want to transition this interview into a different sort of question where we're going to imagine that that key north star metric that you picked of watch time is unfortunately suffering a three to four percent decline over the past couple of weeks and so or you know weeks two months maybe uh let's approximate but um the time range isn't so important as insofar as what i want you to do is to imagine that you're a pm at this team um and you know you've done this thoughtful consideration around watch time but how would you investigate what's going on here or what's causing this decline in that north star metric yeah absolutely so i think the most important thing whenever this happens in an interview situation or in real life on the job as a product manager you want to start by asking questions you never want to immediately just run off and start diving into the data by yourself like make sure that you're asking some key questions about this new trend so i think the first thing i would want to ask is how long has this been happening did this happen yesterday has this been going on for a week maybe a month yeah let's say that it's been happening for weeks to a month so maybe like four weeks um to a month or two ago like we've seen this decline so um within the range of like two to three months or one to three months or so got it so sounds like it's been going on for a while does that mean it's been a gradual change or was it acute like three months ago and we're just now getting around to dealing with it yeah we're seeing the gradual decline to the state where it is today over the past few months got it okay so i think my next question would be are we seeing this decline across the board in terms of geography or is there a particular geography where we're noticing a decline we're seeing the decline actually mostly in the domestic market so united states um we're not seeing the decline as much in emerging markets or foreign markets got it that's good to know okay let me think a little bit you know normally i would ask is it just this product or a broader product but since it's youtube that's kind of the whole thing i guess i could ask um there is youtube tv and youtube premium is this focused on watch time of a particular segment um that's a great question it's not focused on a particular segment so we're seeing this decline kind of across the board on youtube over the past couple of months okay all right this is going to be my last question before i'll put into buckets my kind of issue triage identification which is is there a particular device type that this is focused on great question um there isn't a device type associated with this decline so the devices seem to be uniform across iphone android as well as the different device types within those categories so like the different iphones or the different uh you know samsung or um like uh pixel phones got it okay so at this point i'm gonna pause my general questions and then i'll focus in three different buckets of where i think the issue could be arising and i like to think about this as internal to my team um so i should probably ask you what team i'm on um an internal to the company and then external are the three general buckets i look into so i i know i'm about youtube is there any particular team that i'm on um let's say that you're on the um general youtube team you're not on premium or youtube tv you're on the youtube product that people generally see when they come to youtube.com as a non-paying user got it okay so given what i'm hearing that this happened gradually from weeks to months focused in the u.s and it's happening across the board this would lead me to think it was either due to something that happened externally potentially with a competitor or that something that happened internally that has happened over time um that's either due to an experiment or a de-prioritization but since it's gradual this is telling me that it's a user behavior shift normally more than it would be um an infrastructure issue or a system that went down which would typically demonstrate a more um acute sudden decline so uh i'll run through a couple questions i think first i'll ask the the burning question which is you know uh did our competitors launch something within that three month period that's drawing um share away from our products because that's kind of what i'm feeling is maybe most at play yeah so um definitely we've seen a lot of competitive products in this space in the past couple of months um and so there is tough competition and we do know that from just like a marketing or partnerships perspective but it's a little unclear to us right now if that's the reason that we're experiencing the decline but that definitely could be something we want to investigate further okay got it um all right let me think um maybe i'm also thinking externally i know we work with uh partners and third parties so is there anything that happened potentially with a partner or someone that we work with that would have gradually caused this decline maybe they were migrating off of um like i know we offer youtube on tvs maybe some older tvs no longer support our youtube app or or something like that that could be causing the issue yeah that's a great question and you know a lot of for example um android devices might partner with certain oems to make sure certain apps appear there and there could be a case where the youtube app did not appear on a device partner but in this case um we're not seeing any changes in partnerships in fact the partnership season let's just say is in two months so the renewals of all those contracts would have happened then and it wouldn't have happened within the past few months got it um let's see so i guess this is my last external question before i go into internal to the company problems was there any bad press that happened about our company um maybe a couple months ago that would cause users to slowly but surely abandoned it's a great question again i mean uh cedric himself has been testifying in front of the u.s government um so there could be something there but for now probably for purposes we're not seeing something so damaging or so shocking that would have caused a huge change in the measures got it okay so now i'll ask some questions that probably are arising from uh internal to the company issues um but i'm still holding on to that it could have been something with the competition in the back cool so my second leading theory uh second from competition would actually be cannibalization so did we launch something new that's separate from watch time um that would maybe be drawing users slowly away um or maybe something we're experimenting with that's being rolled out gradually that could be impeding watch time yes so we've actually launched um a feature on youtube where you can see it's kind of like instagram reels or tik tok um where we've created a lot of like short clips of videos that users can browse through really quickly youtube shorts yes um yeah well i would say that that would be likely since it's shorter length content to lead to a decrease in watch time three to four percent seems pretty consistent where i wouldn't expect it to be an enormous sea change but as users are adopting this product more and more um if that length of videos is shorter than the typical youtube video length then you would see a corresponding drop in watch time um like that's not necessarily a bad thing you know um given all of the engagement especially among younger audiences with tick tock this could be a strategic choice um and it is a strategic choice to launch this product so i wouldn't say that it's necessarily a cause for alarm in the near term but we should definitely be thinking about you know how much watch time are we willing to give up um to support this new product launch and you know maybe maybe watch time isn't the right north star maybe it's you know how much enjoyment users are getting out of the videos and since there's a general trend towards shorter videos maybe you know a total max watch time isn't necessarily the best demonstrator of people enjoying this new product totally um and yeah like you know in this particular case that we're talking about today um we can imagine that youtube shorts was just launched um and so in that case the algorithm wasn't quite right maybe let's say to ensure long-term engagement or longer term watch time and so that might have cannibalized some of the youtube traffic that we normally expect with users watching short form content that is not as effective at finding the algorithm to get users to continue to watch the experience which could definitely happen when a company launches a new product but hopefully in the future cases youtube wouldn't see that decline as much yeah absolutely i think since algorithms learn from user behavior it's almost expected in the near term if you're moving away from a longer content form to a shorter content form that it's not going to immediately get it right totally um so congrats to getting the right answer i know it's stressful and i thought you did a great job kind of discussing a lot of the different criteria i'm curious also for you if you were going to just dig any further is there anything else you would have explored had you had the opportunity to to explore some of the other avenues or answers yeah so um i think a similar side of the coin i would ask you know are we de-prioritizing our regular product in favor of another one which is kind of the exact same question but focusing instead on on the relative prioritization of the two products which i think is absolutely true um sometimes i look at internal to the team which i think with a product like youtube maybe is a little less valid but sometimes you can think okay did my particular team launch an experiment that's impacting another team or vice versa um i think here it's easier to think about it as a total platform effect because of the way youtube is as a product but that's always an area i like to explore as well totally um well selena you can take off the interview e-hat you did an awesome job i want a debrief yeah i want a deep brief um like how this was for you and um i wrote down some thoughts on what i thought you did so awesome with this interview and i thought it was a really rich conversation that's really educative also so i'm really grateful you were able to do this but before i jump into my thoughts did you have any self-reflection or self-feedback on how that went for you yeah i think overall it went really well i mean full disclosure i did intern at youtube uh about a year and a half ago now feels like forever so i did have a little bit of familiarity with with the company um and yeah i think the the one area that i felt like i could have done a little bit better was defending why not to go for monetization um you know i think it's always a little bit of a snake chasing its own tail between engagement and monetization but you can't run a company without having money at the end of the day so i think that's something that's important to understand when is the right time to focus on that and i think that's something a lot of companies struggle with um but otherwise i thought that that part was good i think maybe you were looking for a different trade-off that i didn't mention during the the metric identification so i guess i'll i'll listen to you on that front um and then uh in the second half around the issue identification i think there's two kind of flavors for this sometimes interviewers are looking for you to identify um the case and sometimes they just want to hear how you would talk through that so i've experienced both in my my interviewing experience and i think you know either way you want to just be reasoned with your thinking and if you don't always get to the right answer even if that's what they were looking for and you talk through it in a structured way i think you can still do well totally that's awesome advice um yeah let me run through i mean i agree with what you said i some things that i thought you did really really effectively celina were one you you did such a great job checking in with me i felt like i was always on the same page with you throughout that interview um and you know you even did stuff where you would say like you know i'm gonna move on to this section and you really communicated with me frequently throughout the interview especially with execution questions it gets really easy to kind of slip into this pattern where you're gonna say hey like is it this is it that is it this and then um i as an interviewer don't really have a sense of what you're thinking or how you're thinking about that problem and so that's not an effective tool you know for the interviewer to assess your capabilities but um one line in particular that stood out to me was you know you said i'm going to keep competition in the back of my head or over here and you know i'm going to move on to this section and i thought that was really effective because you you sort of acknowledged the interviewer you know i haven't fully figured that out and i know it's still an open issue but i'm going to investigate these other things and i thought that communication style was really really valuable i also thought you did a great job of breaking down the problem so you'd often enumerate or um split up the problem space into different components for example internal external to the company external or sorry internal to the team internal to the company or external that was a really awesome way to break down the problem and to know where your head was throughout each of those steps so i thought that was really effective as well you did that a couple times um i think you know i may have jumped uh ahead of you on this one but mentioning countermetrics beforehand or you know making sure you slip those in is really important um i actually didn't have a specific counter metric that i was looking for so it's funny that you thought that maybe i was just putting on my tough interview or face or something but um it was more more honestly like uh the counter-metric discussion that i think you did was a really thoughtful one and um if i had more time i would have asked you more about you know like time well spent what does that look like in that experience um i do agree the monetization angle we could have had maybe a richer conversation around that but i didn't think that that was so um important to the essential interview question that i was asking it was kind of a side tangent that i took you along um so honestly overall and the last thing i wanted to mention is that you really did a great job of thinking about the goals of the company first these questions often seem tricky but they um really make a lot of sense when you think about the goals and so i love that you thought about the goals first and then kind of boiled it down so i don't have too much constructive feedback i thought this was like a really awesome answer it was a really fun conversation also um and then yeah maybe maybe one last point that i would say is just the execution part at the end um you know being specific about how you'd identify those issues so you know you asked if they're competitors um but there might be more specific ways to identify competitive competitive uh companies in the space um so perhaps you could have asked about like um you know are there other analogous products within google's ecosystem or partner network that have seen a decline or maybe are users of youtube's api seeing a decline or you know embedded videos on youtube seeing a client that might start to hint if it's a competitive issue or not so that was just one constructive piece of feedback but overall really awesome interview do you have any thoughts reactions comments or last tips for the viewers no i think that all sounds great and i think that's a really awesome suggestion to you know think about other other ways to plug into is this an internal or external issue outside of the the core product so i think apis was a great suggestion if that's the type of product you're focusing around to get at that same question from a different angle um thanks again selena for being on the show thanks it was really great to see you and for those of you watching good luck on your upcoming interview thanks so much for watching don't forget to hit the like and subscribe buttons below to let us know that this video is valuable for you and of course check out hundreds more videos just like this at tryexponent.com thanks for watching and good luck on your upcoming interview

Channel: Exponent

Views: 129,258

Keywords:

Id: 3Qx9cVRJ06I

Length: 28min 32sec (1712 seconds)

Published: Mon Feb 08 2021