LinkedIn Metric Interview Question and Answer | Data Science Interview | Decode and Conquer

hi it's emma welcome back to my channel since i posted a real magic interview question from lyft people told me that it's more helpful than simply sharing frameworks i hear you so in today's video let's go over another real metric interview question the question is from this book decode and conquer it's a book about product management interviews and i have found it also very helpful for data science interviews the original question is on page 124 under the headline what metrics will look at to evaluate the success of a product the question is linkedin is testing a new feature asking a new user to upload their profile photo during the sign up phase currently a new user is asked to upload a profile photo after the sign up process what metrics will look at to evaluate the success of the future this is a typical kind of metric questions that can appear in a data science interview and we have covered a few steps to answer this kind of question in a previous video when it says testing most of the time it refers to epi testing which is one of data scientists core competences in reality stakeholders rely on data scientists to design ib tests analyze the results and make data informed decisions the book provides a simple answer but i don't think it reflects the real interview scenario because the candidate does not clarify the function of the feature and the candidate comes up with many metrics which are not very feasible in epi tests so i will provide another way to answer this question and i will expand the question to make it more like a data science interview question let's get started linkedin is testing a new feature asking a new user to upload their profile photo during the sign up phase currently a new user is asked to upload a profile photo after the signup process what metrics will look at to evaluate the success of the feature before i begin i like to make sure i understand the feature it's uploading profile photos optional or mandatory it's optional okay so for the new feature users could choose to not upload a photo during the sign up process but upload it afterwards correct what is the goal of launching the new feature what do you think would be the goal in the current sign-up process it's possible that users forget to upload profile photos after the sign up so the new feature moves uploading photos into the sign-in process and asking more users to do it and that's the function of the feature in terms of the business goal i think profile photo is one important indicator of the profile of completeness and the more complete a profile the more active the user will be after finishing sign up so i think the business goal is to improve user engagement by having more users upload their photos it's also possible that the goal is to detect fraud adding a profile photo can add credibility to a new account i mean a user with a profile photo is less likely to be a fraudulent user compared with the user without a profile photo to help us maintain focus i'll use improving engagement as a goal does it make sense makes sense continue give me a second to collect my thoughts there are three metrics i would consider i have two success metrics and one guardrail magic the first metric is to measure user engagement because that is the goal of the feature we could use the average number of invites sent per user per day or the number of posts made per user per day we want to see that the newly registered users are more engaged than those in the control group the second metric measures whether this feature is useful at all i will use the percentage of new users who upload their profile photos as a measurement this metric should be significantly larger than that of the control group because otherwise it indicates that lots of people choose to not load their profile photos which makes the feature useless other than the success metrics i suggest that we have a guardrail metric to measure things that should not degrade in pursuit of this new feature we could use the percentage of users who drop off in the signup process we don't want to see this metric increase at all it's ideal if the number stays the same or even decreases would you like me to come up with more metrics no that's good what if after running the test you see the increase of percentage of people upload profile photos but the number of people who finish sign up has gone down i think we need to see the real numbers to make a decision without seeing the numbers it's hard to evaluate the business impact what if the number of invites sent went up by two percent but the number of users completing sign up has gone down by one percent and the both are significant when you say significant do mean we see a p-value less than point zero five yes okay it means that engagement has gone up significantly but the drop in the number of users completing sign up is also significant this is a tough call it seems that we make one step forward and one step back given that the current objective is to improve engagement having more engaged users is worth it in other words i would recommend launching the feature the candidate did a good job clarifying the questions coming up with metrics to measure the success of the feature and interacting with the interviewer all of them are important to get a thumbs up from the interviewer a few comments i like to share with you first there are different metrics you could choose for this question the question is about testing a new feature rather than building a dashboard to monitor the feature so you don't need to exhaust them during the interview for epi tests providing 3 metrics will be good enough secondly a good metric should be easy to measure so that it's better to provide a time threshold for example instead of using the number of invites sent the candidate uses the average number of invites sent per user per day lastly a good metric should also fit the context if you're just simply using a metric framework such as a popular aarr framework and try to follow the acquisition activation retention referral and revenue steps without considering the specific use case then the metrics you provide may not be very convincing and you may sound robotic during the interview so there you have it a simple answer to a real magic interview question from linkedin metro questions are open-ended there are different ways to approach them let me know how you would answer this question and what you think of the sample answer i provided any feedback is welcome thank you so much for watching this video i will see you soon

Channel: Data Interview Pro

Views: 16,429

Keywords: Data Science Interview, Data Science Interview Questions, Data Science Metric Interview, Data Science Product Interview, Product sense interview, product metrics interview

Id: JjA6hvmaK7I

Length: 7min 48sec (468 seconds)

Published: Wed Jan 06 2021