Indispensable Growth Frameworks from My Years at Facebook, Twitter and Wealthfront

well this is cool eight years ago Brett was building amazing stuff and I just left the company in LA where we had to layoff 200 people and I moved up to sleep on my couch and carried everything into you all but all that is just to say like I feel pretty thankful to be in this position and to share whatever wisdom I've learned over the last eight years with everyone in the group and hopefully it's useful so what I want to talk about today is actually helping everyone in the room do one of two things either identify who's gonna be your head of growth and who can be exceptional at that or if you are somebody who works on growth at a particular business is to give you a repeatable framework where you can develop your skills within that framework and go from good to great and to be exceptional what you do because what I realized over the last eight years now in working on growth and chatting with various companies about the growth initiatives is that four years ago this idea of a growth team was still pretty new so the number one question that I would receive is do I even need this growth thing it sounds sounds ridiculous all right and then I have to chat with them about what a growth team actually does but the the tone of the conversation is switched to where the number one question I get now which is sort of the next logical step which is okay I understand I need to scroll thing but who the hell do I hire right Who am I looking for who's going to be that exceptional growth leader everyone assumes that you're gonna be able to find that individual outside the company what you may find is that you can actually find that person inside the company and that's the most likely route but what I'll talk about in specific detail are some of the frameworks that I use for thinking about my job on a daily basis and again I think you can you can reuse that as a repeatable framework all right so what function does grow serve then sort of got into this if I'm going to dive into it a little bit more and what I'd like to is think about growth like a finance team and if you're the steward of a business you're running a company and you're thinking about bringing somebody who is a growth leader into your business because maybe you're 50 or a hundred employees and you have great organic growth and now you want to accelerate that what you want to do when you're interviewing that person is actually find out if they can go beyond the surface level of explaining and understanding what growth does inside of a business right they need to be able to go beyond simply saying something like growth is about measuring and testing stuff I think at this point we've we sort of identified that that's one of the tactics or the skills but that's not nearly enough and so I'll give you an example of how I think about growth and I think you can use this example in the interview process what the individual is that that you're interviewing to head up growth for your business so here's how I think about it a finance team by definition measures understands and improves the flow of capital in and out of a business and that's important because they contribute to all sorts of incredibly important business decisions all right business decisions along the lines of how many people can I hire what can I pay them what's the equity and comp schedule gonna look like if we need to find a new office how big will it be what can we pay and where is it going to be at you know what is the rate at which we're gonna scale other parts of the organization that are capital intensive so they use that knowledge to help the business operate but what's interesting is every company and certainly the finance team eventually comes to the realization that the single biggest lever that they have for maximizing revenue potential is to actually have more users or have more customers the revenue products that you build those in itself or secondary optimization of scale and so it's a bit of a cart before before the horse scenario and that you build out this really robust finance organization but you don't build a growth team whose responsibility is to measure understand and improve the flow of users or customers in and out of your business or in and out of your product all right so that's how I want you to think about growth that's the role of growth within a business finance measures improves and understands the flow of capital growth measures understands improve the flow of customers in and out of your product in business and so when you're evaluating ahead of growth you should be digging into these questions around how does growth fit into the business what role do they play and can they explain something beyond the superficial explanation of why they measure and test stuff hey that's not enough and then there are three particular skills that I also want to focus on when it comes to that individual who's sort of the champion of growth and this is a definition that I think is closer to a director or executive level but I think you can water it down a little bit depending on the leveling of the individual you want to bring in but ahead of growth does three things right first is they contribute to or they build the the data models that you use to to simply explain how your product grows into scale right so it's a very basic data model that captures the core levers that explains why Cora gross or why Facebook grows or why LinkedIn grows or why Twitter or why wealthfront grows all right so that individual needs to have that proficiency second is they need to be able to build testing and experimentation programs within the company right they need to be able to organize a product design and engineering team around a testing schedule and they need to be able to justify the experiments that are important and the third is they need to be able to be able to identify tests and scale new customer acquisition channels those are the three things if you're gonna be an executive level leader on growth you need to be great at at least two of those if you're great at three then you've got someone special yeah and I'll talk about these in a bit more detail so building growth models there are two that I want to share and the first one I actually learned from someone who was my boss and a couple other folks in the room he was their boss as well as name is shamov you ran growth at Facebook and now runs social post capital and this is one of the basic growth equations that he uses at the at the highest level of assessing whether or not a product or a business has the ability to grow at scale for an extended period of time and as these three main come opponents and as I'm thinking about the products that I'm working on I follow this religiously and here are the three components first is can that product have a robust top the funnel meaning can it capture traffic and can you convert that traffic and an increasingly higher rate into some meaningful level of usage this is more the tactical bit I think it's probably the least important parameter in this equation and I'll explain why in a couple minutes second is magic moment I'll give an example I think a lot of folks here have experienced remember the first time I downloaded ubers app hailed a car car showed up black car picked me up my friends looked at me like I was cool I had no idea what was going on I just knew a car was coming I didn't realize it was like this fancy limo thing and then I got in it they gave me a bottle of water the person greeted me by my name drove me to the location without making me feel carsick dropped us all off no exchange of cash no driver telling me that they didn't have a credit card reader none of that stuff and then I got out and I was like what the hell that's what it should be like and I experienced that every time that I fly into LAX and I can't get over and I have to take yellow cab it sucks and so my definition of the magic moment is something that elicits an emotional response in your user of the product and ideally you do that early and often in the product experience we have these experiences with wealthfront the product that I get to work on and one of those experiences actually led to me saying like I got to work on this I went to bed one day I woke up the next I logged in and checked my account and it captured three thousand dollars in capital losses for me while I was asleep they didn't ask me to do anything it didn't charge me it just did it and that's when I realized like one that's cool I'm gonna have more money left over at the end of the year - I got to work on this right we can introduce many more emotion eliciting experiences in the product that's how finance should be alright so that's magic moment three is core product value how big is the market are you solving a legitimate problem and was your product market product market fit hypothesis correct right and what you'll find is that for products that do B and C really well a naturally happens it's like the stuff Brett was talking about right the top of funnel fills itself that's when you're in a wonderful position so when you have tremendous top of funnel growth you don't even have to fight to fill the top of the funnel at that point you just need to remove friction where friction exists and allow the product to grow as big as it can get all right so that's framework one so when you're chatting with the head of growth you should be asking this person like what are your frameworks for evaluating growth of companies right dig into that ask that person for a degree of thoughtfulness that you have needin implied to your own business but you can make this practical this is one of my favorite examples like when someone comes to me and says hey I want to work on growth and I love your product one of the first things I say is okay well how do you think we grow can you just write a simple diagram on the on the whiteboard that attempts to express how wealth run as a business grows can you do that right I think that's a basic necessary skill and I go through this exercise with products pretty commonly on my own time usually if I'm engaged for the company one on one it's the first thing that I'll deal with them which is to have a simple expression that captures what you think are the main dynamics around the growth engine for that product or business right I like to use Amazon example when I look at Amazon this is what I see at a high level these are the parameters that I've identified as being part of its growth engine so let's talk about a vertical expansion when Amazon launched sold books so you could say well Amazon's growth at that time was contained to or limited by sort of books as an as a vertical right they could sell a certain number of books to a particular size of audience and then consequently that's how big it could get right so there's some argument there that as it moved into additional verticals it unlocked additional lasing latent growth potential and then it could expand its growth at a faster pace right and so they moved into music and into jewelry and they continued to expand on verticals today so every single time they move into new vertical there's a growth function there and I probably would want a team of people figuring out what verticals till I move in into and what order and how do I make that a machine so that's that's the first one the second is what I would refer to is sort of the product inventory within that vertical or the depth right as Amazon selling ten books know right they probably understand the sell 10 million books right and they want to sell every book on the planet if they can and so their growth is proportional to the depth of product inventory again that's another thing that I would focus on and I'd probably write that on the whiteboard right after that you have something that's more standard you all of a sudden have these product pages that are all over the web and they're capable of yielding additional traffic now I want to convert that traffic at a higher rate to buyers to people buying product all right so that's another variable in their growth after that is conversion to purchase and then the average value of each purchase or the amount that people are spending and at that point that's when Amazon was injecting recommendation engines into the product right here's a good example that's sort of relevant this time of year I bought a he-man costume for Halloween skooby sweet it's got like it's full of you know fake muscles and everything so that's gonna be a lot of fun and then I was checking out just the the he-man full bodysuit and it recommended his sword I thought well of course I got to buy that that's a really right that's a really good example that just experienced so my average order value goes up and then eventually the business goes gets so sophisticated that they can do things around driving repeat purchase behavior introducing products such as Amazon Prime right you just buy and you buy and you buying you buy so when you're interviewing I had a growth and they say hey I love your business I love your product ask them to do this first if they can't do that they're full of it or they're not very good at their job right the next thing that that a head of growth needs to be able to do is develop rigorous experimentation programs right this is more the classic definition of growth which is run a bunch of a/b tests but it goes a hell of a lot deeper than that right and I want to dig into this using some real-world real-world examples and what you should be doing when chatting with somebody that wants to run growth for your business is a again evaluating the depth of their knowledge from an experimentation side of things right asking them questions around how do you identify what part of the funnel to focus on right how do you identify when to run a test when not to how do you identify what is the right test to run versus one that isn't an interesting test to run all right here's some examples one of the behaviors I've seen is that large companies they really like small optimizations and that's because they have a data advantage right at Facebook there'd be a part of the product where we would generate a hundred million visitors per week so we can run a navy test to one percent of that traffic and inside of 24 hours I have 100% confidence on the outcome of that experiment and if it gave us a 5 percent lift in something like sign up rate wonderful I'm gonna do as many of those as fast as possible and collectively add those together and let it compound right that's exactly exactly what we did I remember in 2008 I think we exited the year at somewhere around 125 130 million monthly active users and then the exact staff and the board went away to set the goal for 2009 and they came back and they said guess what you have to go to 300 million users which was scary and daunting because it had taken so many years for the company to just get to 100 million then they wanted us to get to 300 million and it turns out that at the time we are growing roughly at around one and a half percent week over week and if we grew at 2% week over week well we could hit that goal so that's the path that we went down and so here's a simple example that's probably relevant to some midsize companies today right in the first column you assume that you begin with 10 million users and you're growing at one percent week over week so at the end of the first week you have 10.1 million users at the end of the year you have 16 point eight million users that's cool pretty good numbers I probably write a check to that alright and I call them next to it same company grows at 2% week over week so they start with 10 million monthly active users into the year that 28 million that's even better right scenario 3 which is the last column is you start at 1% week or a week growth but then let's assume that this growth team is working on its active user rate and every four weeks it improves that growth rate by 5% and it does that consistently over the next 52 weeks then you end up at thirty point five million users which is more than the second scenario that's why big companies like small optimizations and that's where you conjure up the examples of of hearing about Google engineers testing 40 different shades of blue on a particular signup button but when you're a smaller company that stuff just does not work right because you don't have a hundred million users to throw a 1% test in front of you have to do something different is probably my favorite graph which is not scientific or accurate whatsoever but on one axis you have thoughtfulness how thoughtful or are you as it relates the to the experiments that you're selecting and how you're designing those in the order in which you're implementing them right how thoughtful are you it turns out that when you have tons of traffic people tend to not be that thoughtful they have a bunch of half-thought ideas and you do a bunch of really quickly but sometimes that works when you're startup and you don't have as much traffic you have to be incredibly thoughtful right because you have only so many opportunities in which you can run an experiment in the product and your time bound you said you know I raised 40 million dollars and that gives me two years of runway so I need to execute and deliver on some numbers within that time box with that said you need to run experiments that matter right and experiments that matter and the start in the startup world when you're using smaller samples have to be incredibly thoughtful I'll give some specific examples using actual website data all right so let's look at the impact of when you produce a large lift in an experiment over the control State right so we have example a example be an example see let's imagine those are three a B tests right and look at the column in green that's the percent change over the control so that's saying that an example a we ran that test and it was five percent bad better than the control thing you were testing it against an example see it was 30% better right and then look at the impact on the required sample size you you need so that's saying I need 72,000 samples in the experiment bucket and in the control bucket and I'm going to have to run that for 72 days in order to detect that that thing is five percent better than the control that's the lifetime when you're a start-up right you know 24 months a runway and you just went through like two and a half months of it on 180 test that's not going to foot the bill right and then conversely when you look at example see if it's 30% better you need two days everyone wants those results so the lesson in that is be thoughtful but also be dramatic in the changes that you're willing to make be dramatic don't take a page and say here's this button I'm gonna take that button and sort of just move it over there I'm gonna give it rounded corners I'm gonna try 40 different shades of blue it's not gonna get the job done you may run that experiment given that you have small traffic sizes and because of the small lift you may run that for months or years so produce dramatic lifts then some folks will say well what if it's dramatically worse like well you'll know that inside of two days as well and that's okay cut your losses and move on it was dramatically better earn that reward right now and let it compound as fast as fast as you can scenario two so let's look at the impact of different based conversion rates on the outcome of an experiment and the duration of that experiment so let me give an example let's go back to example a B and C so the base conversion rate just refers to the ultimate conversion right so let's say you have a hundred people on your homepage you know I'm gonna run an a/b test all the way you know from the homepage all the way down to the bottom right so I'm gonna start with some homepage change but a hundred people started on the homepage and three of them converted that's a three percent conversion rate but let's say I look at some test somewhere midway through the funnel and there are only ten people in a given day that are on that part of the funnel but three convert that's a 30% base conversion rate so that's what I'm referring to they convert the base conversion is usually a proxy for depth of the funnel the higher the base conversion rate generally the better for the same reason you'll notice that with a high base conversion rate if I'm starting at a 20 conversion rate and I lift that by 10% I can detect that inside of two days versus the other scenario of I have a very small based conversion rate and I lift that by 10% so I'm trying to statistically detect the difference between a three percent conversion rate and a 3.3 percent conversion rate it turns out that's hard to do you need a lot of samples in that test in order to detect it at high confidence so the lesson here is go deep in funnels start there it's especially true if you have a product that requires a significant sort of purchase decision right so if I'm going to buy something worth a thousand dollars online and if you want to drive the conversion rate on people actually buying it don't run a test on the homepage run a test at the part of the purchase funnel all right we're about ready to enter their credit card details that's where you want to be testing you can resolve those tests more quickly probably because you're dealing with a committed customer a committed couple customers more likely to respond to a different designer experience that's where you want to be testing so when you're chatting with whoever's your head of growth can they explain to you how they pick their spot in the funnel right usually the response you'll get is I'm gonna test everything because I can test everything really quickly I'm just gonna throw spaghetti at the wall and see what sticks that's not your executive leader maybe that's a good entry-level person maybe that's good enough to get started maybe you have somebody internally that can already do but what you really really want is somebody that understands the value and the benefit of intelligent experiment selection and design all right so a couple final thoughts these are just random thoughts that I like to share you can't sustainably grow something that sucks so I think the lesson here is if you don't have healthy organic growth hiring a growth person is not the solution right that's a product challenge hiring your your growth team or your head of growth I suggest happens after you demonstrate organic growth after you prove or the organic growth growth proves that the market appreciates the product that you've introduced to it and that that the market considers it valuable at that point growth is fuel on the fire it's not the thing that starts the fire right so you can't sustainably grow something that sucks nothing replaces an incredible product you don't need or want a growth hacker I say this from a recruiting perspective because it can be more of an anti signal right how many growth hackers do you know have successfully executed on two three or four years worth of growth work at Airbnb or Dropbox or Facebook or Pinterest or Twitter or LinkedIn I don't know any but I know a lot of growth hackers that are clever and sophisticated in their own way and they understand tactics really well and they've worked on smaller products but that's just part of it if you're looking for a growth leader like think of it this way if I'm looking for a head of sales an enterprise company am I gonna hire a sales hacker ahead of finance am i hiring a finance hacker whatever that is it sounds like extortion really so probably not so I think the optics that are interesting but don't fall sucker to that right you're looking for substance not title and your growth elite needs to be a product person right again it goes back to my message on this isn't just about experimentation and what you may find is that some of the most meaningful product changes that you may actually experiment with come from a deep understanding and empathy for the consumer for the user of your product right with well friend as a business like what we're finding is that the most meaningful changes that we can make to the product that when we test them and they move the numbers in a meaningful way they're actually derivatives of deeper behavioral finance understandings when somebody goes home for a weekend and reads a 30 page white paper that a couple finance researchers had sort of conducted two years worth of market research on a company on how to get individuals to contribute to the 401k more often may find some behavioral finance difference that we can integrate into the product and move the numbers in that way okay so you had a growth needs to be a product person because the risk that you run is that all companies identify themselves on some spectrum of what I would say is sort of the clash of of science and art right your DNA as a company aligned to more of the the art style like I'm just gonna build great products I'm not gonna look at data whatsoever I'm not gonna test anything I'm just gonna build and then you have the other side which is I'm gonna measure and test everything I think companies eventually realize that they need to sort of revert to the mean and strike some balance between the two but if you hire ahead of growth who can only do the science bit but has no empathy or passion for the value of quality product experiences I think you're gonna create that or exacerbate that tension between science and art that you probably don't want to exacerbate so that's it that's my guidance thanks for the time I appreciate it if you have any questions grab me outside [Applause]

Channel: FirstRoundCapital

Views: 15,794

Rating: 4.9648094 out of 5

Keywords: growth, leadership, wealthfront, startup growth, head of growth, hiring, startup, entrepreneur, management

Id: OxNrMeRme0E

Length: 25min 37sec (1537 seconds)

Published: Mon Dec 03 2018