[Music] hi there my name is Tom blumfield I'm a group partner at Y combinator and today we're going to be talking about one of my favorite topics metrics and why they're so useful for startups so why are metrics important first of all it's pretty obvious that with better metrics you'll make better decisions it's like flying an airplane with no instrument you're Flying Blind you don't know what's happening to the aircraft and you're not in control having great metrics is like having great instruments in an aircraft it lets you tweak and iterate and make sure you're really in control of your startup we often see and I've seen in even in the last two or three weeks Founders who've had these great launches they've launched on Hacker News on product hunt and they've had hundreds of people come and use their service day after day but they have no idea how many of those news users are new users or returning users they don't know if they're daily active or weekly active they could be churning off all of their users instantly and they don't know at all so the first thing they do after launching blind is to go back and build metrics in we would advise you don't do that you should build basic metrics into a product before you launch and as an investor it's really easy to tell Founders who are in command of their metrics versus Founders who aren't and it's really impressive when founders can talk about what percentage best signups are da or W or what the annual revenue per user is and we'll go into some of these in detail but it's a big differentiator when a Founder can talk so fluently about these metrics before we dive in I want to give a couple of warnings The Other Extreme is also bad so it's a Founder who even before they launched has a dashboard with perhaps 500 metrics maybe they've been a product manager at a big tech company or they've just watched too many YouTube videos but they want to make every decision in their startup with metrics and when you have only have a few hundred or a few thousand users that's basically impossible you know they want to split test everything should this button be blue or green and frankly it doesn't matter and you don't have the volume of users or data to make those kind of split tests sensibly so what you should do is certainly split test the really important decisions you know should the cost per user be $80 per year or $200 per year that's a really good experiment to split test but you know making buttons red or green that's not really something you have the scale to split test until you're really at the the size of Google or Facebook a final warning don't hide behind your metrics you've still got to get out of the building and talk to customers Brian from Airbnb still hosts Airbnb users in his home it's an obsession with staying close to customers so you can't let metrics get in the way of that so let's get started you're planning a product launch in perhaps a week or two and maybe maybe you've not got any metrics in place yet what do you do the first thing is to pick four or five key metrics to track accurately not 30 or 50 four or five is fine this number will grow over time we'll talk about what those key metrics should be in eight a little bit you should pick the most straightforward analytics solution you can operate it might just be uh your SQL database making simple SQL queries to count to the number of signups post hog from Winter 2020 has a great SQL Analytics tool you can use on top of pretty much any SQL database so you should check that out you should also agree the definitions of these four or five key metrics and stick with them so it might not be the absolute perfect definition of an active user but constant arguments about what your key metrics are are even worse than having no metrics at all so your whole team has to come together and agree that an active user is someone who uses the product every day or at least once a week or at least five times a week it honestly matters matters less the precise definition than you actually all agree with it I've remember so many disagreements where the marketing team said you know we've sent you 2,500 New Leads this month and the sales team says no no no they weren't qualified leads they don't meet our definitions and this disagreement internally just destroys the productivity of meetings where metrics are involved so you really have to have centralized definitions of metrics that are written down and everyone agrees on so say you launched and perhaps the metrics aren't quite what you hoped the weekly active users aren't quite as high as you you originally wanted in that situation Founders are often tempted to pick a different metric or change the definition of those metrics so instead of a weekly active let's go for a monthly active the number looks a bit better honestly you're only fooling yourself in this situation it's really really important that you keep the definition of your metric consistent over time to see if you're improving or not that's why it's so hard to compare metrics between different companies definition just vary so a weekly active user at my company monzo was someone who transacted who made a financial transaction at least once a week an active user at some of our competitors used different definitions maybe it was every two weeks or eight weeks and so this active us account between companies just became totally meaningless just important that you keep it internally consistent so you're keeping a good track of it so now let's talk about what those key metrics are those four or five you should really start tracking from early on it will vary from every company back in the early days of the internet companies like to use metrics like page views or unique visitors or something like that because they're really really big numbers and startup Founders love to report really big numbers you've probably heard the term vanity metrics these are numbers that seem really really big and perhaps they keep increasing they're not actually tied to the success of your company more recently common vanity metrics are things like gmv or gross merchandise value that's the total dollar value of goods that are sold on eBay rather than eBay's Revenue itself or gross transaction value for a fintech like monzo you could report we're transacting $50 billion do a year it sounds like a really really big number but the revenue that the company makes can be very very different and so almost always especially for B2B companies your key metric should be Revenue if you pick another number take gross transaction value you'll find that your employees and eventually you might might start optimizing for that number I worked with a Neo Bank a couple of batches ago in the Middle East and they were reporting gross transaction value and they're very very happy they came to every group office hours and their gross transaction value was growing 50% every two weeks it looked really great and we scratched beneath the surface a little bit and it turned out that they were signing up much much bigger customers who had higher transaction value but giving them massive cash back massive rebates to transact on the platform so whilst gross transaction value was Rising every two week weeks Revenue actually was pretty flat for about the last two months the founders were tricking themselves they were fooling themselves into thinking their company was succeeding when in fact it was pretty flat in Revenue terms so revenue is the key metric I would suggest for most B2B companies and don't hide if your Revenue isn't good one of the most impressive Founders I've ever worked with sent 10 successive monthly investor update emails with zero with a big zero as the main metric at the top of the emailed she kept herself honest she was honest with investors and it became clear what they needed to focus on to fix the company so if you ashamed of this number you hide it away it's easy to kid yourself but I think if you put it up front and Central and pay attention to it it's the right thing to focus on so two other key metrics especially for investor updates alongside your Revenue please include burn rate that should be net what that means is monthly costs minus your revenues if you're loss making which most early startups are it's the amount by which your bank balance decreases every month that is your burn rate and your Runway is a function of that so say you have a million dollar in the bank and your burn rate is $100,000 a month that means you have 10 months Runway that means in 10 months you're going to run out of money and the the startup will be bankrupt those three numbers Revenue burn rate and Runway are absolutely crucial to include and if they're not at the top of your investor updates honestly I always uh assume this founder has something to hide for Consumer companies we have a separate video diving deep into the metrics that are important for those kind of startups and often revenue is very important but for the earliest days of consumer companies often you're trying to get some kind of critical mass or network effect and so growing the active user base in the early days for a consumer company might in some cases be more important than Revenue so we've talked about the main three metrics that should be at the top of every investor update that's Revenue burn rate and Runway now we're going to dive a little bit deeper one of the most important metrics for all startups really is retention this is the idea that if you sign up a 100 paying customers say you sign them up in January how many are still paying you in February March and April two months 3 months four months later that's your retention rate so it might be 80% or 70% it's a sign that people love your product they keep coming back and they keep paying for it so you can measure it for all customers who signed up in January and the subsequent months and that's your January cohort and then you measure the same for all of the customers who signed up in February that's your February cohort and you can stack these cohorts on top of each other and there are a number of different ways of graphing this you might have a heat map and some analytics tools do this really nicely you might have a curve that sort of decays over time those are two pretty common ways of graphing it and we'll show some examples but I'd like to suggest a third way this is is when it really clicked for me I'd heard everyone tell me that Revenue was really really important but it only really clicked for me after I worked at a dating startup that actually had very bad retention and ultimately failed sadly but it clicked when I stacked these cohorts on top of each other so you see the January cohort at the bottom and then the February cohort and then the March cohort and the April cohort and what happens if you have sticky cohorts if your retention is really high you know 80 90 100% is you cohorts stay really fat over time and you build up this layer cake so you can imagine what happens two or three years later if you have dozens of these monthly cohorts stacked on top of each other they're all paying you money they're all contributing to your Revenue every single month even three years later and if you're in a low churn business it adds up to a layer cake that looks something like this so this is an 18month graph of monthly cohorts each month you're adding a new layer on and after 18 months you've got 18 cohorts still paying you this was like my first company go cardless it was a a recurring payments company very similar to stripe and with those kind of companies people Implement a payment solution once they don't really like to change it every month there's a lot of effort so the customers are very very sticky and you can imagine the team at go cardless goes on holiday you know for a month after 18 months and the revenue stays pretty consistent the beauty is actually expanding Revenue so if those customers you signed up in January launch grow with the company they're using goard or stripe as a payment processor they're transacting more volume in year two and year three they're growing their business the revenue for for strip or go carders actually increases as well so the team perhaps goes on holiday or signs up no new customers the business still grows Revenue that's the beauty of this this High retention business it's sort of growing constantly underneath you you're adding these layers and layers and layers of Revenue and eventually become Unstoppable but that only happens if your retention flattens out at some point if these Decay curves flatten at some point and it almost matters that they flatten out at any point as opposed to a high point you know I I take a 20% retention that flattens out over a higher retention initially that goes to zero because if you sign up 100 people in January and by month three or month six they've all churn off they've all stopped paying you get a very different layer cake that lovely flat layer cake that builds up and up over time if your customers all churn out looks like this so you can see customers you sign up in month one by month three have more or less gone and let's fast forward to month six they've all gone by month 9 or month 10 and so rather than building up secure and steadfast layers month after month you're actually scrambling to fill up a leaky bucket you're pouring water into the top of the bucket and it's leaking out of the bucket just as fast as you can fill it up you can imagine this is an impossible task and so if your business has customers that that don't retain where retention goes to zero you'll reach some natural Plateau where you're working as hard as you can to fill up the customers who simply churned out last month and it's very very hard to build a big business like that it's a futile Endeavor so we talked about overall retention number of customers for B2B startups people often talk about net dollar retention this is just a fancy way of calculating retention mostly used in B2B SAS companies so let's take an example we've started an AI customer service chatbot very uh invogue at the moment and say we we sign up 10 paying customers in January in the first month and they're each paying $10,000 a month so 10K Mr each you're at 100K of monthly recurring Revenue feels pretty good right let's fast forward a year and in each of those subsequent months you'll have signed up more customers let's let's ignore them for now focus on those 10 initial customers you signed up in January but fast forward 12 months perhaps two of those customers have canceled their contracts at some point in the year so we're down to $80,000 of monthly revenue from that cohort but you've also upsold three customers perhaps you've introduced some new functionality or they've they're using it more and instead of paying 10,000 they're each paying 20,000 perhaps you do phone chat as well as text chat so that's $30,000 of additional Revenue so we've lost 20 but gained 30 so that's net 10K plus so $110,000 of monthly revenue from that January cohort that's why it's called net dollar retention it's the amount you've gained that's netted off against the amount you've lost so that cohort that was making $100,000 at the start in January of year one a year later January of year two is making a $110,000 that's equivalent to 110% net dollar retention so a net dollar retention above 100% means your cohorts are growing over time if your net dollar attention is below 100% they're shrinking over time youve got to pour more water into the funnel to fill up the the Leaky buckets we talked about and that's what gives very sticky businesses like stripe like go carders like PayPal this exponential growth it's adding new customers every month but having existing customers grow underneath them as well and that gives you this exponential growth curve that's very very impressive the final thing as a benchmark any early stage B2B SAS company should be looking at net dollar retention well above 100% this is for several reasons first of all you've probably underpriced your product with your first launch so you might charge $10,000 a month for your initial customers you realize pretty quickly uh that the product could be sold for 20 or $30,000 secondly you're adding features the whole time presumably you're improving your product and so that makes it more appealing and customers are willing to pay more money thirdly you should be getting it better at sales and upselling over time as well it' be weird if you weren't getting better at that and so those three reasons net dollar retention for early stage B2B startup should be 125% 150% would be great even higher than that for mature companies uh in the same range 110% 120% is pretty good net dollar attention if your net dollar attention is below 100% especially for Enterprise B2B SAS something is wrong you are churning off customers they don't love the product and I would invest in fixing that talking to customers and figuring out why they're turning off rather than trying to to shove more customers in the top of the funnel by investing in sales and marketing for example net dollar retention is absolutely crucial for B2B SAS companies okay the second Deep dive we're going to do on B2B metrics and this is applicable to Consumer companies as well is gross margin gross margin is your Revenue the money you get from customers minus the cost of goods sold so you can imagine that if you're a grocery store that's most obvious you're selling you're selling sandwiches for example the cost of good sold is the cost of the bread and the cost of the butter and the the filling that goes in the sandwich that's cost of good sold for a software company it's any cost that varies per customer or for each incremental customer you incur more cost so let's go back to example we had earlier we were running a an AI customer service bot and you're probably using something like open AI or anthropic to power the core model behind that and so the cost the credits that you pay to open AI or anthropic or someone else is your cost of goods sold we didn't used to talk about this very much for B2B SAS companies because the cost of goods was very very minimal for Pure software it might have been your AWS bill or your bandwidth bill or something like that it's minimal and so pure B2B SAS companies in the past might have had gross margins of 95% you know you sell $100 worth of software and it's only $5 of cost and so people just assume it's very very high margin but these days as software sort of taking over more and more Industries gross margin has become more and more important so for AI companies today the gross margin the amount they pay to open AI or anthropic or others for the foundation model is a really important cost and by the way just because you're getting free credits doesn't mean that that's a cost that doesn't exist it just means you're hiding it for the moment so companies that hide behind open AI credits and claim that they've got these huge huge gross margins have a nasty shock coming when those credits run out it's also why heavily operational businesses are so tricky and when a company joins YC with a with something like a grocery delivery business or really any kind of business where a lot of humans involved a lot of operational processes going along you maybe you paint houses or install heat pumps or something you have to pay a lot more attention to the gross margin because it's very rare that it's as high as 95% you might be down at 5 10 15% gross margins which means you have to do a lot more work you have to get a lot more customers a lot more Revenue to generate the same gross margin and that gross margin is the thing that can then pay your head office rent your engineering salaries all of those remaining costs that don't vary per customer but still have to be deducted before you get to profitability and so for operationally intensive businesses we often try to work with Founders to see if there's a software only version of their business that they can run at a much higher margin so for example instead of running a delivery company where you have Vans and bikes and and delivery people instead can you take the software that powers all of that and sell it to other delivery companies you're going to have a probably a much easier life certainly you're going to have much higher gross margins if you do that so in the zero interest rate environment sort of 2010 to 2021 period companies were scaling negative margin businesses because Capital was so cheap famously Uber did this they used capital as a weapon so they took these businesses that were initially negative gross margin that means they're effectively selling $10 worth of service but only charging $9 so losing money on every single order but trying to get to a sort of a network effect or a Tipping Point um for Uber famously it was a certain number of drivers certain density of drivers and riders in a certain city that gets the flywheel going but when they launched in a new city they didn't have that density and so they had to subsidize drivers and subsidize Riders which made it a negative gross margin business and so they raised enormous amounts of capital to expand across the globe before competitors could catch up but burnt tens of billions of dollars of invested money in doing so and that Blitz scaling approach that that scaling of negative margin got popular with Founders and so we saw it in ride sharing then we saw it in 10-minute grocery delivery we saw it with electric scooters and honestly there's like a a whole Wasteland of startups that tried to do that and then realize they couldn't continue to raise money as investors just didn't want to keep subsidizing these businesses and certainly now with much higher interest rates Capital has become much more expensive investors are really really loed to invest in negative margin businesses and it's much much harder to scale those those negative margins we did this at monzo so monzo was an online bank in the UK and for our first half million customers or so we were losing money on every customer 30 or 40 per customer customer we scale to more than half a million of those customers it costs a lot but we had a plan to turn it around so we brought technology in house we didn't rely so much on external vendors we introduced charges for certain things we introduced new products that customers were happy to pay for and over time we flipped those negative unit economics so rather than losing 30 or 40 pounds per customer we ended up when I was there making 30 or 40 pound per customer and now three or four years later monzo is profitable so if if you start with negative unit economics you really really have to have a plan to fix them and I would really advise you don't scale your customer base you don't try and grow as quickly as possible whilst you have negative unit economics you fix them first and then you scale pH we covered a lot of stuff today so as a recap we talked about revenue and why it's the best core metric for most B2B companies then we talked about retention and it's fancy cousin net dollar retention and why having a net dollar retention above 100% is so important for B2B startups and we finish with gross margin and why it's so important not to scale businesses with negative gross margins I wrap up with some final thoughts make sure you're tracking your four or five key metrics before you launch don't launch without metrics in place it's like Flying Blind be rigorous in what you track track the right metrics don't fall for vanity metrics like gross merchandise value your impressions or unique users have a clear definition of each of your metrics and a central way of measuring them in your company to avoid those pointless arguments that derails meetings don't hide behind your metrics you can't split everything especially as a small startup so a lot of these decisions just have to be made by talking to your users and using your product intuition you still have to get out of the building and talk to customers that's so important so I hope that helps run your startup with the right blend of metrics talking to customers and product intuition those three are a vital blend thanks for watching [Music] today
