Why Software Estimations Are Always Wrong

estimation is just a fancy word for guessing we're usually guessing about the problem that we need to solve how we'll solve it how long it will take and so how much it will cost oh and if we're really Advanced we may try guessing about how much money we will make from the idea too so what is the point of an estimate really and why are they always wrong oh and while we're exploring those ideas how can we do a better job of choosing what to work on next [Music] hi I'm Dave fley welcome to my channel and if you haven't been here before please do hit subscribe and if you enjoy the content today hit like as well we can estimate all sorts of things but they are by their nature always just predictions of the future and so just guesses and all are deeply prone to error we can't really know how much money our killer idea will make or how long it will take us to build a software that's new in some way if you at your aim is to create great software products this is always an incremental progression great product ideas in reality don't spring fully formed from the mind of some genius they evolve over time in a series of steps and get better and better until they are great this stuff is really difficult to predict most in software when we discuss estimates what we're really talking about is estimates that help us to decide whether or not we should do something is this project or feature worth all of the effort fundamentally the question that we are attempting to answer is what should we work on next so if that's the case how do you figure out what to work on next the traditional response is often probably based on hippo the highest paid person's opinion but hippos really make good products what we'd really like to find is the most value for the least work so how do we pick those easy high value things I have a simple tool that I've been using and recommending for some years now that I think really helps us to focus on this question sometimes though the winds don't always come from where you expect I had a long-term client a big company doing complicated things I've been working with them for a couple of years and one of the people that I was enjoying working with quite a lot was their lead product owner he was a big lean thinking proponent within the firm but was struggling to help the product owners that worked for him to establish a leaner approach to product ownership they were struggling with big overc complex estimations the feeling the need for certainty not feeling confident to make a choice unless they are detailed plans in place with estimations for the work and the cost of the work as well as detailed market research and guesses about the value of the features being discussed we were discussing all this and how much work was involved and how to do better when I remembered something that I had learned from my friend Jeff Patton a kind of informal qualitative planning tool that works really nicely to visualize the planning problem my friend the product owner loved this subsequently its adoption made a huge difference in improving the planning process for his organization and helped the product owners to focus better and more easily on the features that really mattered to people and so ultimately delivered better value to their customers this approach helped them to prioritize things much more easily and with much less work by now I'm imagining you imagining some big complex tool but that's not it it is really simple simple enough in fact that you can remember all of the details that you need to know to be able to use it next time you're doing some planning let me pause there to thank our sponsors we're fortunate to be sponsored by equal experts tricentis and transfit all of these companies offer products and services that are well aligned with the topics that we discuss on this channel every week so if you're looking for excellence in continuous delivery and software engineering click on the links in the description below to both support our sponsors and for you to maybe learn some things that might be interesting to you mostly as a species we are rubbish at predicting the future so planning is always difficult and plans are always wrong though the good ones can also be useful so we need some tools to help us to do a better job than simply just guessing some kind of structured guessing maybe what is generally regarded as the gold standard for rational planning is probably something a bit more complicated than the thing that I'm going to end up recommending in this video it's called CD3 cost of delay divided by duration fundamentally to make rational planning decisions in software we need to make some kind of cost benefit calculation and do a tradeoff how do we maximize the benefit to us for the minimal costs there are lots of organizations that don't work this way these are really cargo cult kind of organizations they they only see the costs and so always organize their work to minimize them that's certainly part of what we need to do but these organizations will often do rather silly things things like delaying work on valuable ideas because the plan says that they should be working on some lower value thing right now that's because the benefit wasn't included in the prioritization decision only the cost this is stupid however low the cost the next step up from this is to estimate the business value for each new feature and prioritize work to ensure that we always focus on the most valuable things most organizations don't do a great job of this partly because this is a very difficult thing to do well how do you work out how valuable an idea will be if you haven't even tried it yet at this point the common mistake is to assign the failure of this approach to bad planning and blame it on inaccuracy of in the estimates of either the cost or the benefit or both the most common response to this that I've seen is to up the level of planning effort with the assumption that this will improve the plan by increasing the Precision of the estimates for both costs and benefits but this is wrong in two ways this is mistaking accuracy for precision accurate estimates would be nice but can easily also be vague so vague to be useless it's accurate to say that we will deliver in 10 years but not very helpful precise estimates are completely irrelevant Beyond a fairly crude level it's precise to say we will deliver on Tuesday at 2:30 but irrelevant someone recently told me that their boss insisted on estimates to the nearest quarter of an hour what a crazy waste of time money and development effort that is in science this problem is solved by defining what they call error bars for any measurement we should understand the degree of uncertainty involved in the measurement or ES estimate and there is no point measuring or estimating things more precisely than that in software estimation the error bars are huge at the start of a project research says that it's out by a factor of four four times your estimate which actually caus much estimation into question altogether hence the no estimates movement so working to increase Precision is completely the wrong thing to aim for another problem is how do we estimate the value of a feature traditional measures are value don't usually Factor the costs in at all most development organizations can't even tell you if a project paid for itself even after the project has finished let alone before it starts the mathematical model is simply wrong however accurate or precise the estimates because we're missing a whole important dimension in guessing the value time which feature should we build first this one that will cost 3 weeks but we'll ultimately have a business value of 2 million or this one that will take five days and ultimately have a business value of 100,000 well obviously 2 million's better than 100,000 so option one is better right well that depends on what ultimately a value means in those statements if ultimately for option one means 10 years and ultimately for option two means one month then after one year option one is worth 200,000 and option two is worth 1.2 million another kind of problem with all of this is what about option three that will take five days but has zero value for customers obviously this is a no-brainer we don't want to do this one except what if this is a regulatory change and as is sometimes possible with such changes is what if there's a fine every week that this change isn't in place how do we calculate the value of that feature to prioritize stuff we're missing several important things here and those things are what CD3 helps to solve instead of asking what is this feature worth the much better question is how much does it cost us not to have this feature per unit of time now we can see the real of going more slowly if you'd like to understand this better there's a link to a more detailed explanation of CD3 in the description to this video the problem with CD3 though is that once again it still relies on us coming up with a guess for that cost of delay and it's easy for us to fall into the Trap of being over accurate and over precise in our guesses great if you can calculate this not much help if you can't do that that very well as I said predicting the future is difficult at heart I I'm a pragmatist and so in general I like Simple Solutions to things so let's look at this simpler more pragmatic alternative it starts by asking yourself two questions about each feature that you think you'd like to deliver first how much do you think your users would value this feature do you think that they would think it was worth a beer their monthly salary a holiday their car or their house and then ask yourself how much is it going to cost to build it will it cost a beer a monthly salary a holiday my car or my house the things down here are obviously a complete no-brainer of course we want to work on things that users think are worth their house but that will will only cost us a beer to produce so we always pick these things to do first things down here are stupid why on Earth would we spend a house worth of effort to produce something that's only worth of beer to our users so we are definitely not going to do these things at all everything in the middle we don't understand well enough yet to decide what to do with them so our next step then is to figure out how we could learn more to help us to make a choice maybe this could be some kind of experiment in production to refine our view of the value our users will place on the idea maybe an AB test of some different variations of the idea to help us think about it from different angles a simple version of this feature that makes it a lot easier to make perhaps none of this stuff needs or should be very precise it's all qualitative and approximate but qualitative and approximate is where error bars are for software estimation so another thing that helps is working in small steps even without deep accuracy and irrelevant Precision this model helps us to focus our minds on the things that really matter and to stop us wasting time and effort on things that don't so I really like this simple slightly silly but extremely useful tool for planning one more thing before I go I've been describing this as Jeff Patton's model for years now I was recently having a beer with Jeff and mentioned how much I liked it and how much I'd used it and I thanked him for for for introducing me to the idea at which point he looked at my picture and didn't recognize it as anything he'd ever said so now I have no idea who I stole this from but it certainly wasn't my idea and seemingly it wasn't Jeff's either thank you very much for watching thank you also to our patreon members if you enjoy our stuff you're not already a patreon supporter please do consider supporting the channel and our work here by joining our patreon community if you do choose to support us at any level you can get an annual membership at a 15% discount thank you and [Music] bye-bye

Channel: Continuous Delivery

Views: 49,641

Keywords: software estimations, how to estimate software development time, software estimation, estimating software, how to estimate software projects, software estimation without guessing, software project management, software development life cycle, how to manage software development projects, how to manage software projects, devops, Continuous Delivery, software engineering, software development, Dave Farley, no estimates

Id: OS6gzabM0pI

Length: 14min 22sec (862 seconds)

Published: Wed Oct 11 2023