The Most Powerful Software Development Process Is The Easiest

I guess I'm a top-down thinker I like to try and have a mental model of the problem that I'm working on as a starting point to solving it so if our problem is how do we build better software faster they may be thinking about an ideal model for software development is quite a good place to start then we can try to figure out how closely we can approach that ideal so what does an ideal version of software development look like and what would it take for us to approach that ideal [Music] hi I'm Dave Farley of continuous delivery welcome to my channel if you haven't been here before please do hit subscribe and if you enjoy the content today hit like as well we're extremely fortunate to be sponsored by equal experts trisentis transfig launch Darkly and Roost all these companies offer products and services that are very well aligned with the topics that we discuss on this channel every week so if you're looking for excellence in continuous delivery and software engineering click on the links in the description below to check them out if you'd like to learn more about how to build better software faster check out my training course of the same name if you're wondering if this is the course for you here are a few comments from people who've already attended this course in this episode I want to paint a picture that I hope we can all to some extent agree on of what an ideal software development process would look like then I'd like to show how to get as close to this ideal as we can in the real world I think that this is a sensible way to try and address a problem after all we may as well strive for an ideal even if we can't make it all the way actually in software development I think that we can get quite a lot closer to Ideal than most people think I'm going to try and convince you of that today so if my ideal seems unrealistic to you in the early parts of the video please try and suspend your disbelief for a little while until the end to see if I can convince you otherwise let's start with what software development is really for you might write software for fun or to learn something new but if we're being paid to write software then our job is to solve problems for other people with it so if we want to do that there are three things that we have to do we need to understand the problem that we're trying to solve we need to solve the problem and we need to confirm that the problem is actually solved so as an absolute minimum we need to some form of requirement that describes the problem that it is that we're trying to solve and if we're using software to solve the problem we need some code of some kind that's going to solve it and we need some tests that confirm that the problem is actually solved when we've done if our development process is to be ideal then we'd certainly want great quality too and we'd like to achieve all of this with the minimum amount of work there's going to be work sure but we'd like the work to be only what's absolutely essential to the delivery of our great quality software so what if in our ideal development process these were the only things that we worked on requirements code and tests I suppose it may depend on what we mean by all all three but let's start there the next set of principles that I think really matter are the ideas that help us to learn which is why they're at the core of my book on software engineering the reality of software development is that at the start no one really knows the answers we don't know what our users want they don't want to know what they want either and even once we guess what they may want they're almost certainly wrong we won't know how to solve the problem and once we start solving the problems we'll find out the places where our Solutions are wrong too unless our software is so trivial that we instantly know the answer and so short-lived that it is disposable then we will need to retain our ability to change our minds and change our software too because even if our code is simple the circumstances in which it is used our understanding of it and our users use of it will all change this change is outside of our control and is inevitable working to retain our ability to change our software then is how we retain our ability to respond to external and internal changes in our understanding to respond to the things that we learn as we make progress so for that we need two things in our approach to working we need to organize to be able to spot the points where we need to change something and in our ideal approach to development we need to work in ways that maintain our freedom to make those changes easily safely and with confidence once we recognize the need crucial to our ability to do both of these things is that we need to make progress in a series of small steps I've been lucky to have worked with some genuinely great software developers during the course of my career and the closest that I can come to understanding what makes them great is that they all make progress in very small steps and check their progress all of the time if the steps are too big then we won't spot when we make a mistake until we've already done lots of work also if the steps are too big when we need to undo a misstep then it will be more difficult so these small steps are a key tool in maintaining our ability to spot mistakes and change our system when we notice problems and spot new directions in which to proceed because then we can verify our progress as we go software development is significantly about learning allowing us to spot when changes are necessary and managing complexity so that we can change our systems easily when we screw up right so far in our ideal development process we've laid out some principles we'll try and do the minimum of work maximizing our time spent on the things that we must do we'd prefer to organize our work in many small steps rather than fewer big ones growing our system incrementally as our understanding of the problem evolves and demonstrating to ourselves that the part of the problem that is our current focus is actually solved the ideal that I am painting is a very incremental evolutionary approach to development the starting point for any new feature in our system is going to be a kind of vague fuzzy idea because we don't know the answers yet it's always like that however great the idea initially it won't be detailed and precise at least it won't be detail and precise enough to know exactly what the software we're going to write is going to be so our job is to translate these vague fuzzy ideas into something that a computer can understand and act upon it's a dumb idea to think that we can do all of this in a single bound that if we are smart enough and think really hard we'll be able to perfectly predict what our users need how to solve their problem and how to deliver that solution to them so a need to organize our work in ways that allow us to explore the problem as we go discover its intricacies and React to what we find along the way at the start of a software project our guesses will be less informed than later on software development takes time and as time progresses stuff changes so even if our guesses would fit perfectly at the start it's inevitable that some of our guesses will be wrong later on this matters because it means that we are if we are sensible we must retain our ability to change our minds and change the system that we're building to match our new Under understanding and thinking as it evolves so I'd say that in an ideal development process we must retain our ability to change our system at any point we need to flatten the cost of change curves so that whatever we change whenever we change it it's going to take a similar match of effort and time so at the start of our ideal software development we won't have and don't need to have all of the answers because if we make a mistake we'll be able to spot it and correct it so in our ideal we want to get the fundamentals in place whatever they may be in a way that we can build on top of them perhaps the most fundamental behavior of our system is our ability to get the code up and running and available for use yes you may want to deliver the first simplest version of your home page or the basic foundations of your login system but wherever you decide to start until you can run the code and try it out you don't yet know that it actually works so although I said earlier that we only need to capture requirements write code and test things in our ideal approach we do need to do that to the point of having a working usable system maybe you think that this is cheating but I'm going to count getting my software up and running Deployable as coding and I want to be able to confirm that with tests too so even in a perfect ideal approach coding means everything necessary we should get our system into a working usable state the next ideal in our ideal model is that you keep the system working all the time after all why would you want to do anything else it's obviously better that it's working than not this means that after every Small Change we could if we chose to release our changes into production ideally with no extra work remember we're talking about ideals here so suspend any disbelief so now let's start thinking what it would take to approach these kinds of ideals our starting point is to organize this as a process of translation a series of small steps too we'll start with the vague fuzzy idea and take a tiny step forwards by capturing this idea in a very slightly more precise language I use a story or similar the idea of this is to consciously avoid any hint of the solution right now if your user story describes your UI or how you expect your system to work you're on the wrong track and have taken too big a step this story should always be from the perspective of a user of your system not necessarily the end user if you're working on a platform team but your users nonetheless the point here is to express the need not the solution it's very common mistake to try and specify the solution in the requirements at this point but it's a big mistake instead we'd like to specify the outcome that our users need stories written this way are much more durable in the face of change we should be able to find a variety of different solutions that could meet the needs of the story at this point the big danger of taking too big a step here into solution is that it fixes the solution too soon and so ends up with the requirements being too tightly coupled to that solution now any small change invalidates requirements tests and solution altogether how much better if different types of change needed only one place to fix them coupling like this builds in lots of unnecessary work and rework so we definitely want to avoid that much better then to translate in smaller steps The Next Step from our slightly more specific solution than our vague wish is to identify an example that if it existed would demonstrate that the user need expressed in the story actually exists these are acceptance criteria for the story this is another small step but these examples are now a bit more specific and a bit more precise in defining the behavior that we want from our system now we're ready to turn each of these examples into an executable specification we're going to translate our example into a simple grammar that we can run as a test once again we've made another small Advance towards being more precise and more specific now we have an automated specification for our work that asserts what our system needs to do in a very specific way and that demonstrates that a system delivers a feature that our users really want there's still no hint of a solution here yet so however our system changes it won't break these specifications the only way these specifications are wrong is if the user needs changes these things are fantastic tools to drive the development process have you ever worked on software where you didn't really understand what it was that it was actually doing have you ever wished that your developers and domain experts could understand each other better specifying the software like this fixes both of these problems but without us adding lots of extra work or lots of extraneous bureaucracy the idea here is to allow development teams the freedom to come up with solutions that solve the problem based on their knowledge of and focus on the problem specified by the executable specifications and their detailed understanding of the system as it stands by avoiding defining the solution in the requirements we leave the dev team Freer to create great Solutions now is the time though to start getting into the design of the solution the development team has the executable specification so they know what the target is they know the software so they know their starting point for the new feature so now they start working on the design and implementation of the system to fulfill that feature remember in my ideal approach development teams will maintain their software in a releasable state to do that we need to check after every small change that everything is still working to do that as a developer I need to know that my change works with your changes too because if our changes don't work together our software is by definition not in a releasable state so we need to integrate our changes together frequently you might even say continuously in continuous integration we must check that everyone's changes work together at least once per day that's part of the definition of continuous integration most CI experts including me would say that once a day is not really enough it's a bit too slow to build great software in small steps I want reassurance that each tiny step works as I take it so we need fast feedback on our designs and we want to be a confident in our changes for this we need a deployment pipeline the first stage of the deployment pipeline is called the commit stage it's optimized to give Fast clear feedback and is essentially just continuous integration I usually recommend aiming for feedback from the commit stage in under five minutes this stage should also provide us with a high level of confidence that if all the tests here pass then only any other tests that we choose to run later are going to pass too I generally suggest that you aim for 80 confidence that this is true which can easily be measured in your pipeline if you want to by checking how often tests in the commit stage fail compared to in the rest of the pipeline once these commit tests pass then the next thing we want to know is is my software Deployable so we deploy exactly the code that we plan to deploy into production into an acceptance test environment using the same tools to configure and deploy it as we will use when we deploy into production so if this all works we know that it's going to work when we deploy to production too next we want to know if our change is releasable so we check everything that determines the release ability of our system during the acceptance testing phase we can use the executable specifications that we created at the start to verify that our code does what our users want it to do we may also want to test the security of our system its performance resilience or Regulatory Compliance and so on to ensure that we get fast accurate feedback we can automate all of these things in our deployment Pipeline and treat it as being definitive for determining the release ability of our system that is if all of the tests in the pipeline pass there's no more work to do before we're happy to release this change into production if we've been careful in how we Define our executable specifications and have kept the clear distinction between what our system does and how it does it then these specifications also document the function of our production system and they're human readable as a side effect of the user Focus so we can automate the generation of things like release notes or support documentation from these specifications if we ensure that the deployment pipeline is the only route to production for any changes then it also represents the perfect audit Trail for those changes if we capture who came up with a story who committed against it we can decorate each change with a description of all of the tests that ran and passed to Define its release ability and maybe recalled who pushed the button at the end to release the change into production this is by far the best way to work in regulated Industries but even for other kinds of Industries it gives us a wealth of information that helps us to more quickly and efficiently home in on any problems that crop up all this may sound too good to be true but it isn't what I've just described here is how continuous delivery Works in many companies all around the world using all sorts of Technologies all sorts of Industries and at every scale think about your development process and think about the things that stop you working in this ideal way then figure out how to stop doing those things that get in your way because as far as we can tell this is possible for every type of software thank you very much for watching and if you enjoy my stuff please consider supporting our work on this channel by joining our patreon community there's links to that in the description below too thank you [Music]

Channel: Continuous Delivery

Views: 65,282

Keywords: powerful software development, software development process, what is development process, best development process, sdlc, software development life cycle, google software engineering, amazon software engineering, netflix software engineer, computer science, software engineering, software development, Dave Farley, continuous delivery

Id: nCuDrWxlh4Y

Length: 19min 50sec (1190 seconds)

Published: Wed Jan 18 2023