Martin Fowler – Continuous Delivery

for the second talk I'm going to move on to one of these technical practices that I mentioned in there and that of continuous delivery it's one Verte has been one of the most interesting ones that i've seen develop over the last decade or so when we wrote the agile manifesto back in 2001 continuous delivery is actually written in the first principle which is part of why that name was picked at his practice we had this notion that we should be able to constantly deliver software rapidly but it was something that's very hard to achieve had been achieved in some places Kent Beck and his team was doing this in 1998 in Switzerland they had a very small team they were working for this very rich insurance company this Switzerland after all and they had a spoon they were using small talk so we're cheating by using the best technology ever anyway but it was doable but in most places it is much much harder what do we really mean by this what does continuous delivery look like well we start with a programmer and an existing body of software I never know how to visualise software so I go for cogs we also what can I do and the developer needs to make some kind of little change and some all change that then has to be integrated in the software to produce the new version of the software we all do this all the time but then the question is how good is that change you know you spend a few hours making a simple little change to an application works on your machine how happy are you that that's really done because in many organizations that's done right the developer says yeah I'm done works on my machine you say ok maybe he hit you maybe they develop a hit f5 to check things out maybe not who knows right we call to continuous delivery saying well now now this isn't done at all we need to prove that this software really is working effectively so at the heart of aces we do we basically subject this change this new piece of change software to a series of more and more difficult tests first tests are easy now does the thing even compile does it actually build on a server development server rather than just somebody's workstation that's enough to fail a lot of cases right you've seen cases where that's not worked but then we begin to throw in more and more tests low-level unit tests higher level integration and functional tests maybe performance tests that say you know has as it has the developer done something that's actually going to slow the application down significantly and introduce a performance hotspot a series of these things until we can be confident that this thing is good but it's green we call this series of thing of tests a deployment pipeline it's usually can be just a single stage for small projects for longer projects it's a multi-step process and it goes through environments that get closer and closer to a real production environment ideally that last step has to be on as close a mirror of the production environment as you can get so you can be confident that if you flip the switch and put it into production then everything's going to be good and your users are going to be happy with the new feature continuous delivery is about making sure that every little change your developer makes can go through this process and you can confidently send it into production at this point it's it I need to make a distinction between two very similar words continuous delivering continuous deployment continuous deployment you probably heard a fair bit about companies such as Etsy or Facebook talk about this about how every change a developer makes gets deployed to production and these things happen every day you know you might see some little page I think Flickr was one of the first to do this but to say now we deployed 15 changes to production since this morning and this kind of the bigger a lot of these websites do this kind of stuff continuous deployment is about taking all of those changes and just putting them through and putting them out into production continuous delivery is slightly broader in what it says is that we want to get to the point where we can do this but we may choose not to because in the end that decision about whether to take every little change and put it out of production is a business decision not a technical decision so it may be that the business people say we don't want that degree of churn and change in the actual application so we'll do it every two weeks but the technical people are still able to deploy every change so if they change their mind and said oh I know deployment isn't until next week but this is a really nice new feature can you put it in put into production right now all the development team has to say yep fine tap-tap tap-tap okay it's live you want that degree of confidence and so in order to do continuous deployment you have to be at you have to be doing continuous delivery because that's what gives you the confidence to actually do it but the choice is still then become a business choice and it may be other practical reasons as well I mean one organization that does continue but took on continuous delivery and we're very happy with it was HP's LaserJet outfit and of course they can't put necessarily deploy all the time right because they've got to update firmware on machines that you know not connected necessarily to anything but they still went through that same kind of process so how do you pull this off how I mean I've talked vaguely about the pipeline what are some of the key things you need to get involved and do this well at the start of it is automating everything that you can think of that can be automated so obviously the build has got to be completely automated you've got to be have a complete executable built from scratch in a completely automated way that's increasingly common now but certainly 10 years ago I remember hearing stories of y'all so there's a build person who gets you know files from one developer and files from another developer and figures out and it takes weeks now it should be a single command this also means deployment into any environment needs to be equally automated so I can deploy into a test environment I can deploy it into a staging that's a mirror product deploy into production there are all automatic steps and so hence you see a lot of effort around deployment automated deployment tools to sort this kind of stuff out also it means that even provisioning new machines needs to be done in an automated way so that you can do this quickly and of course a lot of reliance on automated tests I'm an extreme programming background extreme programmers are known for being really anal about automated tests and and for work C's has that habit as well we are very very heavy on tests and that's because we need to be able to ensure that we're confident to deploy each piece I should also say it doesn't necessarily mean all tests should be automated there is still an important role for exploratory manual testing but automation is kind of the heart of what makes this thing tick in order to be able to automate with this kind of effectively you need solid configuration management you always want to be able to know I can get to any version of this software and I can put it through the pipeline and pull it out and any that includes any past version of the software so this means everything has to be kept in configuration management in well-known places not just the source code which is fortunately most people now put in proper configuration management but also things like database schemas deployment scripts all the necessary stuff to configure servers absolutely everything has to go on the version control because if something breaks in production you may not know right away it may be a process something that's only done every couple of weeks then when something's broken you've got to be able to quickly figure out what change caused the break the DevOps term comes into play here I think of DevOps as a cultural form anuman it's really saying that application developers have to be aware of operational needs and take them into account when building application and it means that operations people have to think how can we smooth rapid deployments how can we make it be less of an event and that it happens by the two groups collaborating so there's a lot of collaboration between application development and the operations site they're no longer seen as separate walls I mean some places that have taken on this whole DevOps thinking of seem to make things works by introducing a defect DevOps team to sort of introduce extra boundaries and burdens between getting between developers and operations but the heart of it is this communication and cultural shift and it's a cultural shift on both sides and because traditionally application people have not spent enough time kin concerned about the operational consequences of what they do and when the last major ingredient is continuous integration but every developer integrates with everybody else's work frequently how many people here do continuous integration instead of interest actually keep your hands up does everybody on your team commit to the mainline trunk at least once a day if so you may keep your hand up okay do you have a solid battery of tests of it you're confident that you can find any bug before it went into production if so keep your hand up and the last test is if you get a production if you get a production failure and you've got your pipeline breaks do you fix it within 10 20 minutes okay those are our hands up consider doing continuous integration and that's the important point all right everybody commits every day to mainline you have the tests to make sure you're confident that everything is okay and if they fail it's if nobody has a more important job than fixing the the breakage that's the way Kent phrases it very carefully that's what CI is about and that's required to get continuous delivery going that test of free questions by the way comes from Jess humble he does it every time he gives a talk like this he says and that's always what happens lots of people the hands up at the beginning hardly anybody buddy yet so it's not unusual so why do we think continuous delivery is worth bothering with what are the benefits of continuous delivery I focus on three and the first of these lowering risk if you've got a large body of work that needs to go into production there's lots of potential things that can go wrong you shrink that change down and make it very small there's much less to go wrong also should something go wrong it's usually easier to identify and fix I mean the simplest fix is you roll back to the previous version you should always be ready to do that anytime something goes wrong you're monitoring you suddenly see oh the amount of new orders has dropped unusually roll back I mean you should always be able to do that rapidly and but the point is with smaller things it's easier to find a problem the example I gave how about something that you don't know it's broke until two weeks later because it's only something that happens every two weeks well if you've got configuration management and lots of little changes you can fairly rapidly isolate where the problem is and say oh there's this particular change of the curd three days ago that introduced the bug and you've got much less code to look at you can figure out how to fix it those small changes lower risk and it sounds counterintuitive because you think well if I'm making 10 production updates every day isn't that going to make things more risky well no because there are lot smaller it's it's one of the little adages of agile thinking is if it hurts do it more often because what you'll figure out is by making smaller changes you're able to lower the pain and that's the one of the big surprises for people who do continuous integration it sounds very scary to say everybody's committing to main line every day and all this kind of thing but when you do it you begin to realize integration goes away as a problem because each piece is so small and you're doing it so frequently it doesn't matter anymore and that's in fact how continuous deployment works for a lot of organizations they just don't think about deployment issues anymore because they're forced to practice on doing it frequently and they keep the changes small so that's the first reason lowering deployment risk the second reason has to do with this notion of done I mean we've all seen project charts like this right where you've got a scope target this is the classic scrubber chart backlog all going well so you look at a chart like that you look at the predicted line you're feeling good but then what does done mean is if a developer saying hey I'm done with this then how happy do you feel about that chart and what it's telling you not very you me however how do you feel if they say now it's deployed life each of L steps is a live deployment then you feel a lot more confident so the second reason the continuous delivery is it gives you the sense of real progress even if you're not deploying live but you're deploying into as much of a replica of the live environment it's not perfect but it's still a hell of a lot better than developers saying yeah I finished up piece of coding so the second reason continuous development is you know what progress you're making the last reason is more a thing from continuous deployment but you don't get so much from just the delivery thing if you're not deploying frequently enough but we know that when we put stuff into production and we get actual users working with a system we're going to get all sorts of interesting things that come out of it some users don't understand our beautiful UX changes and say what the hell's going on sometimes I didn't want that you've taken away something important or whatever we learn by putting stuff into production we know we can ask people to we're blue in the face what would you like but it's only when you actually give people something they will tell you what they don't want we've known that that's been an inevitable feature of software development and the plan driven folks sort of desperately tried to predict and not have this loop but now we're learning to take advantage of a loop and so let's get something anything out there and then see what people do don't listen to what people say they want but actually monitor the software see what they're doing and then use that to decide and what we do next and that you need continuous delivery to be able to pull off so you can get that kind of learning loop into place by taking that user feedback and learning from it now continuous delivery isn't an easy practice it does take months to get it into place and to be able to do it well but these three benefits give you a lot of feedback a lot of positive return on that investment and that's something that I've seen when I started with thought works and four works began in serger all stuff back in 2000 we started with continuous integration but it took a long time to get to the point where we could do continuous delivery on projects because there's just so many steps in in enterprises to be able to get from working in a lab to actually in a production like environment but it's been one of the biggest benefits I think that we've seen and we've learned a great deal from doing this and it very much recommend it if you want more the canonical thing for this is the book by Jess humble and Dave Farley which goes into a lot of detail about continuous delivery there's a page on my website as well where I at least outline some of the things I've just talked about but this probably is one of the most critical practices critical hearts have actually be able to really get that two star fluency and agile thinking so that's the second talk you you

Channel: Thoughtworks

Views: 110,347

Keywords: Thoughtworks, Continuous Delivery, Computer Science (Field Of Study), Martin Fowler (Computer Scientist)

Id: aoMfbgF2D_4

Length: 17min 6sec (1026 seconds)

Published: Sat Jan 31 2015

Reddit Comments

[u/nakilon]

OMG, why the hell he calls Continuous Integration a Continuous Delivery and Continuous Delivery a Continuous Deployment?

He sounds like a manager who understand nothing in the shit he talks about. He does not explain why exactly you divide the pipeline in steps because he does not know.

And you don't "go closer and closer to production env" -- you should better start from manual acceptance testing and then go lower and lower in directions of those modules of software that tend to have errors or debugging difficulty. Otherwise you can spend a year of testing a module or a source file that will never fail. Also low-level unittests are not the best way to start with because your CEO understands nothing in their results.

But he tries to make an impression (also dressing like Steve Jobs) of a guy who knows shit.
Very weird and stupid stuff. He's so annoying and probably unpleasant to work with, that I even can't force myself to watch/listen more than 40% of this video.

UPD: checked who he is on Wikipedia. WTF, did he went crazy to say all this stuff after this kinda huge experience?