Martin Fowler – What Does Tech Excellence Look Like? | TW Live Australia 2016

so my femur really is to talk about setting up a technical excellent software development organization and this matters because all of this stuff we're talking about digital transformation and all the rest of it doesn't really make any difference unless you can actually deliver technology get it into production and get things moving but when we look at what a technical excellent organization is about it is important to think about the why why is it important and I was thinking about how to approach this for the course of this discussion and when I mind drifted to very document that was written about a year or so ago the state of DevOps report which contained this rather striking quote so it all sounds very impressive but when I read something like that in a document I have an immediate reaction but in this case I had a reason to counter that reaction and that was because I knew one of the authors of the report this guy here is former fort worker Jess humble maybe some of you have heard him give a talk about continuous delivery he's not the kind of person who makes statements that sound like they come from one of our least favorite consulting organizations so I gave him a call and I said so jazz what you've making a rare a strong statement here about IT performance and business performance where does this come from is this just the usual marketing crap or do you actually have some substance behind this and he said yeah we've actually got some substance behind it and he set up a call with himself and myself and this lady Nicole Vaz grun who is really the kind of the person who focuses on the finely statistical analysis behind much of this and she led me through what was behind that statement and it begins with something in the business and financial schools which is this rather obscurely named paper and I defy you to figure out what on earth that paper was about from the title but what it generated that was interesting was what they call a latent construct a latent construct of organizational performance so what do I mean by latent construct essentially what they do they get a bunch of experts in a particular field in this case fine and business finance and they say give me a bunch of measures to say what a good organizational performance might look like and they asked for butter now maybe half a dozen or so of these things then they look at these measures and they apply them to a whole bunch of companies and they look to see if they correlate together into a cluster some of them will some of them probably won't but if they get three or four of them correlating into a cluster they say okay that's an interesting construct in itself that's what they call the latent construct of organizational performance and that's what this paper was about and perhaps more importantly to someone like me it correlated to the return on assets of an organization so I might not understand what the organizational performance might be and I honestly have not read the paper to dig into the details but I do know that return on assets is a kind of good thing to have now what they did with this state of DevOps report is that they came up with a number of these latent constructs one to do with IT performance and we'll talk about what that means in a moment but what's important as a kickoff is that it's another thing that correlates with organizational performance and return on assets now of course you'll notice of being careful to use the word correlate and it's important to raise to bring out an important piece of expert opinion on statistical correlation at this point we're not saying that high performance in IT causes a good organizational performance but we are saying they kind of go together so there's something interesting that's going on here but don't take it too far because you know correlation isn't causation so what is this aspect of IT performance how does it break down but basically they brought out three particular measures that they found did this clustering behavior how frequently you deploy into production what's the lead time between being able to having the software and putting it into production and mean time to recovery now those two first to the deployment frequency in the deployment lead time these are really quite interesting because they boil down to a very important measure that many people like we've been talking about for a while which is that of cycle time cycle time is how why is the period of time between somebody in the business world or in the domain that the software supporting having an idea of something they would like to see in software and how long does it take from that idea to form and then go into software running in production the faster you can cycle ideas into production that gives you an advantage in experimenting with things and being able to try things out and what the and the study indicated is that that cycle time had a direct correlation with how good overall an organization's performance was the other factor that where I brought up this mean time to recovery and that's actually quite interesting in itself there's a phrase that goes out there and comes from John all spor but mean time to recovery is more important than mean time between failures for most failures and the point being that by putting stuff into production it's more important that you can realize that something's gone wrong and fix it than it is to prevent the failures in the first place and that's not something that many organizations really understand they put a lot of effort into trying to prevent things go wrong but one of the things that we're learning is it it's more important to detect and fix failures than it is to prevent that at least the most of the time at least the non catastrophic cases so that's an important part of the picture is being able to recover fast but let's go back onto this cycle time cycle time as I've said is a very key measure and the reason I said that is important is it cuz it allows us to get early information about whether our ideas are worth pursuing or not and there's a nice little illustration I have that helps bring out the value of early information and early feedback about ideas and there's a little example around lottery tickets so imagine where that we have lottery tickets and some lottery the ticket consists of two numbers between 0 to 9 and you have a little scratch area for each number you pay $10 to the ticket and if the two numbers are both nines you get to win $900 so imagine that we've got Alice and Bob and Bobby's got a load of a hundred thousand of these lottery tickets and coming up to Alice and says would you buy these tickets let's assume the lotteries n't for somebody that you particularly care about it's something like the benevolent society for merchant bankers or something like that so treat it as a purely financial transaction if you are Alice how many people here would take that offer and buy those thousand tickets okay we can all do basic arithmetic that's that's a plus but imagine Bob's got AB it's more awkward of a situation here he's already bought the tickets at a reasonable discount um but he can't actually cash him in because you know laws and regulations and things he has to sell them analysis of course is sensible she's not buying so he thinks well what I need to do is to come up with another alternative let's drop the price drop the price to $8 so again you're advising Alice or you are Alice would you take the deal at eight dollars a ticket nobody okay I thought you could do math let's go through this you paying $8,000 for the 8,000 tickets a hundred of them on I've read sorry 10 of them on average you're going to win at $900 each that's $9,000 you gain $1,000 make sense how many would take the take that take it buy the tickets then oh I'm finally finally getting some people hit hands up you got to work with me a little bit on this because I want to take it forward a little bit and I know I mean I had a bunch of programmers go through this and they were a lot more enthusiastic about this than you were I'm pretty disappointed okay so we have a deal here potentially but then we have another offer somebody called Charlie's around and he also has a thousand tickets and the reasons are entirely to do with the fact that this is a made-up example Alice can only buy from Charlie or from Bob Charlie's deal is slightly different he says you take a lottery ticket if you you pay $5.00 and I'll scratch one number off then you look at the resulting ticket and you can buy it for $10 the full $10 so that means if you actually buy the whole ticket you'll have paid $15 right five to scratch-off one number and then if you choose to buy it $10 for the ticket as a whole so what I want you to do is to just go with your gut feeling here don't try to work out the math go with what your instinct tells you who's offered you think would be the better one to go for Bob's or Charlie's how many people think it would be Bob's like few people how many people think charlie okay Charlie fans keep your hands up how about if Bob drops his offer to $7 seventy seven dollars now not eight would you still still stick with Charlie you think oh but a lot of consistency how about six dollars few people drop down five dollars 50 okay those people your hands up you are really persistent if you do the math it works out like this you gain still $9,000 right because there's still 10 winning tickets on average but you pay 5,000 to see all of the tickets and scratch on number off but then you're only buying a hundred of them that have actually got the nine on there so you total payment is actually six thousand dollars so six dollars was a cut over point if you were Provera gating at six dollars you were right the difference is a free two one difference compared to the eight dollar price so it's not just a better deal it's a much better deal now I don't know how you reacted to that when I read this example it was in Dunn Reinertsen spoke about product development flow I was surprised my gut didn't tell me that it was going to be that big a difference that's the real question is did your gut tell you there was going to be that larger difference I was surprised and the re and that's the reason that we want to do fast cycle time is because we want this early information we want the ability to scratch that number first number off and see whether we're on the right path or not and it's actually worth paying more money than we would instinctively think to get that degree of information so that is the why that's why we want to highly efficient technical organization and what a technical organization looks like to the outside it's an organization that can give you rapid cycle time it allows you to take ideas quickly put them into production see whether they're working and then be able to pursue switch and do the rest of it so how do we what does that what do we need to have this one of the first things most obviously is what we call continuous delivery I mentioned Joe humbles book that talked about this in more detail he co-wrote it with Dave Farley um it's something we've been pursuing a long time at fort works it is all about the ability to build software in such a way that you're always able to put the current state of the software into production the crucial thing about continuous delivery is that it isn't the decision of the programming development team to say this software is able or not to go live it's purely a business decision that says do I think this gives me enough incremental shift but I can learn from it but it gives me value and of course learning is a very important part of that value it's a whole bunch of things behind that automatic deployment good version control things of that kind but the fundamental fact is that you're always in a state that you can go into production and that requires a lot of discipline from a team it's not an easy thing to do and we when if we've been in plenty of organizations they say well that's just not possible we've done it in enough places to know that it is but we've also done it enough places to know but it's not easy it takes a lot of effort to get to this point but it's hugely worth it because that makes a massive difference the cycle time so that's one of the key things that you want to see in a good technical organization but you're able to do continuous delivery another feature of a good organization that we're increasingly finding is the way that sets thin teams are set up and organized you need small teams that are able to collaborate very tightly together but they need to be business focused you may have heard the phrase tossed around a lot from Amazon about two Pizza teams right don't have any team that can't be fed by two pizzas of course this is America and only you've seen exactly how big an American pizza can be it's pretty frightening but we're still talking of teams of a dozen people not teams of 50 or 100 people now the two Pizza term is commonly tossed around but there's another part to the Amazon notion of two pizza teams that often isn't mentioned and that is that they should all have a path through to the customer as you'll all be customer-facing and that is an important feature it's not I think all the way through for all organizations but it is a very important element but there should always be a business function there that's driving things and there should be a real business connectivity with all of those teams and this is an important shift a lot of organizations try to separate teams by areas of functionality and we'll you'll hear me moan a bit more about that later on the vital thing those are focused along business lines we've heard earlier that there should be no silos in organizations everybody believe we should have no silos yeah get a few hands up you're not very good at putting your hands up are you in this group if you're saying no silos you're living in a world of false optimism humans naturally silo they will we that's what we do we get along with groups of people the question is not you can't avoid silos but what you can do is to try and organize the silos in such a way that they are most beneficial at least least harmful and the way that we find that we do that is by focusing on business oriented teams also teams that have a relatively long life connection with that business group I've come across some organizations where they say well we do a project we have six months we assemble a team and then they all go back into the pool and then we do another project the problem is there's no long-term coherence between the people working on the team and either the business function or the code that they produce and that leads to poor quality code and a lack of understanding about how the underlying business functions for so you want them to be business oriented and relatively stable when the last point is you want them to be autonomous you want them to be able to have within the team everything they need in order to be able to deliver useful software to the business and that means that they have the ownership not just of the user interface and user interaction but all the way through into the backend ever they have the connections to the business people there's nothing more frustrating than a project I visited not that long ago where the work we were working for a large client and they were doing some fairly sophisticated analytical work for a group of business users an internal IT group and I asked her how many people are actually the users of this system and they said about half a dozen people which is not unusual when you're talking about product analysis and matching and purchasing and things of that kind you have a small group the sad thing was that they had never spoken to any of that half a dozen people in fact they were forbidden to speak into any of those half a dozen people instead they had to talk to a product manager who was actually managing things and that had for product managers over the course of six months I mean to me that is craziness basically every person in us in one of these software development teams every developer who writes code should be on first-name terms with one of their users and should be able to talk to them and find out what's going on that's that business orientation again and the autonomy as well of a team to be able to go through come up with ideas make this make movement the next thing is very important in organizing teams is a very touchy-feely thing the team has got to be able to trust each other trust each other so that they can share knowledge but they can share problems that people aren't afraid to ask for help but we have an integrity within the group that if somebody says what they believe and what they want to do but you actually trust that that's actually what they mean there's no hidden agendas going on this is a really often overlooked thing but a very crucial think it is something that's gradually beginning to surface more there was an hbr thing not that long ago that bring in to highlight the dangers of toxic people and it was more important to get rid of toxic people but it was to you know hire the best people whatever that means there's also a lot I've got some references down the nerves of reports from Google where Google went all across their organization they do have small teams uh and they do have relatively business oriented teams I do the things I was talking about in the previous slide and they were looking for commonalities and I found the team's varied hugely some people like to socialize after work some people didn't some people had a very strong leader who kind of ran things and other teams didn't they were looking for common elements of successful teams this was the thing that they found was a common element if people could trust each other within the team and it was a healthy psychological safety then that allowed the team to excel this is so easy for people to overlook I don't know how many organizations I've seen but we'll ignore this factor and yet it is crucial so don't forget the importance of this and that the team should also to a certain degree beach technology led you a comment made from one person I was talking about with this he says that it's not that the leader of the team should necessarily be programming every day themselves but they should be able to they have that technological and they understand what's going on within the software that's being built and that doesn't necessarily mean that they're the project manager but they are the leader the project manager and leader isn't necessarily the same thing the product owner and leader isn't necessarily the same thing and that leads to some important consequences that recognition comes from your peers it comes from a value of continuous learning there and also that any purchases of software and tools and equipment is done by people with that people who are going to use it they're not done as so often in companies they're done because somebody's done a good stick and gone out on the golf course and sold them a nice product but actually isn't usable for the people on the ground and so the result is that within the team what you need to combine is software development and business orientation but with a value of each other and a communication through the short cycles of a rapid delivery of software the the analogy I like to think about it is that trying to deliver software in this kind of way it's like having a vehicle that needs to do a long journey it could be a car it could be a person it could be a horse and you're traveling over many days and weeks in that situation the you need to be careful of the health of the vehicle because if you you say oh I've got to go as fast as possible and you try to do that when we're on a on yourself or in a horse you're just going to break down even the car will break down if you just say oh I'm going to drive and drive and harder not stop to change the oil alright so the software development function is in many ways concerned about that health of the of the vehicle and the vehicle in this case is the codebase the quality of the code is very important and also the health of the team of a team interacting well with each other are they learning etc etc what the business operator provides is a direction to travel because you can have the healthiest vehicle in the world but if you go in the wrong way it's not going to help you but you need to combine both you need to combine both the setting of the direction and the health of the vehicle if you're going to succeed and the key the COG that makes all this turn rapidly is that short cycles of delivering you business valuables software that's the crucial thing that makes it all go so with that second part I've given you a sense of perhaps what a well-functioning technical organization looks like derived from the conversations I've had with my colleagues and contacts around the industry but for people in large enterprises that's often seems like a big leap to get to because you've got a big group of people and the question is can we really do this throughout a large organization and the answer is almost certainly not because it's just so much to do you can't change a big organization all at once it often has to be changed in pieces and so when looking at this change one of the things that again comes out from talking to people is we really have to figure out how to prioritize and the prioritization comes from saying to ourselves what are the strategic pieces of software and what are the utility bits by strategic I really mean how does it affect your differentiation in your business is it something that's vital to how you do things better a utility example might be your payroll systems for instance you probably aren't getting any strategic advantage over your competitors by having a better payroll system than them so with the utility system what are you concerned about you're so concerned about keeping the costs down typically and you'll want to avoid any disasters all right so little things going wrong you know or also but if you miss payroll completely that's probably a bad thing so that's the balance that's your criteria there but in the strategic software where you're talking about how do I compete with my competitors then it's a whole different thing you want speed you want to get that cycle time I'm you want inventiveness to say let's come up with ideas put them into production get there before our competition does cost well actually it shouldn't matter that much because if cost is a factor that means you're not generating enough value that implies that what you're doing isn't really strategic anyway if you're in a strategic space value should overwhelm the cost and you should focus on how do you generate value rapidly not how you reduce costs and that's a very important distinction so where you want your technical excellence obviously is in your strategic part not in your utility part and that also implies something else it implies that you need to manage and control software projects done in strategic areas very differently from how you do utility areas because you measures of success are different the things that you're striving to achieve are different and the lot of organizations most of their software development tends to be looked at with a utility mindset and it really implies you've got a shift and start thinking about a strategic one for some areas of your software I mean ideally it would be wonderful if everything could be done that way but reality you have to prioritize now as I talk about two different ways of managing software development projects you might be thinking all that sounds familiar having a hired people talk about two-speed IT from one well-known consulting organization Gartner calls this bipolar IT sorry now bimodal IT not just checking you're listening to me now there is a degree of similarity in which we're talking about two different modes of managing software and similar to the McKinsey one my division is bit arbitrary to rather than n the important thing is the difference rather than the number but the big difference between the two is however divided up the McKinsey approach talks about customer centric front-ends and transaction-oriented backends similarly you hear people talk about systems of engagement and systems of record those terms sounding familiar to you you can nod you know if you're awake enough to be able to do that notice that what they're doing here is they're talking about software but when we talk about strategic versus utility we're talking about business function the distinction is not whether something's customer centric all transaction oriented it's about is it helping my differentiation and this is really important because I believe that the bimodal IT idea is actually going to lead people into a lot of trouble because if you're working in a strategic business function to try and get your software further forwards you can't just focus on the front end it has to focus on the back end as well if you're doing anything significantly important with your business cycles it's not just putting a pretty web web UI you've actually got to make back-end changes you've got to change the way you're doing your stocking of stores you've got to change the characteristics by which you classify your airline seat so that you can sell them using the different parameters to what your competition are doing all of these things require backend change as well as front-end change and we've seen problems of organisations that say oh it's only on the front end and then they can't actually follow through properly with a strategic change because all the backend systems are in some completely different process that geared around avoiding disaster and keeping costs low and then you lose your speed and you cannot come up with inventive ideas and put them through so be very wary of the whole bimodal think it's it is a good idea to think of different ways of running organizations according to different rules but it shouldn't be based on from them versus back-end there's another very important point here as well when you hear people talk about two-speed IT they say well your customer centered systems they go fast and because you want less speed and flexibility you're inherently going to get lots of errors and that's why you don't want to do this we've your back-end systems because you need high quality and therefore you need to double triple check everything can be really slow there are two problems with this the first problem is well I'm not convinced early to get out a good idea to have lots of errors in your customer facing software I mean you can tick off your customers it's hard to tell perhaps when you're losing customers because they're really ticked off because of your user interface but they'll be gone and they're hard to recover losing customers can be very expensive but the real problem I have with this is that if you talk to people who know software development they'll tell you well that analysis is fundamentally flawed if you want speed and flexibility you've got to have high quality if you've got lots of bugs in your software that's going to slow you down if you don't pay attention to the way the software is constructed somebody to nicely modularized and clear and easy to understand it's going to slow you down what we found which was surprised a lot of people from the early days the way we started using agile software development techniques and that was quite a while ago now time and time again I'll go in to visit a team and to talk about what they were doing and say well because we're using these agile fast speed flexible techniques we're finding that we have an order of magnitude less bugs in production than is the typical for the organization these kinds of things that I'm talking about continuous delivery the whole agile mindset they actually lower burg counts you actually get more reliability this way and so that I think is really the crucial point here but don't expect that quality means slow speed it's a natural thought quality means expensive quality means slow but it's actually not the case it's one of those counterintuitive things like scratching off the ticket how powerful the information that is so that's how I want to finish I want to come back to this point we need this combination of thinking between software development and business communicating through short cycles focus primarily in the strategic areas of the business and then within that strategic area a full-stack thinking so that we have a fast vehicle going the right directions to the right kind of places and that's me done

Channel: Thoughtworks

Views: 67,808

Rating: 4.906137 out of 5

Keywords: Thoughtworks, Technology, Business, IT, Consulting, Programming, martin, fowler, agile, tech, excellence, australia

Id: Avs70dZ3Vlk

Length: 31min 5sec (1865 seconds)

Published: Thu Oct 13 2016

Reddit Comments

[u/chronodekar]

My takeaway is that you need to organize IT around business features as opposed to frontend vs backend. It makes a lot of sense. To have any meaningful change to the frontend requires some kind of backend change and if changing the backend requires a considerable effort, it reduces motivation for developers (and managers who cannot understand why a feature is taking so long to push out). Good video, well explained Martin! :)