Extreme Programming 20 years later by Kent Beck

my name is Kent Beck I'm a programmer coach of Facebook and when I got the invitation to come speak I was I I asked questions about well what what would you be interested in hearing about know just to tell us some stories about extreme programming so I had no no really really what would you what is the audience want to hear about what just tells some stories about extreme programming so since I didn't get a decent answer to that question I'm going to tell you some stories about extreme programming it is a little sobering to think that this is 20 years later that the evolution of an idea would take that long as a young programmer just coming out of college I would never have believed that anything could last that long at all much less that that's just the beginning of a trajectory of a set of ideas so I can appreciate the timescales now and especially when I compare what we're doing to what's happened in the lean manufacturing world where the timescales are more like 50 years you know 20 years is a blink of an eye just not if you're raising children so the here's the outline now the primary rule of storytelling has always do things in threes so I didn't do that I figured I'd break the the talking two to two sections the first one is and I'm glad that I saw another one in the previous talk is just a timeline what happened over those 20 years and a little bit of prehistory as well and then I'm going to do something that I'm very suspicious of as a as an audience member this kind of lessons learned which usually strikes me as a heavy with rationalization a kind of aren't I smart to have figured all of this out you know and then you sprinkle in some well here's what I would do differently you know proving that I'm even smarter now than I was before so I'm always very suspicious of that but I'm gonna give it a try and we'll see what happens so here's the here's a timeline of my experience with with extreme programming now if you have questions about any of this I'm going to leave some time at the end feel free to ask them I'm going to give you the short versions of all of these stories or skip some of them altogether so my experience with programming methodologies started the first day of classes as a computer science student at the University of Oregon the head of the department in the very first class stood up and gave a lecture about structured top-down programming where programmers of the future will will take a concise precise definition of the problem and break it down into code in in such great detail and discipline that in the in the career of a programmer of the future you might be able to point to four or five errors that you made ever and I walked out of that lecture aghast I'm here I'm 18 years old I've been programming for probably six or eight years at that point and I just thought oh this cannot be the future this doesn't sound like any fun I'm going to change the way that people do that now that seems pretty arrogant for an eighteen year old to say something like that and since then I can reassure you have gotten even older but that's really where it started I knew that I was looking for something different than than what was what was out there my first job coming out of school I was lucky enough to use a programming language called small talk which was very different than the other languages that were available at that time I was able to experience programming as a as a learning experience it's not a production it's not sitting there and cranking out widgets it was a learning experience an exploratory experience an intensely creative act something that was intellectually and emotionally absorbing something that was a profoundly social experience as well and that gave me a hint of what it was that I wanted to encourage on a much larger scale small talk was great for two people sitting at a computer it had troubles of those scaling up at some point there I went to independent as a consultant and I began to recognize the value of automated tests in the programming experience I was going to a client in Chicago and I can remember I knew this was like on a Sunday midday Sunday I'm thinking I have to fly out and I'm going to tell them that they need to write automated tests but I don't know how to tell them to write the tests so before my plane leaves could I come up with a tool that would let programmers write tests in a way that programmers are familiar with capable of using and two or three hours later I had the very first version of this xunit framework which has now been copied and used by lots of people all over the place and took it to the clients and in good consultant's fashion said so you guys need to write some tests and and of course here's how you do it they said oh we use this tool yes is it okay we'll use this tool you know yeah absolutely so gave me another lesson in the value of confidence having invented it you know twelve hours before that and it was a at about that time once I got used to having automated tests that I remembered a book that I'd read as a kid my dad was a programmer too grew up in the Silicon Valley you know geek brat my dad brought home calculators and stuff for me to play with he'd bring home books about programming and so I read this book it said here's how you write programs you take the input tape now when I'm speaking to younger programmers I have to explain that an input tape is like a long thin floppy disk and and a floppy disk is is like a spinning solid-state anyway but you guys get this so you take this input tape and you type in the output tape that you expect this to to create this is when I T was a tape to tape activity and you'd have these long strings of programs and each program would convert some input into some output and then that would be fed into the next program so you take a concrete input tape you type the output tape that you expect and then you program until when you feed the input tape through the program you get exactly the matching output tape you expect it so I had this shiny new testing framework and I thought let's let's see so if I wanted to follow that same model the first thing I would do is not write a program and then write a test for it I would write a test for this program that doesn't even exist yet now this is a stupid idea right clearly you know the test is going to fail it won't even compile because you haven't even written the program yet so how useful could that be so following my you we'll strategy if an idea is really stupid it's it's definitely worth trying if it an idea is good and it turns out to be true somebody else will have done it if an idea is stupid you have a chance that nobody else is dumb enough to try it then if it happens to work you really have something oh how do you think of these great ideas it's cuz they're not great ideas when I think of them seems straightforward so anyway I thought okay so I remembered this computer book that I'd read it and I used to read the books my dad would bring home I just read them over and over and I didn't understand anything I was I was that kind of kid which you may not have encountered before but I thought let me just try it so I just tried it with the stack as my first example push pop push two things pop - and I just I'd write one little test I'd make it work I wrote the next test I'd only program just enough to make the next test work and the next one and the next end nah I can't think of any more tests that could possibly fail and it was like cheating that that moment that heroic moment as a programmer when things are about to get out of control and then through the sheer force of your intellect and your will you pull the order out of chaos that moment was gone instead I just said oh now the program needs to do this okay it does it and it needs to do that and it does it and used to do that and it does it needs to do that and by implication I can't think of any other inputs that would cause this program to break oh okay so that's how TDD test-driven development this this odd style can came about for me in 1996 March of 96 I was invited to a project at Chrysler called c3 it was year 2000 project to replace a very complicated payroll system payroll is a simple problem until you get the employees the unions and governments involved at which point it's no longer a problem they had some performance problems with the wit they were trying to use small talk derived technology which wasn't very mature at that time and they had some performance problems so I got there and I said well where are the tests by this time I was a testing guy where are the tests that will prove when I make things go faster that I haven't broken anything I said actually the code doesn't really work yet and I said I can make it go really fast if it doesn't need to work and they didn't think that was funny kind of like you guys just now and so I realized very quickly that there was more going on here and I found myself at the end of the week in the office of sue anger who was then CIO of Chrysler so by far the most powerful person that I'd ever talked to face to face and I said sue you got three options that I can see you can keep going like you're going this is a two-year project that had been six months from shipping for a year and a half yeah you've been on that one too so you can keep going and nobody's happy about that you can just cancel the project and outsources or find somebody else to do it I said I think that's a shame because you have some very good people here really understand the problem that they're trying to solve or you can give everybody a week off and throw all the code away and start over and I think you'll be successful and she said I'll take option three and you're in charge I said you don't understand I'm a consultant I don't do in charge she said you don't understand I'm su longer the CIO of Chrysler you're in charge when I got on the airplane to fly home that night my hands were shaking so bad I could not drink my whiskey what have I gotten myself into and then I realized hey I'm in a absolute no-lose situation I'm gonna get paid to do this if it's a failure was already a failure in physics if it's a success on my hero so I don't have anything to lose so I went back and the this is the the birth of extreme programming I went back to the team a week later everybody had had a week's vacation and they really needed it and being Americans the I sat down with the project manager I said okay well introduce yourself tell me what you do on the project and then I'll tell you how the project's going to run so he said okay my name is is Steve on number I'm the project manager and I did this I do this in this in this said okay so how's this project going to run and I said okay so we're going to structure the project in three week increments and at the end of three weeks we will have enter it will have implemented a few more stories and we'll have a demo for the users and then show them the progress we've made and then in the next three weeks we'll implement some more stores so okay that sounds that sounds good next person comes in introductions how's this project going to run I said so we got these three weekend increments and and we'll implement some stories in each one and each story will have some some acceptance tests that will prove that the story's been completed Oh next person comes in hi so and so I said okay so we got these three leaking increments and we got the stories and the tests and we're going to do this hair programming stuff oh okay okay by the end of the day I had formulated extreme program it was completely improvised but carefully prepared inspiration is a preparation plus panic and and neither alone is sufficient but when you put them together in just the right moments I take in everything that I learned about people writing software together and explained okay here's how we're going to write the software now many of the decisions that I made in the heat of the moment that seemed so insane to me like having running software every three weeks you know that's that's passe now there's there's no way anybody would iterate that slowly at this point I open didn't just offend anyone that didn't need offending but the the basics were really there that was the structure that the software was always going to be running that we would measure progress by how much more of the of the input we could run correctly but we could always do something we would always be working end to end and that was really the the birth of extreme programming and I have my my notebooks where I was trying to figure out the name for it and there it is April something rather extreme programming because I would tell my friends about okay well I'm running this big IT project and it's crazy and they there's got these iterations and and oh that sounds good and I got tired of saying this the way we run this project so I had to give it a name and that's that's what I came up and I'll talk about that a little bit more later but that's that's that another milestone on on my about the software development was the development of j-unit so I'd written this testing framework in one language and then we moved to Switzerland in 97 and my friend Erich gamma and I were flying to a conference back in the States I wanted to learn Java which was brand new stuff at that time and he wanted to learn about the testing stuff so I said well let's build this testing framework in Java and because we're programmers we're going to write tests for the testing framework in itself as we go along so we're going to develop this framework one test at a time all the while the whole thing's testing itself it's a kind of a macho thing to do on an airplane and by the time we landed though we had the first version of j-unit and fortunately my friend Martin Fowler was at the conference and we handed it to him it was just like two guys for four hours on an airplane until the batteries ran out right how important but we gave it to Martin and Martin came back and said this is fantastic other people need to see this and and j-unit spread from there and then that architecture has been copied many many different times something that goes on with je-yoon that I think people miss is the extent to which it's a political statement it's not a piece of technology primarily as a piece of technology most programmers could write the equivalent functionality in a few hours but the making the political statement as a programmer I accept responsibility for the quality of my work at that time was a revolution and so j-unit yeah it's it's this little framework but much more importantly it's a symbol of the acceptance of responsibility there's not okay I'm a programmer and I've made it good enough and now I'll give it to the QA people who will make sure that it's really good enough which it's not yet says no I'm the programmer I'm responsible for the quality of what I'm doing and and that was really a revolution at that time and it's a piece of that junit philosophy that I think is kind of missing while we were in in living in Switzerland I wrote the first edition of extreme programming explained mostly on the train between Zurich and Munich turns out that's a really good stretch of railway - right - because I can just kind of I could just kind of stare out the window type let my thoughts flow there was interesting stuff outside but not so interesting and so I get a couple of hours in and most of the first draft of it was actually written on that on that train so that was a very popular book by technical book standards when it came out which is a little like saying a very skilled football club by American standards but it did shake things up a bit to say all of this all this stuff around programming that's taken on such importance this document and that document in this phase and that meeting at the end of the day you have a program where you don't have a program and if something you're doing supports the program then it's worth keeping and if it doesn't it's worth considering dropping it and and that mind shift was was tectonic for for some people to me it just made perfect sense and I didn't understand why everybody wasn't already working that way and I still feel that same way but that's my problem any of yours up two years later I was flying to an extreme programming conference and I don't remember who recommended the book but I had a copy of Toyota Production system and I can keenly remember being seated in business class a flying transatlantic and opening this book up and it wasn't very many minutes before I sounded like someone at a religious revival meeting oh yeah uh-huh oh yeah mmm the entire business class section is staring at me I I'm oblivious I didn't like I sort of registered their reactions but not so much by the way the same thing happened last night in the restaurant when I was telling this story but the the the resonance of these ideas to my experience just thinking of over-production as a problem too rocked my world and the and the water level metaphor just I've used that so many times that absolutely fundamentally changed the way I was thinking and the way I explained extreme programming a little bit later I was visiting a client in Seattle and the CFO was in the meetings it was a kind of startup kind of environment 100 sort of people and they were having problems and they wanted to try some of this extreme programming stuff and the CFO is sitting in the corner the entire morning three hours glare arms folded goes off to lunch he comes back with a huge smile he almost hugs me this being America and not France and said why didn't you tell me that this was lean for software well I kind of didn't know that he said I was at Boeing during the transition to lean manufacturing a Boeing I watched that transformation and that's exactly what we need here that was a an emphasis to me like oh yeah there really is a deep connection between what I'm doing and in the lean manufacturing world and that's when I began to take that really seriously there's the agile manifesto which was a meeting of 17 privileged white guys in a ski resort in Utah I can't tell you any stories about that because I was extremely sick I was taking some very powerful antibiotics and I was just in a daze the entire time fortunately however my name begins with a B so the publication is called Beck at Al but that's about the extent of my responsibility for for the contents of that in 95 or 96 we're asked to update extreme program it had been a successful book they wanted a second edition of it and I handed a copy to my wife Cynthia who's here today and said would you look through this I knew that she worked her way through college grading English papers and when it came back to me there wasn't okay there were two pages out of a hundred and sixty-two pages that didn't have a wash of ink on them comments this needs to change how can you say this if I'd known this was in the original book I wouldn't have let you publish it and so we began a process where we rewrote the entire thing from scratch there's two pages this driving metaphor where the you know you don't just point the car in the direction that it goes you're constantly making adjustments that one's remained consistent between the two editions but the rest of it we completely rewrote and we rewrote it together there's a big shift in own with the second edition the first edition is very programmer centric and I'm still prone to being programmer centric but it was a snotty disrespectful towards the other people involved okay there's programmers and there's some other people overhead you know you have to have in order to get paid to program but they don't matter that much users sure users but programmers and users and then the rest of it is something who cares don't need it and shockingly that alienated a large number of people most people who write checks actually get an alienated when you tell them that their jobs redundant so that wasn't a good business move one of many that I could recount my my career but that was a complete rewrite with the intention of being much more inclusive okay who needs to be involved to create value with software and how can they be involved in a positive way then in 2011 I joined Facebook and that's the beginning of a story that I won't even tell you here I've been there for four and a half years I coach programmers after three weeks at Facebook I decided to forget everything I knew about software development because it was so completely different either they were morons or I was turns out I was I had so much to learn so I relearned software engineering from scratch and four and a half years later I'm just beginning to be able to explain the method behind the apparent madness there but I'm not going to get into that that's a story for for another day one of the lessons I've learned at Facebook is is always to be informed by data that no matter what I'm doing I can find ways to be to gather data that will help me make decisions so this is I try to the word fad came up I think in the morning keynote so I just wanted to show you the graph of a fad so this is this is interest in extreme programming the the time accesses is to scale here but about 97 98 the the first part of the data is from the Google engrams mentions in literature data and the second part of it is Google Trends which measures the frequency of searches so and unfortunately neither of them cover the whole time span so that gives you an idea of what it looks like there was this huge surge of interest in XP not not vertical it it built but once once it hit a peak then it's rapidly falling off to background radiation so that's what looks that's what it looks like when you have a successful idea or at least this one in in this particular case okay so there's the timeline of extreme programming starts with my experience saying hey people really do need to change the way they do this because this isn't going to be any fun or very effective through to to today now here comes the the self-congratulatory rationalization lessons learned slide what would I do again thing one this is absolutely derived from experience the best compliment extreme programming ever got is someone said it matches the behavior of programmers in the wild I thought okay some programmers are more effective some less if you look at the programmers who are really effective they're doing something kind of like this so I thought okay I would definitely derive what I do from experience when I went to explain extreme programming I explained it on three levels a value system of communication simplicity feedback courage and respect a set of principles like incremental improvement principle of flow as I would call it now and a set of practices that are derived from those principles and I found that to be pretty effective to get away from the staring dog problem you know the staring dog problem if you try to point something out to your dog your dog is going to look at your finger you said no no it's the rabbits over there and the dog just looks completely fixedly at your finger if you explain an idea in terms of practices concrete test-driven development pair programming continuous integration people will fixate on the practices and they'll stop thinking and I found that this values principles practices triad is a not infallible but a reasonably effective way of communicating the intent behind ideas and not just have people follow things simply something else that I would certainly do again is is idealism I would rather start with exactly what I want to see back off from there to something practical and then make incremental progress towards the ideal then start where I am today and make things 5% better because I run the risk of making 5% better in the wrong direction I run the risk of not making them 20% better when I could if I just thought of well what is it that I really want I want the software always to be running oh well we you know can we do that how close can we do that what we could do that at once every three weeks okay turns out we can do that every day okay turns out you can actually do that every couple of hours oh and as experiences gathered people are more and more able to have running software as the the base state of a project not this impossible goal as the seconds tick away towards some deadline that somebody made for some reason a long time ago now there there are some things I would definitely do differently the name has been a big impediment extreme programming I wished that I had a better name at the time I'm I was faced with a very difficult set of constraints I had no resources and no money to to brand so I needed something that would be catchy I needed a name that Grady Booch would never say he was going to do I was actually my criteria and I wanted it to be something that was accurate and that's the best I came up with at the time and it's it's been a positive in some ways but it's definitely had its costs the wrong kind of people were attracted to extreme programming in the early days people who were looking for an escape from responsibility instead of a path to responsibility and the name played a definite part in that I would have changed the tone from the beginning the I look at that first edition and I just cringed that I would have written stuff like that yeah it's kind of a manifesto and a manifesto needs to be emotional but I don't think you have to be a jackass about it so and end scope now this is a tricky one because I'm a programmer and I see the world through programmer flavored lenses that's a jet lag kind of metaphor I would like to so what what Eric Ries for example is done with lean startups isn't I think the more appropriate scope for something like extreme programming to take it all the way to the customer and cycles of learning for for everybody not not focused so John on the programmers so what am i doing about this now I'm continuing quantitative studies one of the great things about working at Facebook is there's a lot of very smart programmers who leave behind these breadcrumbs of their development activities so for example when I was trying to explain that we get along almost entirely without estimates for tasks at at Facebook I was quickly able to discover that 1.5 percent of all tasks have a due date set and I just make a habit of asking and answering questions quantitatively now like does programming language does the programming language have an effect on the size of the diffs you know you submit some change to the code that's a diff so a more verbose programming language should cause larger diffs right doesn't but I love just being able to dig into some data and say oh well this would make perfect sense what is the data say oh I'm stupid again that's a great the most exciting words an engineer can say is it turns out I'm also expanding the scope of of what I'm doing one of the persistent frustrations with extreme programming was that people like to play this scope card the this sorry the scaling card oh well that might be okay for small teams but big teams you know big projects you can't do that I work at Facebook now ah nobody can play the scale card on me that's the scope the number of eventual end users involved the amount of data involved the number programmers involved projects can get very big very quickly at Facebook and looking at how to to continue that is an ongoing process for me and I'm also working on continuing my work on programming tools to further scale programming processes I think there's there's some missing fruit in code review there's missing tools in configuration management emerging that you know the programmers could do a much much better job with a little bit better tooling so I continue my efforts along those lines along with my one-on-one coaching of individual high potential programmers and the challenge which I really haven't figured out yet of coaching coaches to do a better job of coaching so every I'm trying to obsolete myself in finding myself very frustrated that my inability to teach what it is I do to to other people so that's what I do that's the history of extreme programming and I would like to thank you all very much for your time and attention here today it's cocktail time but there might be time for a question or two before we start that I'll let somebody else make that decision so I don't get run over on your way out we have questions I'm sure you it's easy to summarize and the posters in my office it says nothing at Facebook is someone else's problem that's it that's the whole thing now how do you get 13,000 people now is 2,000 people when I joined 13,000 people to take that poster seriously when they're providing an essential service for a billion and a half people every month that's the hard part and that's the that's what I'm trying to explain but there's a culture of personal responsibility at Facebook that's just still I can't believe it until my wife tells me that messengers broke again and I have to go file a task for that because I do because nothing at Facebook is somebody else's problem so it's it's typical for people will post and say oh the the iPhone app has been crashing for me I'm going to use the the stable app that we the more stable app that we send out to to regular customers because I just can't have a crashing on me all this all the time because I'm trying to get work done and the first time you say that you will be treated to clear and direct feedback that part of your responsibility at Facebook is making sure the bugs get caught before they get to people and if that's painful for you go fix it and some people don't like that and they leave and some people love it and they thrive thank you so much you

Channel: InstitutLeanFrance

Views: 75,578

Keywords: Lean management, Institut Lean France, Kaizen, Respect, kent beck, lean it, lean it summit, agile

Id: cGuTmOUdFbo

Length: 41min 49sec (2509 seconds)

Published: Thu Dec 03 2015