Clean Code - Uncle Bob / Lesson 5

growing a better world together quite the goal we set there for such an ambition we need the right skills and as developers we code so let's sharpen those skills and start coding a better world together [Music] this first edition of coding a better world together is on clean coding and clean architecture and who better to teach us more about this than mr. clean code aka Uncle Bob the one and only Robert see marki [Music] and here are your hosts for today [Music] ons clubbers and Fedora spice [Music] hello everyone I said hello everyone welcome here today a travelling for the second day of the clean code and clean architecture event I'm Hans clavos and this is Fedora's we will be your host for today when you want to share something about the contents of the talks of the day you can do that via Twitter you can use the handles you see above when you have something to say about the organization the facilities or behavior of people you can address that with me or one of the people in the blue shirts we are here to help you I want to make it a fun event for you to those would you like to say something on behalf of the Utrecht jerk yeah thanks all hands welcome everybody good morning I hope you will enjoy our great program for today on behalf of Woodruff Jack I would like to also share that we are really honored to be co-hosting this great event today I will not be saying yet why this event will be great again today we've been organizing quite a few events for the past three years however today is a big day also something more a warm welcome to the people who are watching us from the livestream and Hans I think we invite Digg float on the stage Nick is our chief architect Rabobank I didn't want to give you a welcome on behalf of rubber ring good morning everyone welcome to the second day of this event with Uncle Bob my name is Nick flood I am the chief architect of Rabobank and it's a pleasure to welcome you all here in the auditorium which we use for internal gatherings but also for external events like this one so also a warm welcome to all non Rabobank IT colleagues here in the room and the people who are joining us on the live stream I hope you are able to stay with us for the rest of the day it's a real pleasure I'm actually to have a robot or Bob Martin here today he brings a wealth of experience and insights in IT so much that many of us like to refer to him as Uncle Bob and maybe explained to you later today how that Uncle Bob name came to him already a while ago I'm pretty sure that every one of you works at the moment in an organization where the HR way of working has been adopted big time so do we here at Rabobank we have around 400 DevOps teams working agile and because we are that's quite a crowd a group we are also applying scaled agile techniques to keep that all a little bit together and as we speak we are taking the next step in transforming ourselves to work based DevOps so we applying the Spotify model with stripes and squats and chapters and because there are so many of these IT squats we also have an intermediate layer called areas between the tribes and the squats and that's we are moving to the aim of that is to create base DevOps teams with as much autonomy as possible so that helps us to get speed and agility in the organization in our digital transformation and now some agile evangelists will preach that with emergent design techniques you don't need architecture anymore and architecture is a dying major as a chief architect of course I have a little nuanced view to that I think emerging design is something that will definitely work in in startups and small scale organizations but as soon as you start to have some larger scale developments especially in an organization like this you need something to glue that together you need something what scaled agile calls intentional architecture what I would rather call a target architecture as we always used to call it but maybe just enough target architecture I'm not a believer in big design up front I've seen too many cases where we had hundreds of pages of design up front and we never realized that or if we realize it we only used a small percentage of that so no big design up front but the art is in finding the sweet spot of just enough architecture not too much not a little and also just in time architecture not too late and not too early I'm sure today we offer to talk about this this afternoon there's also session about agile and architecture we also using architecture guidelines to help us realize the three main IT roadmaps that we have set for ourselves the next two years and I'll quickly walk you through them the first is our legacy decommissioning roadmap as a large and already long assisting organization we also have quite some legacy within the bank but we are pretty far already getting rid of that so most of the new systems the more modern systems that we want to use in the future are already here and our already running and as we speak we're moving functionality from the old to the new systems and in a couple of years we'll be able to get rid of the old systems and we'll be on modern platforms the second IT roadmap that we are working towards is to the public cloud last year we decided that public cloud is the future also for banks and we are moving to the public cloud the IT for commodity functions preferably to software-as-a-service solutions and the rest of our IT preferably to platform as-a-service solutions to make use of the innovative functions that are available in the cloud so to make us more innovative that's the second IT roadmap and the third IT roadmap is our enterprise data roadmap where we are establishing a data foundation in the form of an enterprise data Lake which will be the foundation for advanced data analytics artificial intelligence and to truly become a data-driven bank so these three roadmaps we are working towards and within the boundaries of these IT roadmaps we try to to get devops teams that are as autonomous as possible and one way to achieve autonomy for the best DevOps teams is to try to stimulate them and maybe if you teach them to think like architects and that's what I particularly like about the event like this where we have a roomful of people not all architects most of your probably software engineers and if all of you and and also business people start to think a little bit more like architects then we can achieve maximum autonomy of the best DevOps teams but still make sure that we have sustainable IT going forward in the future and and wouldn't it be nice if that is what the future will look like so I'm really looking forward to today I'm sure you all do I will be here all day like hopefully you thanks for joining us I wish you a very inspiring day and enjoy the presentations thank you thank you dick now we next we will get to the person we all came for today but let me first introduce him Uncle Bob has an impressive career in the IT industry he has been working there for over 40 years he had always had a keen eye for what's architecture what's a good architecture and who and how to arrive at it he shares his view as often as possible and to give the shortest possible summary of his few it's about people not technology as one of the authors of the agile manifesto and the manifesto for craftsmanship he helped the industry to become more agile but he hates it that the agile movement is taken hostage by conference organizers and consultants who are so busy with certifying people scrum masters and project managers that they abandoned the programmers and with that they abandoned the values and disciplines of the agile itself and I think that's a sad message but Uncle Bob stays positive and he keeps him sending the message about craftsmanship so let me welcome Robert C Martin better known as Uncle Bob [Applause] [Music] okay these lights up here are probably incandescent lights probably I don't think they're light emitting diodes they could be nowadays because we can get light-emitting diodes that are very bright but my guess is that they're incandescent I think I could figure that out too you do carry diffraction gratings with you don't you some diffract because you never know when you want to know if a light is incandescent or an LED I carry quite a few of them with me so I tend to lose them yeah those are incandescent lights no doubt about it if there's a smooth spectrum coming out of them they're incandescent now these things here I hope they're LEDs probably are because they're not hot those who have those would be very hot these would not let's see oh yeah no definitely LEDs well probably LEDs yeah LEDs green and red yeah no doubt about it those are LEDs how about these yep LEDs why do incandescent lights glow why do they light up cuz they're hot okay why do hot things glow you're glowing right now in infrared radiation because you're at a temperature of 98.6 degrees Fahrenheit why are you glowing why do hot things glow why do when you hold a piece of metal in a fire why does it turn red and glow what makes hot things glow energy good good answer yes lacking a bit of detail why do hot things grow what is the mechanism so now imagine that you are an atom of tungsten that's you probably what's glowing up there the filament in those lights is probably made of tungsten metal they they make them out of tungsten because tungsten has a very high melting point so you're on an atom of tungsten and you're getting hotter and hotter because there's an electric current flowing through you and the electrons that are speeding by you are bumping into you and making you shake and that's what heat really is is just the shaking of the atoms you're a tungsten atom that is shaking you are composed of charged particles electrons and protons so the electrons and protons are shaking as well now imagine that you've got 1 electron shaking that electron is surrounded by an electric field let's say you've got an electron over here that's shaking and another electron over here that is not this shaking electron is trying to repel this one over here so as this one shakes this one begins to shake how long does it take for those waves of charge to get from this electron to this electron the waves of charge this one sets up waves of charge they move at the speed of light in fact that's what light is light is waves of charge the reason that hot things glow is because charged particles are shaking and that shakes the electromagnetic field and causes waves in the electromagnetic field which is light and our eyes interpret that it's like that is why hot things glow so the next time your children ask you why do the lights go on when you throw the switch you will be able to answer that question today is all about architecture and that's a good thing because every programmer in this room is an architect to one degree or another you must understand the principles of architecture if you're going to do a reasonable job writing code because architecture is not one of those things that is entirely high-level architecture seeps down into the individual lines of code that you write now there are lots of principles about the individual lines of code we write and we can talk about those another time but there are also principles at a high level about how software systems are structured and if the lines of code aren't written properly you cannot build those high-level structures the chief architect mentioned something and it struck me said there are folks in the agile community who say that architecture is obsolete that all you have to do is let the architecture emerge this has always been ridiculous none of the people at the agile manifesto meeting in Utah thought that we all thought that it was a good idea not to invest massive amounts of time upfront trying to design every last little bit of the software but none of us thought that architectures emerge from nothing and if you had pulled any of us back then you would have heard us tell you that no software system can be assembled at any kind of scale unless some thought had been put into what the structure of that software system is going to be so let me dispel that particular illusion right now agile is not the absence of architecture agile is simply a framework in which we can apply a good architecture now is architecture fixed when you when you initially decide what shape the system is going to be is that the shape that the system will be and the answer to that is no because as you begin to write the individual lines of code and as you begin to assemble the modules in that system you will find that there are pressures created by those modules and by those lines of code that impinge upon the architecture and cause it to change shape and this changing shape of the architecture continues every time someone adds in feature our apps for a change to a feature or adds a new facility to the system that puts pressure on the architecture and the architecture must respond therefore the architecture is a living breathing thing it changes with the system on a day to day basis it is not some fixed map that we all follow for the next 10 years it's going to be something that we always have to adjust and tweak and twiddle and respond to throughout the lifetime of any software project that was a good rant I liked that one so can I have my slides thank you good oh I should probably do this to it excellent I've built a lot of applications over the years I've been writing software since well I wrote my first line of code when I was 12 years old and that would have been in 1964 it wasn't a line of code though my mother had purchased for me at the age of 12 a little plastic computer she knew my interest in science at the time and she thought that this would be an interesting scientific toy to give her son so she bought this little plastic computer she had no idea what it was the little plastic computer had three flip-flops little sliding pit bits of plastic that could slide between two states three bits and they had a little label on the side zero and one that would show through a window so I could I could see the ones and zeros the three ones and zeros in this little window as I moved the sliders back and forth the sliders had grooves in them and those grooves would allow rods little metal rods to slip into the grooves little metal rods had Springs on them that would pull them into the grooves and then those little metal rods would push on other rods that would cause the flip-flops to change state the rods were coupled to the flip-flops by little boobs that looked like little segments of a soda straw that you would put on pegs on the on the flip-flops on the little sliders if you think about this for any length of time you'll realize that this is a three bit finite state machine driven by these little metal rods which were and gates and there were six of them so it's a three bit finite state machine driven by six and gates and you could couple those six and gates to the set or reset of any of the three flip-flops at the age of twelve I got this little toy and I played with it I opened up the manual I read the experiments in the manual they told me where to put the tubes on the pegs and how does how to crank the machine and you could make it do some very basic things for example you could make it count if you put the tubes on the pegs just the right way it would count in binary from 0 up to 7 and then back to zero again there was another program that would make it count in reverse order and the arrangement of the tubes on the pegs was eerily similar but oddly different and to the twelve-year-old mind I stared at them and going why is why are they the same and why are they different you could make it mad there was a an arrangement of tubes on the pegs that created a full adder so you could put two bits in and then and add the two bits and it would create a sum bit and it would also give you the carry bit by changing one of the inputs there were a number of other experiments you could run about 20 or so and I did them all and I was deeply frustrated because after all 20 of those experiments I had no idea how this machine worked and I wanted to make it do what I wanted to do and I had I had a program in mind it was called mr. Patterson's computerised gate I wanted this this program to execute on this little computer but I didn't know how to arrange the tubes on the pegs the connection hadn't been made at the end of the manual there was a little paragraph and paragraph said if you'd like the advanced programming manual send in a dollar and we will ship you the advanced programming man so I did I got a dollar and I stuck it in an envelope and I sent it off it took six weeks because there was no Amazon in those days but six weeks later in the mail I got a little packet in that packet was the advanced programming manual I keep this in a ziploc bag on a bookshelf in my office it holds a place of honor there I got it down and read it several years ago it is perhaps the most cogent description of boolean algebra written for a 12 year old that is possible to write it is only about 25 or 30 pages long but it's completely unapologetic it assumes that you're curious enough to work and it starts out immediately with truth tables and Venn diagrams and karnaugh maps and goes right into boolean variables and or XOR and / or x / or boolean algebra like demorgan's theorum the associative property of and the associative property of or distributive property of an over or an or over an and all that stuff and I'm reading through it going yes yes I understand yes yes I understand get to the point and after they've done all of this boolean algebra to tutorial within about 12 pages they say all right now you've got an idea for a program here's what you have to do you have to write down the bit transitions how are the bits going to change on this machine convert that into boolean equations in andover or form reduce those boolean equations to lowest terms by using the boolean algebra techniques we gave you you should wind up with six or fewer three bit equations then here is the little map that shows you how to take those three bit equations and turn them into tubes on the pegs so now I got my little program out mr. patterson's computerized gate and I wrote down the bit transitions and I converted them into boolean equations and and / or form and I reduced them to lowest terms using the boolean algebra they had taught me and I put the tubes on the pegs and I cycled the machine and that machine did exactly what I wanted it to do and I was a programmer that was it the conversion had happened the hook had been set I was going to be a programmer for the rest of my life I made that decision at the age of 12 and that's what I wanted to do for the rest of my life and that's what I have done for the rest of my life at some point you probably made that decision too and maybe it was not as dramatic as that maybe it wasn't a moment although I imagine that many of you did have a moment like that at some point you touched a computer and the computer did what you wanted you made it do what you wanted it to do and you realized that you were a god a small God in a very small world but inside that world you were a god and you could make that machine do anything it was possible to make that machine do and some of you probably had that feeling the same feeling I had and thought yeah this is what I want to do it's some kind of personality defect probably but I believe most programmers are infected with that particular particular moment in time when they decided yeah I want to be a programming god I've built lots of applications over the years since then lots and lots and lots of them and they range from all different kinds of applications I've done a lot of work in embedded real-time systems I did a fair bit of of work in telephony I did a lot of machine control applications I've done financial applications I've done factory automation applications I've written tons and tons of code in in languages all over the place and I discovered something in all those years of writing code and writing applications and what I discovered was that the the rules of architecture are independent of every other variable the rules of software structure are the same no matter what the application is no matter what the environment is no matter what the language is the rules of software architecture remain the same yesterday I talked about how software itself has not actually changed much sequence selection iteration the fundamental building blocks of software but it also remains true that the fundamental rules of architecture of software system architecture have not changed since the earliest days they remain stable many many software developers get caught in the trap that something new is better be careful of this one because there's not much new one of the things that's new right now is micro-services and everybody is spinning about micro-services yeah a micro-services you realize that micro services were invented within the first five years of software software development at the initial Fortran compiler on a on the 708 computer the IBM 708 computer was a set of micro services because they simply didn't have enough memory space in the machine to hold the whole compiler so they had to shuttle little bits of it in and out and they had to communicate through a simple simple protocol just like a bunch of isolated micro services old idea been around the block a whole bunch of times it gets reintroduced every 10 years as something new and everybody spins for a while and then we realized oh yeah that was what we did ten years ago and why did we get so excited about it be careful of this stuff the world of of software development is a world that is that at its foundation is fundamentally stable composed of rules that have not really changed in 70 years and yet it's driven by this massive effort of popularity this this wild fashion sense and we wind up with with the new fashion of software every year in the new the new language every year in the new platform every year and the new framework every and everybody goes yes it's the new thing and the new thing is not new it's old it's the same you begin to realize that after well fifty years I suppose I sucked I told you this yesterday that getting software working is the easy part getting software right is the hard part but there's magic that happens when you get the software right and it's magic that very few of us have experienced because much of the software in the world is not right but I've experienced it several others have experienced it there's this magic that occurs when the software is right and that magic is that the effort to build it and the effort to maintain it is minimized the struggle to continue to add to it is minimized the functionality and the flexibility of the software is maximized now some of you think I'm talking in utopian terms some of you think I'm preaching heaven and only if you if you don't sin any more you will achieve heaven that's not what I'm talking about here if you are disciplined and if you build software with good architecture and a good structure you will experience a reduction in the effort to keep that software system working what is design in architecture why do we talk about these things in two different terms why do we why do we talk about software design and then separately talk about software architecture why is this whole day about architecture and the answer to that is there's very little difference between them at all software design and software architecture are roughly the same thing we we try to push architecture to a higher level because it sounds auspicious architecture that sounds high-level and design sounds midium medium level but there's actually no line you cannot draw a line between architecture and design and sample this is where architecture begins and this is where design ends because even at the very bottom level there's a little bit of architecture going on and even at the very top level there's a little bit of design going on it's a continuum and so all of us all of us software developers are architects what is the goal of software architecture some folks would say well goal of software architecture is to build systems that work and can scale and and there's a whole bunch of things that you could say but the actual goal of software architecture the goal of a software architect is to minimize the human resources required to build and maintain software systems somewhere up there it says it's all about people yes it's all about people minimize the manpower required to build and maintain software systems that is the goal of architecture and that is the goal of software of the software architect the measure of design quality is simply the measure of effort required to meet the needs of the customer the quality of the design can be measured by measuring the amount of effort it takes to meet the needs of the customer if effort remains constant or shrinks with time then the design is good but if the effort required to maintain a system grows with time then the design in the architecture are bad it's a very simple relationship as effort increases design quality decreases it's an inverse proportional relationship and it's very simple to understand it's also simple to recognize if you know you are working on a system where the effort has increased you know the design and architecture are bad the data that I'm going to present you is real unfortunately it's also anecdotal because I cannot quote the sources because the folks who want to the sources the source of the data wants to keep their identity private also it's secondhand because this data was not collected by me it was collected by Jason Gorman but let me present it to you anyway and you can assess whether or not it's appropriate this is a case study of a fairly large company that began as a small Internet start-up and then grew and of course that's the hope of all internet startups they want to grow and they want to succeed take a look at the graph on the left there and that's the amount of engineering staff the number of programmers in the organization and notice how it's growing at an exponential rate the numbers on the bottom are the releases so release one had gee maybe 10 programmers 20 programmers something like that by release 2 it had doubled perhaps by release 3 it had doubled again and it continues to double roughly until by the release 8 they had 1200 programmers from 10 to 12 hundred in 8 releases think of the amount of effort that implies a tremendous amount of increased effort why to get to the next release what's going wrong there the amount of effort applied to this application is growing exponentially which tells you that the design and architecture are crap because the effort is growing through the roof take a look at the the write the graph on the right the blue one that's the number of lines of code in thousands from one release to the next and notice that it has the inverse relationship to the effort the effort is an exponential scale the number of lines of code is a logarithmic scale the number of lines of code is not changing much even though there's 1,200 programmers the amount of new lines of code are vanishingly small if you're an accountant and you're thinking in terms of money that would scare the hell out of you for the CFO of this organization that would look like a disaster in the making because you're spending immense amounts of money and you're getting almost no return take a look at the financial signature of the bottom on the bottom this is just the cost per line of code in programmer salary and it's following the exponential curve the cost per line of code is going through the roof something has gone wrong here now what what is it that went wrong well I was an Internet start-up and Internet start-ups do you think they follow any design and architecture rules or do they just kind of throw a bunch of code together and hope that something great happens and in most cases it's the latter so the startup culture especially in the net in the United States and particularly in in the West in general the start-up culture is a culture of making a mess as fast as you can in hopes that you can produce something that will put you on the exponential growth of profit but this is a very bad strategy even for an Internet start-up because it forces most internet startups to experience this particular problem and before they can ever get profitable it kills the company most internet startups especially software internet startups fail not because they couldn't have succeeded commercially but because they can't maintain and sustain the cost of the software they're producing we all get excited about you know Facebook or or other internet startups that really made it big but how many fail that you never hear of and the numbers are astounding massive numbers of them fail would they have failed if they had kept their software clean some of them would have many probably would not because they could sustain the effort long enough to grab on to the profit rollercoaster some people worry that well we've got to be first that's the only way to grab on to the profit rollercoaster is to be first and therefore it's justified that we make a mess early so that we can grab on to the profit rollercoaster and then we will be able to afford to clean up the mess later was Facebook first not by a longshot Facebook wasn't first Facebook wasn't second Facebook wasn't third there were a whole bunch of social networking companies out there Facebook just happened to hit for some reason and it's not exactly clear why whatever happened to MySpace it was anybody on MySpace oh yeah what happened just didn't hit for some reason didn't hit they did something wrong but it wasn't it wasn't because they made it wasn't because they got there first Facebook succeeded not because they got there first continuing on with this look at the productivity per release the productivity per release I arbitrarily set to 100% at release 1 by release 8 the productivity is on the floor I told you about this yesterday the decline in software productivity as the mess builds here's the actual measured evidence and look at that fascinating curve how it dives so rapidly and then bottoms out and asymptotically approaches zero once again I stress this is real data and look at the cost per release in monthly payroll if you're the CFO and that's who that guy is there well that's not really him I just found that picture online but I think it probably assesses pretty well his attitude towards what's going on here there's an old fable they have The Tortoise and the hare it's one of Aesop's fables and the the fable goes this way there's a a rabbit a hare a rabbit who can run fast and a tortoise who go slow and they go into a race and the rabbit is so confident that he can win the race because he's fast that he falls asleep and the tortoise passes him and wins and the moral of the story of course is that slow and steady wins the race have you ever heard that adage your grandmother probably told it to you at least I don't know what it is in Dutch but in English that's what it is slow and steady wins the race one of the things that we face in the software community right now is this issue of the hare and the tortoise because everybody wants to be the hare running as fast as they can to produce Software's as quickly as they can and we hear the effort from from our organizations we have to go faster we have to go faster we have to go faster and every software developer in the room has heard that message we have to go faster and you feel the pressure you you judge yourself worth by how fast you can go that is not your self-worth that is not your worth as a programmer speed is not your worth as a programmer but we judge it that way we think that way that's how it's been impacted upon us for decades speed is everything and so what happens because of that speed because of that need for speed we become overconfident like the hare but what is the overconfidence the overconfidence is not the confidence that we can go fast but overconfidence is the confidence that we can deal with the mess okay we're gonna go fast but we can deal with the mess and we'll go back and we'll clean it up later we'll deal with it we'll deal with it somehow we can deal with the mess because we need to go fast and so we roar down the track and we're going really fast and we're making a mess and the overconfidence is that we can deal with that and what is the tortoise in this scenario the tortoise is the mess because the mess builds slowly but inexorably and eventually it will pass you and dominate everything about the project of course we need to go fast but how do you go fast what's the secret to going fast as a typing faster is it thinking faster what is the secret to going fast the secret to going fast is to not build the roadblocks that are gonna make you go slow because the roadblocks will make you go slow so don't build them you're a programmer how fast can you go well you have a speed it's your speed it's nobody else's speed it's just yours the speed of your thought the speed of your ideas the speed of your ability to conceptualize is there anything you can do to make that go faster can you rewire your brain to make your make yourself go faster well yeah there are some things you can do study practice but they're long-term things they're not short-term things if someone comes to you and says I need you to go faster what can you do what can you do to go faster tomorrow nothing nothing there's nothing you can do to go faster tomorrow you might be able to go faster next year by learning something by studying something you might be able to go faster five years from now by practicing and developing skills but there's nothing you can do to go faster tomorrow so what's the key to speed you cannot speed up what is the key to speed and the key to speed is not to slow down not to build the stuff that's going to slow you down so the key to going fast is to go well do the best job you can do the cleanest job you can make sure the track is nice and clear don't roar down and throw a mess down there build in a you know steady consistent clean way so that you can continue to go as fast as possible which by the way may not be very fast but at least it's not going to slow down we often have the impression that going clean is slow and so we make the decision that temporarily we will go dirty because we can go faster by going dirty many many people think that in fact there is a phrase in English I don't know if you have it in Dutch and the phrase in English is quick and dirty now there's no such thing as quick and dirty at least not in software you cannot go fast by making a mess even in the short term and this is one of the debates we will have some folks will say well I understand that a mess will slow me down in the long term but in the short term I can go faster by making a mess no you can't no you can't the mess will slow you down within an hour or two how many of you have had the experience that you write code on your screen all morning long you go to lunch you come back from lunch you look at your screen and said who the hell wrote that crap because the mess will slow you down in a matter of an hour or two if you are not clean the best builds and slows you down instantly the graph that I'm showing here is from the same guy Jason Gorman did this experiment and I thought it was a good experiment he wrote the same program six times in a row on six different days the program he wrote was a program that he knew very well it's an exercise that programmers often use as a way to practice it is the conversion of an integer into Roman numerals try this one day it's a fun little program to write what he did is he wrote a set of acceptance tests that would demonstrate to him when he was done prove when he was and then he said about to write that program six times in six different days but on every other day he used a discipline on the even days he did not use the discipline on the odd days he did use the discipline the discipline was test-driven development and you can see the results here now notice the bottom of the graph right this is one of those statistical cheating mechanisms right I cut the bottom of the graph off at 22 so what we're talking about here the differences here are relatively small but I wanted to accentuate them so I'm lying with statistics with test-driven development he was always faster notice that he's always getting faster during the entire exercise right every day he does it a little bit faster than the day before but with test-driven development he was always faster than without test-driven development and yet his experience was that test-driven development felt slower and why did test driven development feel slower well any discipline feels slow all disciplines feel slow because when you are exercising a discipline you are stepping through the discipline one step at a time all disciplines feel slow but they speed you up if they're good disciplines test-driven development is a good discipline sped him up every time just felt slow what is the solution to the executives dilemma the dilemma that the effort is building exponentially and the return is falling off logarithmic lean what is the solution to this and the only way to reverse the decline in a software project is to just be as clean as you possibly can and all the developers and all the architects and all the designers follow the disciplines of software development and write the best code they can all the time and the executives and the managers need to that expectation what do we as managers expect from programmers we expect you to do a good job and we trust that if you do a good job you will be going as fast as possible that is the expectation of Management and that is the behavior required of the software developers software has two values there's the value of what it does and there is the value of its structure programmers often ignore the second value we focus on what the software has to do and we believe that our job is to satisfy that need there's a set of requirements out there and we must meet those requirements and if we meet those requirements we have done our job but if we do that we've only satisfied the first value of software there is a second value of software and it is arguably the more valuable of the two the second value of software is software that actually works when you make it work stupid thing yeah there there it is look at that dumb thing the second value of software is the structure of that software and why is that a value it's a value because of what software is yesterday I told you that software is a compound word that it means flexible product software the very nature of software is that it must be changed it must be changeable if it is not changeable the software has no value if you give me a system that works perfectly now but I cannot change it that software will be worthless tomorrow because the requirements will change and if I can't change the software that software will come worthless instantly so they must be able to change it I must be able to modify it and the cost of modifying it must be low so that I can continue to make that software do what it needs to do to satisfy the customers demands if you give me software that does not work but it's changeable I can make it work it is more important that the second value be maximized than the first because if the second value is maximized if the software is flexible I can make it work but if you maximize the first value at the expense of the second if you maximize the fact that it works today at the expense of being able to change it then when the requirements change the costs will inhibit that change and if they inhibit the change too much the software's project will fail these two values are necessary for every software developer to keep in their mind at all times it is not enough to make the software work the software also has to be right and the word right is an unfortunate word because it's a binary word I don't need the binary connotation the software has to be well structured but it doesn't have to be perfectly right it has to be good software it has to be well thought through the structure has to be changeable the value of its structure has to be high I don't see if I can make this stupid software work by the way yeah just a little rant are we gonna see self-driving cars anytime soon oh yes we have we've seen some they haven't killed too many people yet they drive in relatively safe areas during good weather and generally the law says that there has to be an actual person in them to take over we don't typically see autonomous cars running without a human in them although there have been a few cases of that as well but my question is not that my question is are we going to see self-driving cars that you and I can go to and call one up and a car will pull into our driveway without a driver in it and we will tell the car to take us somewhere anywhere and the car will take us anywhere drop us off are we gonna see that no not a chance not a chance this is an overconfidence in software software is not that good as I just demonstrated with that stupid program right there software is not that good we're not going to see that in two years we're not going to see that in five I don't think we'll see it in 20 we may not see it in 50 there's a number of reasons for that we've all gotten extremely used to the idea that software computers are an exponentially growing thing who remembers Moore's law yeah okay so Moore's law is dead right Moore's law died over 10 years ago now clock rates on computers have not increased since about 2002 right my laptop in my bag over there goes at 2.6 gigahertz the laptop before that one a 2.6 gigahertz the one before that one at 2.6 gigahertz and it wasn't until I can get back about 10 years that one was maximized at at about 1 gigahertz clock rates have not increased we've tried to increase their productivity bore their throughput by adding more processors so my about 10 years ago I got my first two core processor and then I got a four core processor I was convinced that the next laptop I was gonna get would be an 8 core processor but that did not happen and then I thought well certainly the next one after that will be an 8 core processor but that did not happen and I don't believe it will now I don't think we're going to get 8 cores maybe we will but I don't think we're on that power of to curve the density of the chips can't get much greater because the number of atoms in a wire is now countable on a hand or two so it's gotten to the point where we can't really get much more density we may be able to go in three dimensions but the heat problems there are may be insurmountable the point I'm making is that the hardware that we have been used to growing at an exponential rate is no longer growing at an exponential rate we are sitting now on a plateau computer power is probably plateaued we can enhance it a bit by going up up to the cloud although there's costs to the cloud don't just say that the cloud is great and wonderful and everything is life one life is great on the cloud because the cloud has a whole bunch of costs to it but we can certainly gain more computer power by recruiting more and more servers on the cloud but all we're doing is recruiting more and more servers and that by the way is a linear growth not an exponential growth so we've got this problem our expectations of exponential growth have set us to believe that in 10 years virtually anything's possible anybody read you know the the the singularity that's coming what was that Kurtz while wrote about the singularity eventually machines will be able to design better machines and they'll design better machines and the whole thing will roar off into some apocalypse no that's not going to happen who heard about Watson right who won jeopardy right the IBM computer that won jeopardy beat all the best jeopardy humans and that was a great achievement it's a wonderful thing but let me tell you about one question that Watson got wrong and the question that Watson got wrong was named a city that has two airports one named for a world war two hero and the other named for a world war two battle now then the answer is Chicago because O'Hare Airport is named after butch O'Hare who was a World War two hero and Midway Airport the other Airport in Chicago is named after the Battle of Midway so the answer is Chicago Watson said Toronto now the question was named in American city and the answer was Toronto now Toronto's in Canada I suppose that's American it is the 51st state in Canada that joke doesn't go over so well but Toronto does not have two airports that meet that criteria so that's clearly wrong and besides it's not an American city and when the programmers dug into it to find out why and noticed that the programmers had to dig into it to find out why why Watson answered Toronto they found that there was a city in Idaho named Toronto which happened to have two airports near it and they matched the two criteria so Watson was correct although stupid worse than stupid irrational no human being would have answered that question that way and if any any human being had answered that question that way we would have sent them to a mental hospital because that makes no sense in human terms Chicago is the obvious answer here not Toronto Ohio to Idaho now imagine a self-driving car and a self-driving car during a rainstorm runs over a two-year-old kills him who goes to court over this where who who who should we blame for that someone must be responsible the two-year-old is dead someone has to be held responsible who do we blame the car the programmers the company that made the car is there anybody who can take responsibility let's put the car on the stand let's interrogate the car car why did you kill the two-year-old Toronto yes I told you about that yes I told you about that stakeholders in software systems believe that requirements and changes are complicated only by scope now a stakeholder can understand that certain changes are big and that other changes are small and they wreak they they believe that the cost of changes will be proportional to that scope big change it should cost a lot little change it cost a little but programmers don't see it that way what happens is that programmers will build structures in their systems and what a stakeholder believes to be a small change will cause massive repercussions through the structure of the software and so that the developer looks at that is a big change this is a faulty architecture faulty architecture is an architecture that lies to the stakeholders by making small changes expensive and big change is cheap programmers see each new requirement as a puzzle piece that must somehow be fit into the existing puzzle of the software and the more puzzle pieces there are the less room there is in the puzzle for a new puzzle piece and the software developers are constantly struggling about how the heck are we gonna fit this doggone thing in here and so every new change gets more and more expensive the solution to this is to keep that puzzle as simple as you can keep it keep all the edges square all the pieces is nice squares so that you don't have this problem of trying to fit everything in there all the time yes I told you that already stakeholders want changes but stakeholders do not have the ability to prioritize architecture they don't know what architecture is they don't know what software design is stakeholders in a system have no clue that you have to work hard to build a structure that supports their requirements all the stakeholders see are the requirements themselves they don't see that structure that's second value they cannot assess the second value and therefore they cannot prioritize it no stakeholder can prioritize architecture and it is honest it is improper to expect that stakeholders will prioritize architecture software developers are very often complaining about the fact that nobody ever prioritizes architecture all they care about is as requirements well of course all they care about is requirements cuz that's all they know they can't understand what architecture is you know what architecture is you are the only ones who know what architecture is and it is your responsibility to communicate that as forcefully as you can to the stakeholders in the system and that puts a very interesting responsibility on you and I we have to somehow communicate the importance of the second value to the people who are paying us and if we fail at that the people who are paying us will pay way too much for their sake we have to communicate the importance of the second value I like to look at it slightly differently I like to say the stakeholders of a system have no right to tell me what the architecture ought to be they have no right to come in to me and say skip the architecture let's just get this done fast they have no right to say that to me because I know how to get this job done well you do too you know how to do the job well we all do we all know how to go well we know how to do it stakeholders don't have the right to say no skip that don't do those disciplines it's too slow they don't know it's too slow they're just guessing it's too slow maybe for some reason you know and you have the responsibility to communicate that backwards back down and how do you do that by asserting your authority because you have the authority you have the authority because you have the knowledge you have the authority because you have the responsibility believe me you know who's going to get blamed so you have the responsibility and you have the authority you must communicate that downwards that is not an easy thing to do communicating that importance to people who are interested in requirements it's not an easy thing to do it requires a certain amount of emotional fortitude and programmers and I'm sorry for the stereotype but it tends to be generally true programmers did not get into programming because they like dealing with people programmers would rather most of them and not all of them but a large percentage of programmers are perfectly happy sitting at a terminal locked in coding away and people are an annoyance especially if those people are emotionally confrontational I learned something a very long time ago and it's a generalization that I will make here I don't have any research to say that it's true but I I believe it's true just from my own experience there are people like programmers who like to deal with things like code but there are other people whose greatest joy is dealing with other people now that's not something I understand right because although here I am sitting on stage right and I'm pretty good on stage being on stage is a formula it's an algorithm I can execute that algorithm and I can execute it very well I do not actually have to relate to you isn't that interesting for all intents and purposes I'm not gonna say what I'm about to say what I'll say is this I can do this on stage if you were here I could be on stage alone with a camera there and I'd be exactly the same and some of you have seen me do that programmers like me anyway are not very good at emotional confrontation but I learned something a long time ago there are people who are good at emotional confrontation they're good at it they've developed the skill and it's a people skill and it's not just emotional confrontation it's relating emotionally to people I'm not good at that but they are they get their joy out of it and very often they become salespeople and managers how do people like that get the truth I know how I get the truth I get the truth mathematically and logically I think it through you know it's a cold thing truth for me is cold but I believe for them truth is hot they get the truth by testing people they want to know what you believe and so they will confront you and they will say to you you don't really believe that I need this by then I need this to happen this way and they will confront you and they want the truth they're seeking for the truth and if you go okay they got a truth but it's not the truth I learned this by sitting in a conference room with a bunch of managers and we had a real problem at this company and we had a bunch of customers there was no way to satisfy all these customers we were going to have to let some of them down there was the customers that I was representing as an engineer there was customers that other people were representing we were having this discussion I saw the discussion going away from my customer and Here I am sitting there you know little Bob's programmer watching these guys take the Reese away from the customer that I was supposed to defend and I completely lost it I started yelling at them right which is very uncharacteristic for me despite what you see on stage and I yelled I looked them in the eye and I yelled at them and I used flowery language do you beg the lightning to strike which I stole from Star Trek by the way and I saw these managers look me back in the eye with a respect I had never seen in their eyes before and suddenly they accepted my assertions and said yeah you're probably right about that we're going to have to do something a little different we've got to pay some attention to that customer that had never happened to me before I'm not sure it's happened ever since but that was a fascinating lesson for me that there are folks and they tend to be the folks who are managers and sales people and people people who learn the truth through confrontation and so you and I who are the defenders of the architecture the defenders of their money need to learn how to tell them the truth and that is not an easy thing now by the way I'm not telling all the junior people in the room to rant and rave on screen but it might be wise for the senior people in the room to learn the skills as distasteful as they are to a programmer of dealing with people because it's about people isn't it this is Eisenhower General Dwight D Eisenhower the in general of the victory in World War two then he became president of the United States and interestingly enough Eisenhower was not convinced whether he wanted to run as a Republican or a Democrat this was at a time in the United States when you could barely tell the difference between the two parties that has changed and Eisenhower said this very famous statement where's the statement previous slide yeah thank you you all read it before me yes okay good I have two kinds of problems the urgent and the important the urgent are not important and the important are not urgent these two kinds of problems are the two values the urgent and the important the requirements are urgent the architecture is important the architecture is important because it has an extremely long term value the requirements are urgent because they have a short-term value they need to be implemented soon otherwise the customers will be unhappy the architecture is important and then Eisenhower put this matrix together which of these are you going to do the important and the urgent the important not urgent the unimportant and urgent or the unimportant not urgent I would make the case that the one on the lower right you're not going to ever do it's not important not urgent what about the unimportant urgent things hmm you might not do those if you have some time you might do them but you might not do those because they're not important they're urgent but a lot of urgent things can be waved away sorry can't do that one I know it's urgent but it's not important then there's of course the important urgent ones you're going to do those you just have to do those and then there's the important but not urgent things and this is where we get into the biggest debate and as developed as defenders of the architecture and defenders of their money those are the ones that we fight over and we'd better win those it's always a struggle these things aren't easy it gets into an area the programmers don't like to get into this whole interpersonal thing but you have to remember something important you are a stakeholder too there are stakeholders that aren't programmers but every programmer is a stakeholder because your reputations your jobs your salaries depend on the job that you do you have a stake in this so you are also stakeholders no less than any of the others and you ever say if architecture comes last the costs will grow and the costs to the organization and the costs to you may grow insurmountable that was the talk I was supposed to do second because that was the Christ that architecture of the software crisis but I thought it would be better to do that one first are there any questions about that before we go on to the next one which is clean architecture that was a little more technical so all the people in the room oh god thank you stop talking about all this people okay questions comments good okay clean architecture do to do at no wrong one that one there we go okay so now we're gonna get techie but no less significant about eight years ago my son handed me the source code of a project and I opened it up and this is what I saw now this was a rails project does anybody here done any Ruby on Rails oh yeah okay a few of you good good well then you'd recognize this as the directory structure of a rails application because all rails applications have roughly the same structure this is rails of about a one oh six seven years ago something like that and I looked at it and I said oh yeah it's a rails application I can tell by the directory structure I had seen lots of other rails applications before and I knew what the directory structure was so it was very familiar to me but that moment for some reason struck me as odd it had never struck me odd before but it struck me odd this time that I could look at the directory structure at the top level of the application and realized that it was a rails application now why is that at the top level of the application oddly what I could not tell from the top level of the application is what the application did and that bothered me I thought wait wait wait at the top level shouldn't I be seeing the intent of the application why am I seeing the framework at the top level and that bothered me it had not bothered me previously but it bothered me on that day and so I started to think about this I thought well now wait a minute rails is a web framework and what is the web the web is an i/o device does anybody thought about it like that the web is an i/o device now you maybe you think it's a really complicated i/o device and they're really cool io device but it's an i/o device it's on the outside of our system its peripheral to the system it is a peripheral device and there's something we learned a long time ago about i/o devices we learned this way back in the 60s we don't want to know they exist we build operating systems to hide i/o devices from us who's done some unix work right stand it in and stand it out what device is that huh we don't care we don't want to know why don't we want to know well long long ago back in the 50s and the early 60s people would ask us to write programs like can you write a payroll program and we'd say yeah I will write you a payroll program you just punched all the employees on two cards and will read them in and punch their time cards onto cards will read them in and will punch out paychecks and we'd write the code that way and the code would literally talk to the card reader and the card punch and then they'd come along five years later and say you know we moved all our records on the magnetic tape and the programmers again oh my god that changes everything because dealing with the tape driver was really different from dealing with the card reader and the card punch and we thought okay we don't want to go through that again so let's just let's just make all the i/o devices below the level of the operating system the operating system was invented primarily for this purpose let's get all the i/o devices below some line some boundary line and then we can write our code and never worry about that again and here I've got the web at the top level of the application the web dominates it the i/o device dominates I thought well that's wrong there's something wrong with that architecturally there is something wrong with that so I started looking around it floor plans of buildings and I I noticed that the floor plans of buildings do not tell me what the buildings are made of the architecture of the building does not tell me that the building is made of bricks or wood it doesn't tell me about hammers and nails what the shape of the building tells me is the intent of the building that's a library and you can tell it's a library if you look at it for any length of time you say well it looks something like a library it's got rooms it's got tables it's got like library stuff and if that doesn't convince you this one will that's a church the architecture of buildings screams out the intent of the building the shape of the building tells you what the intention of the building is how it's going to be used and isn't that what the architecture of a software application should be doing shouldn't the architecture at the topmost level be screaming at you about what the intent of this application is it's a banking application it's a payroll application shouldn't it be yelling that at you instead of saying I'm a rails web app who cares if you're a whales or a rails web app I don't care if you're on Rails web app I want to know what you do and that made me remember something made me remember this the guy you're seeing is Ivar Jakob Singh he wrote a book in 1992 name of that book was object-oriented software engineering and the subtitle was a use case driven approach does anybody remember use cases so little Led Zeppelin there anybody remember laughter now nobody nothing gets that you okay does anybody remember use cases yeah a bunch of you remember you and what happened to use cases the consultants got at them and the consultants said oh use cases people like use cases okay well when I need to be the use case expert and so I'm going to I'm gonna put something on the web that lets people fill out a form for use cases and then the next thing Sultan said well I can do that too but my form is gonna be better and there was this war about the best possible use case form and by the end there were these crazy forms on the web that on the internet it wasn't the web at the time and they were you know full of all kinds of blanks you could fill in what's the primary actor and the secondary actor and the tertiary actor and the preconditions and the post conditions and all this crap and though the world kind of went maybe we don't want to know about use cases that was right about the time that agile started and agile said we got to rename this thing so they named it user story same thing though when the occupation was talking about we're not massive forms filled out with lots of detail a jakob Sounion use case well it looks like this this is the Jakob Sounion use case and assume that this is for a system an order entry system for example and this is the create order use case and notice that it does have a little bit of structure in it it it identifies the input data to the use case in this case it's the customer ID and the shipment destination the payment information notice also that there's no details there I don't tell you what the customer ID looks like or what the shipment destination looks like or anything like that the use case is not the place for that deep don't put that in there it's just a list of the kinds of data we think we're going to need in order to execute the use case and then the next thing down is called the primary course that's what Jakob s'en called it the primary course and this is just the processing steps and in this case it's the order clerk issues to create order command with the above data that's actually not part of the use case that's what kicks the use case off the system validates all the data doesn't say how doesn't tell you what the validation rules are no details or anything just it validates the data system creates the order and determines the order ID that's probably a database operation but it doesn't say that here and then it says that the system will deliver the order ID to the alert to the clerk probably on a webpage but it doesn't say that either it's a very nice concise description at an extremely high level of what the input data is the processing steps are in the output data now Jakob s'en said something very interesting in his book he said if you've got a use case in this form you can turn it into an object remember the title of the book was object-oriented software engineering said if you've got a use case in this form you can turn that into an object that you can write code in and he called that object a control object but the name didn't work because of model view controller and everybody got confused so I renamed it interactor recently other people have have chided me for this and said well Bob you should have called it use case and I think they've probably got a point but I called it interactor so I'm gonna stick with that for the moment an interact or object embodies a use case it's a class that you write in Java it embodies a use case it contains application specific business rules now we need to separate business rules into two different types there are application specific business rules which are all about the automation and then there are application independent business rules which are the business rules you would execute even if there weren't a computer there's a bank charge interest that's a business rule that you're going to execute even if there's no computer the use case is application specific business rules they're all about the automation these are the things you have to do because there is a computer now where do you put the application independent business rules well yeah crimson put those into these things that he called entities nowadays people call them business objects but I've kept the word entity in honor of Jakob s'en these entities are controlled by the inter actors so the use case the inter actor guides the execution of the entities it controls the execution of the entities the entities execute the real business rules the use cases coordinate the business rules and then you've got to get some data in and out and Yakub person called those interface objects he later changed the name to boundary objects I have kept that name and I have drawn them here as Java interfaces there's two of them you can see there the one on the top is the output boundary and you can see that it's the output boundary by that red arrow that points towards it in UML that is a using relationship so the inter actor uses the output boundary to send a data out the boundary on the bottom is the input boundary and you can tell that because of the red inheritance arrow from the inter actor the inter actor actually implements the input boundary interface and it uses the output boundary interface the methods on the boundaries are the methods that allow data to come in and data to go out now let's walk through the process here because the process is pretty simple we've got some user down there at the bottom and the user is interacting with the system and I've got a delivery mechanism there I purposely called it a delivery mechanism because I didn't want to say web but okay I called it a delivery mechanism so the user is dealing with probably some webpage filling out some form and eventually the user hits the submit button and that passes the data into the delivery mechanism into the web server the web server reorganizes that data and creates a data structure which I've called a request model this is a raw data structure it's full of just plain old objects this request model does not derive from any framework it doesn't derive from any kind of web related thing it does not know anything about the web it is just the data structure that holds the input data to the interactor it gets passed through the input boundary to the interactor and the interactor looks at it and interprets it with a bunch of if statements and then starts to coordinate the dance of the entities tells the entities what to do and and what data to do it on feeds the data from the request in then when the int when the entities are done the interactor reverses the flow and gathers the result data from the entities and builds a result model the result model is just a raw data structure has no notion of the web at all has no idea how this data is going to be displayed it's just a plain old Java object if you're using Java and then that result model gets passed out through the output boundary through the web which formats it and does all the interesting stuff and gets displayed back to the user it's a real simple input process output kind of structure but it has a very interesting aspect to it can you test those interactors do you need the web server running to test those interactors no because the web you don't need the web you can go through those boundaries the whole web doesn't have to be there to test the interactor you can test the business rule without the web now that's good because the web is slow and you can run your tests a lot faster if you can test without the web it's also a lot easier to test without the web at 30,000 feet so if you're flying over the Atlantic and you've got your laptop with you and you want to run some tests well you can run tests because the web doesn't have to be there and that's interesting and how do you do that well you feed a request model in and you make sure that the response model comes out appropriately and you can test all the business rules without the web being there you can test everything on your laptop at 30,000 feet now what about model view controller I thought we were all supposed to be doing Model View controller is this somehow related to Model View controller the answer to that is well yes and no where did Baum's view controller come from does anybody know what's the history of Model View controller model view controller is probably the first named design pattern before the design patterns book was ever written decades before the design patterns book was ever written the Model View controller design pattern was well known it was invented by this guy whose name I'm going to butcher he's Danish his name is trig Vereen scout but I've butchered that name because I don't know how to say it I met him once I was in Norway at a conference and I was in the speaker lounge and I was looking for a place to plug in my laptop and this old grizzled guy reached down and handed me a power strip and I took the power strip from him and I looked up and that's chickens Cod and my hand happened to brush his and I haven't washed that hand triggering Scout back in the 70s came up with this idea model-view-controller it's a very simple idea there's a model object in the middle a model object understands business rules right so let's say we're doing a clock the model will understand how to keep time it doesn't know how the time is displayed it doesn't know how input from the time comes just knows about time the controller knows all about input it gathers input from some input source and then restructures that input into commands against the model and the view knows how to display the model data somehow knows how to present it somehow somewhere we don't care where notice the funny little arrow that I used from the view that is an observer relationship an observer relationship involves a callback the view registers with the model and then when the model changes it tells the view hey you better redisplay me because I changed this is model view controller this the way model view controller was invented in the 1970s when in the small talk environment it's a very simple input process output model and it was intended to be used in the small you did not have a model view controller for a page or a screen you had a model view controller for a button you had a model view controller for a textbox you had a model view controller for a little canvas a screen would have dozens of little model view controllers dancing around that was how we structured it in those days so this is all changed now model view controller has changed and it gets changed by people who don't read the original work but let's hear the words and think well model view controller I think I understand that I don't need to read the paper so what it is turned into is this and it's very unfortunate we've got the web out there and the web does all the interesting work of gathering input from the user and then turning it into URIs or URLs or god help you whatever and then eventually the web hands that off to controllers and the controller's job is parce all that input data and then talk to Business Objects off to the side there and the Business Objects do their little dance and then the controller hands the hands the control over to the views and the views gather the data from the business objects and display it and that's how Model View controller has unfortunately evolved now the problem with this is that there are no hard boundaries here the controllers know about the business objects intimately the what happens is very often Business Objects start to accumulate controller like functions or worse controllers start to accumulate business rule like functions the same thing happens to the views the views will often accumulate businessí like functions or worse the business objects will accumulate formatting and display like functions because the boundaries aren't well formed here so although Model View controller is a nice thing to do when it's done in the small it starts to falter and fail and it's done in the large I would come back to Jakob syns model and I've drawn a boundary in here that double black line is a boundary and boundaries are going to become very important to us because boundaries are the essence of architecture that is an architectural boundary there and notice across that architectural boundary all dependencies point in one direction and that direction is towards the business rules all of that is very important that's called the dependency rule across an architectural boundary all dependencies point towards the business rules now what do we see here we see the interactor the interactor has just created the response model the responsive model is going to be passed through the boundary to a presenter presenter lives on the other side of the boundary the presenter implements the output boundary and it lives on the other side of that double black line the job of the presenter is to reorganize the data so that it is ready to be displayed the response model is a raw data structure but it doesn't know how the data is going to be displayed so for example if there is a date in the response model it is a date object if there is currency in the response model it is a currency object by the time the presenter grabs it however it will transform those objects into strings so you notice that the presenter creates a view model the view model is a data structure that contains strings the dates will all be turned into strings properly formatted the currencies will automatically be all be turned into strings with the appropriate commas and dollar signs or currency symbols or what have you if there are menus on the display the names of those menus will be in the view model the presenter will load them if there are buttons on the screen the names of those buttons will be placed in the view model by the presenter if a button should be grey because it's inactive the presenter will set a boolean in the view model the view to make that button gray all the interesting decisions about the view are actually done in the presenter and put into this view model data structure and then the view model data structure is handed to the view and the view I've drawn is this kind of faded thing because it has almost no code in it all it does is take the data out of the view model and put it into the display can you test that presenter can you test it without the web server running yes you can so half of the half of the view side of things I can test without the web server running can I test the view without the web server running yeah that would be difficult can I test the view without the interactor yeah I can I can test the view without the interactor that means I can test the view without any business rules I can just test the view stuff without any business rules that means so that view stuff is the view test is probably gonna go fast this shows the whole left side of the of the structure we've got the interactors on the right and they take in request models they produce response models they pass the data through the boundaries the controllers are the ones that pass the data into the interactors the presenters are the ones that take leave data and present it to the view and there's that big architectural boundary there and notice that all of those arrowheads are pointing to the right towards the business rules that's the dependency rule what about the database I'm gonna show you a picture see if it see if you recognize it is that the database is the database the Great God in the center of the system with all the applications around it bowing down before its majesty who benefits from that view yeah Oracle does right now database companies do you know that's where that view came from that was a big marketing ploy back in the 70s and 80s to gather the to make sure that databases were considered central to everything and it was fascinating to listen to the marketing language at the time they they they coined this term called data assets what a great term when you're talking to a CEO yeah do you protect your data assets I didn't know I had data assets oh yes you do and our system will protect your data assets oh I better buy your system yes you should fascinating marketing single worked wonderful just look around and see all the Oracle buildings around you'll notice that that works very well it's a fascinating marketing strategy but what is a database it's an i/o device databases are i/o devices they store data there are little rotating disks or the software on top of the rotating disks oh they provide lots of interesting facilities query facilities and reporting facilities and all that stuff and let me rant just for a minute about the absolute insanity of creating a text language to access our data sequel a text language to access our data a language that can be squeezed through the input of our systems and somehow used to pollute and interrogate our systems sequel injection had we thought even for a second about security we would have realized that we never ever wanted a text-based language to access our data what do we want to access our data a software API is it fun constructing sequel have you ever done that you know construct a bunch of sequel is that fun you know writing the code that constructs all that sequel crap so that some other thing can undo what you just did and understand it when you could have simply told it but no we have to construct this so I'm sorry that was a rant the database is a detail the database is an i/o device from your point of view from a software developers point of view the database is there to hold data not anything else oh you know querying is fine but we'll talk about that if the database is a detail where do we want it well we want it on the other side of an architectural boundary we don't want the business rules to know about the database we certainly don't want the business rule to know about sequel we don't want the business rule to know about the schema of the database we just want the database below that architectural boundary so how do we do that well you see that interface up there called the entity gateway that's an interface that has a method in it for every query that you want to perform so let's say that you want to get all of the employees who were hired after 1976 you would have a method in there that said get all employees after and then you'd pass in a date every query that you want to perform is just a method on that entity gateway now I've only drawn one entity gateway there would actually be many because you'd probably have lots of different entities and you'd want to partition them reasonably but all right let's say there's this entity gateway and for every day base operation there is a nice little method on that interface which of course is not implemented in the interface and then below the architectural boundary there is the entity gateway implementation which implements all those methods to operate on whatever database you currently happen to have now if it's a sequel database then that's where all the sequel is down there it's not above the line so there's no sequel above that black line there's also no schema above that black line I don't want you taking daos and passing them across the black line you know you've got a relational database and you fetch a bunch of rows out of it and you keep it in a form of rows I don't want rows crossing that black line by the time anything crosses that black line I want them to be objects not rows and what are those objects well they're used by the entities some people would say well they are the entities that's not probably correct but they are used by the entities the entities contain business rules not necessarily business data business data will be used by the business rules so there are probably some little data structures that the entities used that are populated by the database yes nope not here I got to do something you've you've asked me a question and I've been told that I need to throw you a microphone are you ready for this okay all right now if I miss him get ready to catch it and pass it to him you ready for this here we go I don't know yeah yeah okay fine good okay thank you hi you're welcome what is your opinion about or an implementation I'm glad you asked me that question all right you could just slowly pass that well actually just keep it kind of there and and you guys can be responsible for throwing it where does the are I'm what is an ORM what does ORM stand for object relational mapper there's no such thing as an object relational mapper like that that doesn't exist and why is why doesn't that exist because what is an object an object from the outside looking in an object is nothing but a bunch of methods function calls an object may contain data but you're not supposed to know that and and there's no rule that says that an object must contain data an object can simply use another data structure so an object is something that contains behavior not data necessarily a database contains data not behavior so there is no way to map from relational data to object behavior what an ORM actually does is it fills in data structures that's all it does it takes database rows and it fills in data structures not objects a lot of people make the mistake that your business objects are the things that the ORM builds that is probably not the case what the ORM builds is the business data that gets fed to the business objects not necessarily by constructor but through phone arguments where would the rmb in this model yeah it's the Gateway implementation it's below the line the Gateway implementation talks to the ORM which talks to the database it's below the line nothing above the line knows about the ORM there's anybody using spring or or or JSF or any of the what is that the the JPA the JPA java persistence engine anybody using any of that stuff all below the line it all goes below the line I don't want any vestige of an ORM above the line the entities don't know about it the interactors don't know about it all that stuff stays below the line be careful of frameworks I'll rant about frameworks later the authors of frameworks do not have your best interest in mind they have their best interest mind let's see here Fitness this is just a little example long ago my son and I started a project called Fitness this was about 2001 so 18 years ago and fitness was a wiki and it was also a testing engine and we decided to build it and our discussions began around how to store the data it was a wiki so we knew we had to store data somewhere and so we said well we've got to have some kind of persistent storage it's probably gonna be a database and it needs to be open source and what was the best open source database at the time it was my sequel so we said all right well let's get my sequel we'll get a license to it and we'll we'll work out the schema and somebody said in the midst of this discussion wait a minute we don't need to do that yet because there's another problem we can solve first which is the translation of the wiki markup language into HTML now that's a fairly significant problem we thought okay let's solve that one first and we'll defer the whole decision for the database until later so we began and I can show you the structure we began by creating a wiki wiki page object and then we implemented it with a mock and the mock was just a raw thing that allowed you to put data into the page we didn't store it or save it or anything like that just single page it had database functions you see that same function had a bunch more functions too but it had database functions that just weren't implemented and for about three months we worked on the problem of translating wiki markup language into HTML and when and by the way we were doing test-driven development so this was all about writing a test and making a pass writing a test and making a pass so we were writing tests and making them pass and by the time we ran out of tests to write for the wiki markup translation we said okay now we need to get the database because we need persistence we need to be able to link two pages to each other and that's gonna require persistence and somebody said well no it doesn't require persistence it just requires multiple pages and we could put those pages into a hash table and make it pretend like it's a database and we thought well that would be really easy to do so we created another version of the wiki page called the in-memory page which used a hash table to allow us to look up all of the wiki pages and now we kept on writing tests right tests making past write tests make him pass we could link pages to each other and we we actually got the testing engine running so you could run tests took us about a year but after a year of work we had the whole system working with all the data being kept in that hash table now that was interesting because we could demonstrate it to people but it was also deeply frustrating because you couldn't save anything turn the computer off and you lost all your all your data so in the end we had finished all of our work on the operation of the system and now it was time to do the stuff that really did relate to persistence and we said all right it's time to get my sequel out and Michael feathers was there at the time and Michael feathers said well you don't really need to get a database yet because all you need is persistence and you can get persistence by storing the hash table into a bunch of flat files and we all kind of crinkled up our noses and said yeah but flat files you know they're inefficient and they're awfully he said yeah but it'll work fine and then later on you can get the database in there and we said okay it'd be easy and he came back a day later with the whole thing running on flat files and we kept on working now we had persistence now we could save stuff make our tests make an pass write tests make and pass and we're building more and more of this system well now we can take it on the road we can demonstrate it to people it saves the data on in the flat files it's working great other people start to use it they think it's great a whole big user community begins to grow out they're all using flat files it's all terrific about three months later we looked at it and said I don't need my sequel at all duly we'll just forget about that part and we did we just dropped it off the whole project plan for a year then it came back and it came back because one of our users came to us and said we got to have pages in a database why CEO says we've got to protect our data assets well you can't fight city hall okay so we told him this structure we showed him this structure gave him the source code this is an open source project anyway we gate we showed him the structure said all you need to do is make a derivative of wiki page that implements the methods of wiki page to operate my sequel he came back a couple of days later with the whole thing working in my sequel we used to ship that as a plugin but nobody used it so we've dropped it he dropped it a year later - he says it was ridiculous there's a lesson here it's a very interesting lesson when people talk about architecture they often think that architecture is about making decisions early it's about making decisions late a good architecture is a software structure that allows you to defer critical decisions for as long as possible defer them to the point where you've got enough information to actually make them or not make them as the case may be architecture allows decisions to be deferred delayed until the last responsible moment this may fly in the face of what you think you may think no no you've got to make some of these decisions early well there are some decisions that need to be made early for example if you're going to write an application you probably need to decide what language you're going to write it in pretty early we made the decision to use Java for fitness that was a really early decision but the database decision in which we thought we needed to make early we were just able to push right off the end of the project the flat files actually worked better than a database would it was fascinating to us there were advantages to using flat files that none of us at ever conceived of before a for example you can check the whole testing engine into source code control that one we hadn't even thought of so there was a lot of very very powerful benefits to deferring decisions as late as you possibly can good architecture maximizes the number of decisions not made now what I do you're using IntelliJ right everybody uses IntelliJ anybody here using Visual Studio who did I talk to you about this before did I mention this before Visual Studio no do you use resharper oh okay resharper is to plug in to Visual Studio that's made by the same people who do IntelliJ and it makes Visual Studio almost usable resharper is made by JetBrains Visual Studio is made by Microsoft which of these two organizations has the power to completely screw the other Microsoft and why because Microsoft at a moment's notice can change the interface that resharper depends upon if Microsoft wanted to shut down JetBrains all they'd have to do is change that interface once a week sorry guys we changed that interface again we know you need to implement it but oh we changed it anyway is there anything JetBrains can do to Microsoft to make Microsoft not release Visual Studio yes yes I know that was JetBrains actual defense right JetBrains created their own IDE called Rider and hopefully Microsoft people will use it but some of them are so hopelessly dependent upon Microsoft I doubt that'll happen and I hope I have hopes riders a pretty good tool but is there anything JetBrains can do immediately without investing at home inventing a whole new IDE to stop Visual Studio from being released the answer to that is no so we have a really interesting asymmetry here small source code changes to Visual Studio can completely prevent JetBrains from making a release but there's no source code change JetBrains can make that will prevent Visual Studio from being released now forget about the two organizations let's talk about your application are there parts of your application that you would like to be robust against the change of something else some part of your application can change out here and this part of the application doesn't care at all like Visual Studio doesn't care what happens to JetBrains are there parts of your application that you would like to be protected from change out here and of course there are primarily your business rules you don't want your business rules to be affected by Oh save the GUI or the database you'd like to protect your business rules against those kinds of changes and by the way graphical user interfaces change for the dumbest of reasons did anybody get a change to your iPhone Oh a while back that just changed the whole way the user interface worked change the shape of the icons change the way that your gestures work some marketing people go around there you know we need to make it new and modern and so we're going to change the icons shape and move things around it's very frustrating if you're a user to have these marketing people think that they're making it better but anyway they changed everything in there do you hope that the application writers did not have to change all their app Cajun code because of the strange change that was made here and you see hope you do you hope that they had isolated those changes from each other how do you do that isolation you make the graphical user interface a plug in to the business rules like JetBrains is a plugin resharper is a plugin to visual studio you create a plug-in architecture how do you create a plug-in architecture you get all those dependencies pointing in the right direction if all the dependencies point at the business rules and everything on the outside of the business rules can plug in you can build it into a new jar file take the whole GUI put it into a jar file deploy it independently of the business rules plug it in and if you can plug the GUI in you can unplug it and plug something else in like a test engine test engine could use the same interface that the GUI uses or you could plug in a console interface if you wanted a console interface or you could plug in a service-oriented interface if do you in if the user interface is a plug-in you can plug any user interface into it you have to write it of course but you just plug it right in the business rules don't care the database can be a plug-in to the business rules and if the database is a plug-in to the disss business rules the database does not have to be a relational database the database could be flat files the database could be or root or Rinda or any of those crazy no sequel databases there's a question over here someone take that thing and throw it over there and my question is about how do you see the concept of enterprise bus in relation to the plug-in in architecture the enterprise bus I don't know how much trouble I want to get into suffice it to say this there are problems that occur inside software systems because of fashion it's a new fashion to have some interesting architecture like a service-oriented architecture that becomes a fashion statement and that started about 15 years ago maybe a little more than that and everybody had to go service-oriented and some companies by the way there's nothing wrong with being service-oriented that's an interesting interesting structure it's an old structure but it's an interesting structure some companies then get the idea that there's a marketing opportunity there and they make tools that claim to support that architecture you have to be very careful about that because these tools are generally not the grand solutions that they claim to be I'm not going to go much farther than that in my rant against tools on for service-oriented architecture and enterprise buses and so on if you are using an enterprise bus it's an i/o device all it's doing is facilitating communications between services it is an i/o device and you put it behind a boundary just like we put the database behind a boundary and the graphical user interface behind a boundary you don't let any of your business rules know that this bus is there and then you use the bus for what it's intended as a communication method between the various services in your system and by the way some of those buses are fairly elaborate they got nice little tools that allow you to route the messages here and there and very cool okay just from the point of view of the code inside the services there i/o devices keep them below a line No yes I know and I'd and there's a slide that I wanted to do but it's not in this presentation so I'll just do it frameworks part of the fashion that happens in our industry our various frameworks and we get very excited about a new framework there's a new framework out there we got to try that new framework it's called bananas and we have to come up with these weird names for frameworks right Soho yeah let's use the bananas framework that's fine it's okay to be excited about frameworks but you have to understand that frameworks are not made for your benefit right they're made for the author's benefit and there's a huge asymmetry when it comes to frameworks in order to use a framework you must make a huge commitment to them you're going to be coupling your code into those frameworks at a really detailed level the the instruction manuals for the frameworks encourage active coupling into the framework they want you to derive from their base classes they want you to insinuate their code into your code so that the coupling is tight that benefits the framework authors does not benefit you the benefit offers do not necessarily have your best interests in mind because they don't really care if you eventually get your application to a point where the framework is in the way now frameworks have this lovely ability to make the first months of a project easy but then they make the next years of the project difficult because you will want to do things that the framework is didn't ever anticipate and is in the way it gets in your way and it forces you to work around the framework so you will you will discover that as you use frameworks more and more and more and a good architect looks at every framework with a cost-benefit trade-off in mind how is it going to help me how is it going to hurt me and there's always a cause and good a good architect will put a little boundary between the art the framework and the rest of the production code that will hamper your use of the framework slightly but it allows you to decouple from the framework enough so that the framework can stay safe and unintrusive remember this always in order to use a framework you must commit to them they do not commit to you so it's a deep asymmetry and a problem that an architect has to face all the time who's using dependency injection why because it's cool now you're supposed to use dependency injection like that it was new while back and so we're all using it but why it makes a few things easier doesn't it right so what are using spring and and you use a little at Auto Wired's you have those little at Auto wires scattered all through your code if you look if you look through modules just it away don't you show what stuff scattered through your code the framework has insinuated itself into your code right can you imagine trying to get rid of that dependency injection framework what if there's a better one that comes out I don't know we're using this one because they add auto wire it is everywhere now what dependencies should you inject everything or are there a few key strategic dependencies a few factories a few strategies that you could inject into one of the peripheral modules of your system I like to call that main by the way the main module of your system which is behind a boundary you could inject a few critical dependencies into that one and then pass those dependencies around the system by normal means which I know means that you're working in the 1990s and it seems out of date and on the other hand it will keep that dependency injection system from infiltrating everywhere has anybody had the problem that when you do a build and the build fails it's because something in the dependency injection scripts and descriptors have gotten rude up and you don't know where it's just a long time to hunt through it no yeah there's this version in that version and that version and all that XML gets in the way keep that dependency injection small you don't have to inject everything you should not inject everything inject a few critical things and then pass everything else around normally and with that I think I will conclude a one more question it's time for one yes take that thing and throw it down to him right oh there that's better even about architecture what about the fourth dimension so things like concurrency and stuff like that that easily lures you into frameworks like reactive because they solve it how do you can you fix concurrency in this day and age without frameworks well I hate to say so how do you get you know how do you survive in a in this day and age without frameworks and and the answer to that is well you're probably going to have to use some frameworks but you always want to also be looking at the build by decision the build by decision is nowhere near as trivially trivial as you think it is right it is often better to build than to buy even when you're buying something that's free right so a framework might be free but the cost of that framework over say a two-year period might be more than the cost of building the solution yourself most of the stuff that we use frameworks for is actually pretty simple to build yourself because the framework author builted himself didn't really take that long anybody know how many lines of code and a webserver minimal webserver some things will just you know serve web pages it's about 200 lines of code if all you need to do is serve web pages you need 200 lines of code and you know if it's your code do you know how far down the security issues go if you wrote that code you know how far down all the configuration stuff goes if you wrote that code you know how much easier it gets to to manipulate that code if you wrote that how much harder it is for somebody else to break into that code if you wrote that code it's a lot of advantages to thinking carefully about whether or not you need a framework or whether you could actually build the solution yourself sometimes you the framework is the right answer but in our industry we make the framework decision in the wrong direction way too often and with that I will close thank you all [Applause]

Channel: UnityCoin

Views: 187,053

Keywords: programming, software, clean code, polite code, shrunk code, programming language, computing, technology, society, ethics, human relations, uncle bob, robert cecil martin, edsger dijkstra, grady booch, future, rules, java, c#, c++, microsoft, functions, declarations, arguments, cycle, kotlin, InteliJ, methodology, agile, scrum, tdd, test driven development, programmer, responsibility, expectations, architecture, design, development, applications, app, structure, web, study, practice, optimization, productivity, purpose

Id: sn0aFEMVTpA

Length: 130min 45sec (7845 seconds)

Published: Fri Aug 09 2019