ITkonekt 2019 | Robert C. Martin (Uncle Bob), Clean Architecture and Design

hello a lot of people here so I thought it'd begin this way can I have the Melissa's Millikan's up here Melissa how do you say this Melissa the - Melissa's the reason the reason you are all here are these two Melisa's can I get the two of you to stand like this both just like that just like oh good okay now I want you to think of this span from here all the way to here as the age of the earth right so if this is the age of the earth this is about four and a half billion years so one hand is about 250 million that would be 500 million and then a billion two there and then two billion two there and three billion four billion and a half something like that this is where I'm such a privileged son of a gun aren't I this is where the earth formed from the solar nebula and this is today where were the dinosaurs the dinosaurs formed here they died there dinosaurs very recent just within the last hundred million years or so very recent when did life crawl out of the oceans right there and during this entire period here there was no there was no life on land no plants on land no crawling life on land but when did life begin life began here and stayed in the oceans for three and a half billion years until it finally crawled out there yes thank you but and well I'd rather do it myself thank you until it came out here we course humans right there thank you very much both of you [Applause] so I'm here to talk to you about architecture and you and I have about an hour to talk about this so let's see how well this goes but before we do that do you have any idea how important software is let me give you a little clue here how much software is running on your body at the moment now let me show you how much software is running on my body at the moment of course I have my iPhone here how many how many processors are in this machine I don't know how many I've never done the analysis but I can make a good guess there's the main processor of course and there's probably a graphics processor there's a GPS processor it's probably a cell phone processor so at least four and then of course the case has a processor in it because it's got the battery charger Bluetooth connections stuff like that so a bunch of processors in this thing then I've got this these are my iPods or earpods whatever these things are called and and each one of the little ear plugs has a processor in it of which there are two and then of course the case has a processor in it and of course I've got my Apple watch and that's got a processor in it and they've strung me up to this device here that you're listening to me through there's a processor in here of course I've got a load of processors just on my body now how many do you have on your body how many do you have near you and then let's just ignore all that ignore all those computers how many computers are running in the walls of this room what about the speakers do the speakers have processors in them as their software running in each of the speakers probably there's probably software running in each speaker because nowadays it is cheaper to filter out 50 psycho hum by using a little digital signal processor instead of using a capacitor and an inductor in normal circuitry and that little digital signal processor would probably have some C code running in it doing fast Fourier transforms written by some 22 year old at 3:00 in the morning how about how about the cameras back there they clearly have software in them how about the exit signs do the exit signs have software running in them well if there's a battery and there ought to be for an exit sign then there's probably something that will charge the battery and that's probably got a little digital signal processor in it so fine there's there's software hidden in the walls of this room but now go out on the road how much software is running out on the road and each one of those cars a modern car and I'm not talking about a Tesla a modern car has a hundred million lines of code in it that should scare the hell out of you and most of that code of course is in the entertainment system and in the GPS system but some of that code actually controls the motor and controls the brake you'd like to think that when you touch the brake pedal that there's a steel cable that goes all the way to the brake callipers maybe through a hydraulic system to to make the brake work but nowadays when you touch the brake pedal it sends a signal to a computer and that computer decides there's an if-statement in there how many people have been killed because that if statement failed and the answer to that is several dozen people have been killed by the failure of the software in the cars there's a big payout to a number of companies Toyota had to make a big payout because they killed a several dozen people and the reason they killed these people is that the software that controlled the engine failed and the car would accelerate out of control and the brake wouldn't work you and I are killing people the software you and I write has the potential to kill the people now you and I did not get into this business because we want to kill people most programmers got into this business because they wrote an infinite loop once that printed their name and they thought oh yes I want to do this for the rest of my life but now we're killing people and we're losing fortunes who heard the story of night capital anybody heard that story night capital yeah a few of you've heard the cab I'm not going to tell you the whole story let me just tell you that some poor software programmer did one dumb thing and lost four hundred and fifty million dollars in 45 minutes four hundred and fifty million dollars and forty five minutes gone in a flash we are killing people we are losing fortunes your grandmother if you still have one very likely interacts with a software system every minute of every day because in our society in the Western world it is almost impossible to do anything without interacting with a software system you can't watch TV without interacting with a software system you can't make a phone call you can't use the microwave oven if you've got one you can't use the dishwasher if you've got one you can't use the sewing machine if you've got one or the or the one machine or the dryer you can't drive a car you can't take an uber you can't take a lift you can't take a taxi you can't buy anything you can't sell anything you can't get an insurance claim you can't buy a policy you can't make a law you can't enforce the law nothing can be done in the Western society without interacting with a software system you and I rule the world other people think they rule the world but then they write those rules down and give them to us and we write the rules that execute in the machines that make everything in our society happen our society does not rightly understand yet just how important we are just how much they depend upon us and we programmers don't quite understand that either every once in a while someone will get a hint that oh maybe these programmers are actually pretty important you think the the airlines flying the 737 max's are thinking maybe the programmers are a little important right now how about the guys at volkswagen who tried to cheat the epa the california EPA by omitting by cutting down their emitting their emissions whenever they were on a test stand right they thinking software's pretty important right now so here's what's gonna happen one day some poor software person is going to do some dumb thing and kill 10,000 people at a shot boom and you know this is going to happen and you can imagine what it might be it's not that difficult and when this occurs the politicians of the world will rise up in righteous indignation as they should and they will point their fingers right at us and you'd like to think oh no they're not going to point their fingers at they're going to point their fingers at my boss they're gonna point their fingers at my company no they won't we saw what happened when the fingers pointed when the CEO of Volkswagen North America testified before the American Congress and the American congressman asked the CEO of Volkswagen North America how could you have let this happen how could you have cheated the California EPA with software and the CEO of Volkswagen North America said it was just a couple of software developers who did it for whatever reason and the finger came right down where it belonged right at the programmers and with some justification because it is our fingers on the keyboard it is our minds writing that code we are in fact responsible and so when the politicians of the world point their finger at us can ask how we let 10,000 people die at a stroke we'd better have an answer for them because if our answer is oh you know my boss told me it had to be done on Tuesday if that's the answer then the politicians of the world will rise up and say well you guys are out of control and we need to control you we need to legislate we need to pass laws and they will tell us what languages we can use and what platforms we have to use and what books we have to read and what processes we have to use what signatures we have to get and we will become a heavily regulated industry I'd like to get there before they do I'd like for us you and I to figure out what our ethics are what are the standards that we don't go below what are the ethical rules that we follow what are the disciplines that we adhere to and how do we enforce them and I want this to happen so that when the politicians of the world point their finger at and ask us how this disaster could have happened we can say look this was an accident it was not due to our negligence here are our disciplines here's our standards these are our ethics this is how we enforce them an accident yes but not because we were negligent and we could not say that today this may be the best time in the world to be a programmer we have the maximum effect upon the world it is a time of incredible demand we cannot seem to get enough programmers look at how many programmers are sitting here in the room there's seven hundred some-odd programmers sitting here in this room it's not enough we need more and more and more when I started programming which was 52 years ago how many programmers do you think there were in the world fifty-two years ago how many programmers were in the world no not 52 thank you good answer it would have been a better answer if you'd said 42 I don't know how many there were probably a little more than 10,000 probably a bit more than 10,000 but not an awful lot more than that but let's go back a little bit further in time let's go all the way to 1946 1946 how many programmers were there in the world one Alan Turing he wrote the very first code that ran on the very first electronic computer the automated computing engine in 1946 the day after that when there a whole bunch of programmers but he was the very first one how many programmers are there now in the world today how many programmers are there well that's definitely millions but the question is whether it's tens of millions or whether it might be a hundred million it could be a hundred million it could be a hundred million programmers in the world it depends on whether you count the VBA programmers but it could be a hundred million program I'll let's just assume that it's a hundred million right that's that's ten to the what ten to the eight hundred million forties in 1946 it was one today it's a hundred million that was seventy three years ago in seventy three years we have grown from one to a hundred million now what kind of growth curve is that is it linear it's obviously not linear it must be exponential all right well if it's exponential we can assume that the base of the exponent was two and that would give us a doubling rate so how many doublings does it take to get from 1 to 100 million in 76 years how many doublings does it take just to get from 1 to 100 million well a million is 2 to the 20th so that's 20 doublings and 127 is 2 to the seventh so maybe twenty seven doublings twenty seven doublings from alan turing to today would give us a hundred million programmers and twenty seven doublings took place in 73 years you can do the math how many doublings per year is that how many how many years per doubling is that it's about two years there's a number of programmers in the world double every two years probably not because that first decade there were more doublings than that right the first day it was Alan Turing two days later there were 10 programmers a month later there were 30 programmers so in that first decade the doubling rate was much higher I believe right now the doubling rate is something on the order of five years every five years the number of programmers in the world doubles and this does not seem to be stopping now at some point it's going to have to stop because we're gonna run out of people but great cuz a hundred million how many more do you have to go to get to eight billion not that many were already at like point 8 percent of the population so that's pretty interesting do we really double every five years all the evidence seems to suggest we do but that has a fascinating implication it means that half the programmers in the world have less than five years experience and this will always be true as long as we are doubling at a rate every 5 years half the programmers in the world will have less than 5 years experience which keeps our industry in a state of perpetual inexperience we cannot grow we cannot learn intellectually we cannot grow because we are growing in population so fast this is a good time to be a programmer because the demand is still high the doubling rate is still high rejoicing the fact that you are a programmer it's a very good career to have and you will have a deep impact on society the society now depends on you for its existence doesn't quite understand that yet but the Dave the day is coming and therefore one of the things that we should learn how to do well is to structure systems the days are past when we can just kind of throw systems together and hope they work we're going to have to learn how to do systems well so let's talk about architecture what you're seeing on the screen is the directory structure of a system that my son gave to me Oh about eight nine years ago he wanted me to review it for him and I looked at it and I said ah well that's a rails application I knew it was a rails application they'd seen rails applications before who's done some rails out here Ruby on Rails look at that look at that they're all in one little cluster over there Ruby on Rails big popular framework in the United States and Western Europe everybody wants to do Ruby on Rails all the social networking companies want to do Ruby on Rails or at least they did five years ago not so much now because you know in software everything comes in fads and popularity swings and you know every five years you've got a brand new cohort of peak of people joining the joining the the population of programmers and they all want to do something new so here they are we got an a rails application and I could recognize it just by the directory structure and I'd seen many rails applications before but this time something different occurred to me and what occurred to me was that I could tell it was a rails application why could I tell it was a rails application why is the first thing I see rails that bothered me that wait a minute the first thing I should see ought to be the reason the system exists you know why is this system here what does the system do but that's not what I saw what I saw instead was the framework and that bothered me and I thought about it for a little while and I realized that wait a minute Ruby rails rails is a web framework it's a framework about getting data on and off the web and the web is a detail and that was the first time I had put that idea together the web is a detail now that's hard for us to conceive of right because the web seems to dominate us the web is everything we have to have a web system we've got to do web stuff web web web web web web but the web is an i/o device that's all it is an i/o device we can display things on a screen we can get things from a keyboard the web is an i/o device and the one thing we learned way back in the 1960s was that we didn't want to know what IO device we were using we wanted our systems to not care what IO device they were operating on and therefore we invented things like device independence who's done some UNIX programming out here anybody you've done some UNIX right and if you're gonna write something out to an i/o device you don't write it out to that i/o device you write it to standard out and standard out as an abstraction and the software doesn't know what it's writing to later on when you execute it you tell the operating system oh by the way operating system standard out is that printer over there device independence became very important it was the way we protected ourselves from changes in i/o devices and here we are and the first thing I see on that directory structure is this i/o device and I shouldn't be seeing the i/o device at all because the web is a detail it's an i/o device and so I I started to consider architecture and I got a few building plans out some blueprints of buildings and I realized that the building designs like this one here we're not telling me what they were built out of they were telling me what they were for this is a library you can tell it's a library because it's got bookshelves and it's got desks and it's got little computers and there's a den tree over there it's looks like a library right and if that one doesn't convince you this one will that's a church the shape of the building is screaming at me what its intention is I am a church I am there to gather people together or I am a library I'm there to hold books it is screaming at me what its intention is and that's what the architecture of a building ought to do and therefore I thought that's what the architecture of a software system ought to do it ought to scream at me what its intention is what it's for what its requirements are what its use cases are architecture is about intent the shape of the system the way its components are organized the dependencies between those components is about the intent of the system not about what it's built around now I'm not the first guy to realize that there was another guy who realized that a very long time ago this guy's name is Ivar Jakob Singh and in 1992 Ivar Jakob sin' wrote a very famous book this one object-oriented software engineering has anybody read that book to anybody out here read that book there's one two three four sorta well that seems to be it four maybe five okay the room of 700 people five of you have read that book that's a problem find that book buy that book read that book you and by the way just offhand there are a lot of extremely good books that were written before the year 2000 that you ought to buy and read and know who's got the design patterns book oh look at that who's read it okay get the design but this by the way the design patterns book is probably the most important book on software written within the last 30 years get that book read that book understand that book don't let anybody tell you that it's gotten out of date it hasn't gotten out of day don't let anybody tell you that oh well now we're doing XYZ programming and an XYZ programming design patterns don't matter don't let anybody tell you that that's nonsense get that book read that book understand what those patterns are so that you can speak intelligently about software design how many of you have read the structure and interpretation of computer programs got one there that one there back there I see just a few of you the structure and interpretation of computer programs that book will change your life it's got so much code in it and it will just change your life it'll change the way you think about what code is and how it ought to be structured anyway this is Ivar jakob s'en Ivar jakob s'en wrote this book in 1992 and I want you to look at the subtitle of the book the subtitle of the book says it is a use case driven approach does anybody out there remember use cases look at that some people remember use cases good the rest of you don't well because we double every five years and it's been 25 all right so use cases now how many of you have heard the word users story yeah that's even more people right okay and then there are people who haven't heard that word either which makes me wonder but if you have not heard the word use case but you have heard the word user story hey users a use case is a user story same thing and what Jakob s'en said was use cases are significant architectural elements they are the things that tell us the intention of the system so they are important architectural elements in the shape of our software this is what a use case might look like just a simple description of one small requirement in this case it's a create order use case and notice that right at the top there it specifies all the input data notice also that it doesn't specify the input data in any detail it's just kind of a high-level abstraction of the input data it says Alizee customer ID I don't know what form the customer ID is in but it's there and there's a customer contact info don't know what the form is that's okay shipment destination shipment mechanism payment information just just the names of the data without any details and the reason there aren't any details is that this is written at a time when the details are not important one of the biggest biggest mistakes the programmers make is that they assert detail far too soon programmers our detail managers did you know that you were a detail manager that's your job your job is to manage the most hideous details that anyone has ever managed ever before how many of you within the last year have written an if statement that checks to see if a text file has lines that end in backslash R or backslash n who's written that if statement in the last couple of years now that's just the most hideous possible detail but you and I have to deal with stuff like that we deal with all the worst possible details you and I are detail managers but we make a mistake we get ahead of ourselves we assert the details too soon so here's a use case where all the details are abstracted away we're not going to think about them yet we'll think about them later just not yet and then after the input data we see the primary course and the primary course shows the steps in the procedure for processing this use and the steps are fairly simple order clerk issues create order command with the above data that's actually not even part of the use case that's just how the use case gets kicked off and the system validates all the data I don't tell you how it validates all the data that's a detail we'll deal with that later but we do have to validate it system creates an order and determines the order ID of course I don't tell you how that happens probably it's a database activity but I'm not going to say that here because that's a detail system delivers order ID to clerk probably on a webpage but I'm not going to say that here because that's a detail it's a nice concise description of the inputs and the outputs and the processing steps without mentioning any of the details that's what a use case is and then Jakob sinned in his book made this observation he said that is an object a use case is an object you can write the code for it even though you don't know the details yet you can write the code for it because we programmers deal with languages that can abstract things so we can write very abstract code real code that would really execute that would deal with the abstract notion of a use case now I like to call the objects that represent use cases interactors people have said what Bob you should have called them use cases and maybe that's a good point but I didn't I called them interactors so okay I'm stuck with that name jakob s'en called them control objects and that's a problem because of model-view-controller people get confused with model view controller so I thought it would be better not to call them control objects an interactive is a business rule but it's a very special kind of business rule it's an application-specific business rule there are two kinds broad category two kinds of business rules there are the business rules that have nothing to do with the automation these are the business rules that the business would still use even if there were no computer so for example let's say you're doing a banking system interest calculations would be done whether or not you've got a computer right it doesn't matter so those are application independent business rules but then there are other business rules that are all about the automation and use cases are all about the automation there application-specific they're not independent of the application the only reason they exist is because we are automating the second kind of business rule Jakub s'en called an entity entities contain the application independent business rules the interest calculations the rules that would be done even if there were no automation and the interactors and the entities are kept separate because it's necessary to keep that kind of stuff separate in your code you don't want to have application specific business rules that are coupled tightly to application independent business rules you want to maintain that separation and then finally Hawkinson said you got to get data in and out of those things so we're gonna create some boundary interfaces and if you look carefully at these boundary interfaces you'll see that there's two different kinds of arrows they're the open arrow is a using relationship the closed arrow is an inheritance relationship or an extends relationship or an implements relationship the two boundaries are interfaces very much like Java style interfaces what languages do you guys using who's using Java look at a lot of job of people who's using a c-sharp well a lot of people using c-sharp - same language by the way right Microsoft just stole Java just outright stole it does anybody remember visual j+ + anybody long long yeah see people they're really old they remember visual j+ + right and then and then Sun Microsystems said hey Microsoft you can't do that that's ours Microsoft said oh you know you're right and they changed the keyword into a colon and turned it into c-sharp same language now they've diverged since a little bit although even today you can put up a page of Java and a page of dotnet and not see the difference so what else do we have by the way who was doing Python look at that got some Python programmers yeah and we already talked about the Ruby programmers over there who's doing F sharp nobody doing F sharp functional language how about Scala there's a couple of Scala people oh good functional language okay now the interesting ones who's doing Objective C look at that iPhone people way out of date who's doing Swift look at that same people huh fascinating anybody didn't go who's gonna go oh you get to go people big cluster of them right here go hey I'll give you a little hint about go if you've been doing Java or.net for a while you have gotten used to that quarter second startup time of the virtual machine you tell it to go and you kind of there's a little heartbeat and then it goes right and when you're doing go-go Lang that's not there that's not there you tell it to go back and you don't have time to even think Eddie who's doing test-driven development let me see hands test right - oh we need to work on you guys you're not doing test-driven development anywhere near enough come on that's a discipline we have to do that discipline otherwise when the politicians point their finger at us we're not gonna have any excuse did you guys write tests we know we were to post you because my boss said it had to be done really really soon and you know I I did write like one test but I lost it you're laughing aren't you yeah okay yeah well you have to come back and work on you so those of you lucky people who have done test-driven development if you have done it in Java you will know that you push the test button and then there's this little pause and then you see the green bar sweep across and life is good and everything passes and when you do this and go you push the test button go is one of those languages that lets you see how fast the machine actually is Java and c-sharp hide that from you and if in a in a kind of sad way so keep that in mind anyway that those boundaries are interfaces in whatever language you want Java c-sharp whatever those are the interfaces and you can see how data can come in through that lower boundary that lower boundary would accept data in and then it would be implemented by the inter actor so that's an inheritance relationship and the top boundary that's an output boundary the inter actor uses that notice also that only the inter actor talks to those boundaries right the entities don't know about the outside world the entities just believe they live in their own little world they are the pure business rules the inter actors are the ones that coordinate the execution of the entities coordinate the the acceptance of input and the delivery of output now we're gonna walk through a typical scenario we have little dude down there and he's about to push the submit button on a web page I didn't tell you it was a web page I called it the delivery mechanism it might not even be a web page maybe it's some other kind of interface maybe it's a console interface or maybe it's a desktop or maybe it's a service-oriented architecture port who's doing service-oriented architecture look at that right who's doing micro services Oh same thing guys same thing micro services service-oriented architecture and you got to ask yourself why you're doing this why are you doing Microsoft well because it's cool does it make everybody else is doing it so we have to do it to make sure you understand why you are doing micro services I won't say any more today maybe later anyway here we are this is the delivery mechanism probably the web and the user hits the submit button and now the delivery mechanism gathers the data from the input device this is probably the web controller by the way gets all the data from the form or whatever and it moves that data into a data structure that I call a request model that data structure is a raw data structure it doesn't know anything about the web it doesn't derive from any framework if you're doing java this is a plain old java object there's no dependencies at all it doesn't have any methods it's just a bunch of public fields it's a data structure and it gets passed into the input boundary so that the interactor can see it the interactor begins to begins to inspect it it understands what the what its interaction is supposed to be by looking at the interact at the request model and then it controls the dance of the entities it feeds data to all the application independent business rules the entities makes them do their work and then when it's done with that it reverses the flow and starts to pull data back out of the entities these are the answers that we want and it gathers those answers together and the interactor builds a result model the result model is just a raw data structure so if you're doing dotnet it's a plain old C sharp object there's no dependence on any kind of framework or anything like that just a nice little data structure that data structure gets passed out through the output boundary through the delivery mechanism where it's displayed on the screen to the user simple request response interface nicely isolated so that the application independent business rules are separated from the application dependent business rules and there's a nice boundary and then that is attached to the delivery mechanism which can't know anything about the other side can I test the interactor can I test that can I test it without the web server running could I test that interactor at 30,000 feet over the Atlantic sipping a gin and tonic on my laptop well he's spilled gin and tonics on your laptops not a good idea but I could I could test that interactor without any of the rest of the system running this is always a goal by the way you want to be able to run tests on parts of your system without the rest of the system running you want to be able to test your systems in parts and this allows me to do that now what about model-view-controller you've all been doing model-view-controller for god knows how long what does it look like this is the guy who invented model-view-controller back in the 1970s his name I'm going to butcher because I can't speak Danish is a trig Vereen scout now that's a butchered pronunciation buts the best I can do I met him once I was at a conference in Norway and I went up to the speaker lounge and I was looking for a place to plug in my laptop and this grizzled old guy hands me a power strip and I reach out and I took the power strip and I looked up at him and the blood drains from my face and oh my god that's trig Vereen scout and as I took this from him our fingers brushed just briefly and I don't wash those fingers way back in the 70s trig virion Scout came up with this simple idea you've got a model that's a just a business rule a real simple business rule and it doesn't know how it gets its input and it doesn't know how it gets its output it's just a business rule and there's a view up there and that view knows how to display the contents of the model out to the the outside world somehow and there's a controller down there and the controller knows how to accept input from the outside world and hand it in to the model as simple input process output design pattern and it when we were doing this we used to do this in small talk that's where it was invented for the language small talk and when we did this we would have model view controllers for every little widget on the screen there was a Model View controller for a button Model View controller for a check box Model View controller for a menu we did not have a model for the whole screen we just had little individual models for all the little bits this was a design pattern that we used in the small not in the large now this has gotten bastardized over the years so that now it looks something like this this is how we do model view controller on the web you probably have Model View controller web web frameworks that you've used and and you've got these controllers out there and they take all the input from the web by some magic and then they fire off a bunch of commands to the business objects where the which are the models and then eventually they hand control over to the views and the views gather all the data out of the business objects and and put it on the screen and the problem with this of course is that all those lines cross and programmers forget where they're supposed to put stuff so you'll wind up with with view like code in the business objects and you'll wind up with business object like code in the controllers because there's no firm boundaries so here's the model I prefer this is the the Jakob Sony and interactor model and what we're going to explore here is how the data gets out to the web or to whatever your i/o device is so it begins at the interactor here the interactor has created the response model and you can see that there and the interactor is going to send that response model out through the output boundary now look what we look what implements the output boundary there's a little object there called a presenter the job of the presenter is to take the data in the response model and reformat it so that it's displayable if you've got a date in response model that date is just a date object right by the time the presenter gets it it will convert it into a string and put it in the view model and that string will be formatted for whatever your locale is if there is a money object in the response model it'll just be a raw money object by the time the presenter gets that it will convert it into a string with the bright decimal places and the right currency marks and so on if there's some kind of a grid that you want to display the presenter will organize the data from the response model into the view model in a grid form if there are buttons on the screen the presenter will know the names of those buttons and will put those names as strings into the view model if those buttons need to be gray the presenter will make sure that there's a boolean in the view model that says gray out this button any decision at all that can be made about the screen is made by the presenter and it's loaded into get into this view model data structure which is just another data structure which holds mostly strings and then the view is this faded thing that barely exists all it does is copy the strings onto the screen and walk away can you test that presenter you can't you pass it a request you pass it the response model and you look at the view model do you have to have the web server running while you test it no you don't you can test the system in parts test the system in parts this is kind of what the whole thing looks like from the interactor over got controllers off to one side they feed the request models into the inner factors through the boundaries don't I don't show the entities out here but eventually the interactor feeds the response models back to the boundaries out to the presenter and you see this double black line there that's an architectural boundary that's an architectural boundary and notice that all the dependencies cross that architectural boundary in one direction that's the key to an architectural boundary whenever you have an architectural boundary you want every dependency to cross it in one direction and the direction itself is critical all the dependencies point in the direction of the higher-level policy the stuff on the left the controller and the presenter is low-level stuff detail stuff the stuff on the right that's high-level stuff that's business rules look at the business rules over to the right they don't know anything about the details they don't know anything off to the left there I have no dependencies going that direction you could take those business rules and put them in a DLL or a jar file and deploy them independently of the web the web which is off to the left that's a plug in that's a plug in to the business rules and if you can plug the web into the business rules you can unplug the web from the business rules plug something else in if you want some kind of a console interface you don't want to plug a console interface in plug a service-oriented architecture interface in you can plug anything you want into that but you know what you're really gonna plug into it a bunch of tests because that's all you're also your tests boundary that's where you're gonna be writing integration tests into the system yeah but what about the database there I've got 11 minutes I can talk about the database might take me a little longer than that anybody have this view of the database the database is the great God in the center of our system all hail the database all other things bow down to the database the database is everything this is the wrong view we don't want to think of the database this way because the database is an i/o device the database is an i/o device it's a place we stick data it's not some great center of the system who would benefit from telling us that the database is the great God in the center of the system who would most benefit from that well you say the DB ace and you're right of course the DBA is do we have any DBAs in the room okay I'm safe good of course the DBA stew but it's not the DPS because who created DBAs in the first place does anybody know where that Job Description came from who created the very concept of DBAs database companies the people who sold databases to companies came up with the job description of the DBA they told the people they sold the databases too you know you got to have people who can administer these databases too and they have to have control and they have to have all the passwords and by doing so the database companies created a very loyal little army of people that held all the keys now what do you think the argument was how do you how do you convince a bunch of executives that they've got to buy a really expensive database get this right Salesman goes into a CEO all right are you protecting your data assets what a great word data asset and CEO goes I didn't know I had any data assets oh yes you do and our system will protect your data assets all you need to do is buy our system was a wonderful scam and it took place in the 70s and the 80s and it pervade databases everywhere and it made them the center of systems except for those architects and designers who knew all along the databases were actually details and i/o devices and we should not be we should not be centering our systems around them and so what do we do well we put them behind in architectural boundary there you go there's the database down there that red block down there we got some nice architectural boundary crossing the screen from left to right there's the interactor the interactor knows that it has to interact with something that stores the data it doesn't know that it's a database has no idea like doesn't know that it's Oracle or whatever you might be using it just knows that it has to interact with something that saves data and it does that interaction through that interface they're called an entity gateway the entity gateway has methods one for each query you want to do so for example let's say you want to query all the employees that were hired after 1972 because they're old now and you need to fire them right so you're gonna query the database for all the employees hired before 1972 you would have a method in the entity gateway which said get employees hired before and then you'd pass in a date and you do that for every query in your system so that your entity gateways lists every possible query operation you need to make then you implement that down below the line and the database down here becomes a plug in to the business rules where's the sequel anybody out there write sequel who writes sequel I'm sorry it's a shame that you have to write sequence equals sequel sequel is a fine language if you are sitting at a console and you want to query a database it is not a good programming interface it possibly is the worst possible programming interface because programmers don't want to pass strings into an API they want to call functions into an API and wouldn't it be nice if all of our databases had an actual API that we could call functions on instead of trying to construct horrible select where strings that fits some nasty syntax and so what do we do we make our RMS we get the whole problem away from us we use ORM s where is the ORM in this down here no ORM is above the line no sequel above the line nothing above the line except the business rules if you're using an ORM you put it below the line but you don't have to use no RM of because you could implement that entity gateway with anything you want you want to use some no sequel database you could use a no sequel database right there business rules aren't going to know oh and by the way if you're running tests you can rip the day to be out database out completely and just put a bunch of stubs down in there and your tests will go really fast I'm going to tell you a little story and after that story will be done and then there will be some time for questions the story goes like this this is the story of an application called fitness has anybody used fitness Oh several of you have good good to hear Fitness is a wiki that my son and I wrote 20 years ago it's also a testing tool it's for integration level testing it allows businesspeople and QA people to write tables on wiki pages and those tables specify both input data and expected output data and then the tool figures out how to call the system being tested with the input data and compare the output data and we get nice little tests that way now I'm not going to describe that in any more detail I want to talk about how this was done we sat down the day that we decided to do this we sat down and said ok this is a wiki and since it's a wiki we're going to need a place to store wiki pages that ought to be in a database well what database should we use and we said well we want this to be open source and there's only one open source really patient relational database we can use so let's use my sequel we said ok we'll use my sequel and we realized ok well that means we need to come up with the schema of the database and we have to get a license for it we got to power it up and figure it all out and somebody said well wait wait you don't have to do that yet because there's another problem you've got to solve first and that problem is you've got to translate the wiki markup language to HTML now that's a big problem all by itself and we thought ok let's do that first and so we came up with this mock jakirah this wiki page interface which had a to HTML method and a save method and then we created a mock of that and the mock would do the conversion from from wiki markup to HTML it allowed us to do that but it marked out the save method and for about three months we just worked away now we were doing test-driven development so you know little tests little code little tests little code little tests little code for three months getting all this stuff to work and we got it all to work and we can convert wiki text into into HTML that was great and finally we said okay well now we've got to be able to save these pages somewhere so let's go get aged let's go get my sequel somebody said well you know we don't need to get my sequel yet because we could just take the pages and stick them in a hash table in memory and that would kind of act like a database and so we thought you know that's not a bad idea so we created this in-memory page derivative of the wiki page and it just put all the pages in the hash table in memory and that way we could keep writing tests making a pass writing test making a pass now of course we had pages that we could save temporarily and we could implement all kinds of very interesting features we actually worked a year that way we got the whole system working all the wiki stuff and the linking stuff and the multiple pages in the hierarchy of pages and the testing application stuff all that stuff worked but we couldn't save anything which was a bit annoying towards the end of this period we said all right look this business of keeping everything in memory is getting old we need to be able to save this stuff let's go get my sequel and Mychal feathers was there at the time and Michael feather said well you don't really have to do that yet because you could just take that hash table and write it out to flat files that'd be really easy and so he actually came back a day later with the whole thing written out into fat files and we just kept on working that way might a little test make it pass right it has to make it pass all of a sudden we had persistence we took it on the road we showed people hey this is fitness and people started using it and more and more people started using it and we were using it and got more and more popular about three months later we were we didn't need that database at all we never put that damn database in there well year later we did a year later right a customer comes to us and that customer says we gotta have it in the database and we said well why right cuz it works fine now it's fast it's easy and you can actually check your pages in the source code control had to be really tough to do with the database so it's better this way isn't it yeah it's better but we gotta have it in a database anyway because our CEO says we've got to protect our data assets and we said well I guess you can't fight city hall so we showed him we showed him the original design here which is this design that one and we said well look all you have to do is make another derivative sort of like file system page call it my sequel page if you want just implement those functions in there and you'll be able to get it into my sequel he came back a couple of days later with the whole thing working in my sequel we used to ship that as a plugin but nobody used it so we stopped he even stopped a year later because they changed management and the moral of that story is this we took a what you would think of as a critical architectural decision the database itself and we deferred it and we deferred it and we deferred it right off the end of the project a good architecture is an architecture that allows big decisions to be delayed and deferred not made delayed and deferred you defer them until the last responsible moment the moment when you've got the most information to make or not make that decision you do not want to make all these decisions early again this gets back to the whole idea of too much detail too early push those decisions out which you can do by controlling the shape of the system and the direction of the dependencies in that system a good architecture minimizes the number of decisions not made maximizes the number of decisions not and with that I'll open this up for some questions got anybody got any questions do we have microphones for the people who have questions looks like we do yeah runners for the microphone so who's got a question there's one hi hello I'm interested in which parts of the application we need to do unit tests to do test-driven development which parts and what about code coverage which parts of the application do you need the unit test and what about code coverage you only test the parts of the application that you want to work and what's the coverage what kind of coverage should you have who's got a limit who's got a leak on a goal coverage goal maybe like 80% got an 80% coverage go we've got it that's that means it's okay if 20% of the system doesn't work there's only one coverage goal that makes any sense that's a hundred percent you can't achieve a hundred percent it's an asymptotic goal but it doesn't mean you ever stop trying so you're always trying to push closer and closer to 100% now I was a little bit flippant there I said earlier on that you only test the things you want to work okay that's true but there are parts of systems that are very very difficult to test in particular those are gooeys and GUI is just a example of other kinds of things that are hard to test anything at the outer boundary of the system it's hard to test because you can't get code on the other side of it I can't get code on the other side of the pixels on the screen I can't test the pixels on the screen easily there are strategies for that but there but hardly worth doing so my particular discipline is to get as much code out of the GUI as possible test it and then the last few bits of code that touched the screen I simply don't test at least with unit tests I test them with my eyes a good example of that is CSS do you test your CSS right and and I don't write because I just sit there and fiddle with the CSS until it looks right and that's probably about it so you cannot test things that at the very edge of the system and it's also very difficult to test things if you don't know the answer if you've got a function in the system that you need to implement but you don't know what it's supposed to do it's really hard to write a test so for example let's say that you're doing a big finite element analysis here you're analyzing the air flow over a wing and the very purpose for this is to get the answer you don't know the answer you don't know what the air over the wing is so you can't write a test for this but you can write tests for all the little bits you can write a test for each finite element you can make sure that the airflow over a small rectangle or a small triangle behaves properly you just can't test the larger thing you're gonna have to trust the larger thing anybody else question looks like we got one over here six minutes left hello hello so how do you communicate intent with the folder structure how do we communicate intent intent yes of a software that we are developing with a folder structure how do you communicate the intent of what you're building with a folder structure going back to my very first slide exactly wouldn't it be nice if there were a folder up there called use cases wouldn't it be nice if in each one of those in that use case folder there was a class for each use case in the system wouldn't that be nice that would be one way to do that wouldn't it be nice if we had a folder for all of the entities all of the application independent business rules and we could see each one of those that's what I want I want to take those objects and promote them to the top level so that at the top level of the system you can see the use cases and the entities and know what this system does and the the one I showed you that one that was rails what you saw at the top of that was rails this is a rails app you didn't know what it did you could have found out by looking deeper and deeper but I don't want to have to look deeper and deeper I want to know at the top this is an accounting system next we got questions behind you there there's one back there there's a couple over there doesn't want to let go of the microphone do you want to keep that microphone don't ya huh just continue a follow-up question yes yeah because there are limits to the frameworks use or the languages used yes so how they'll go around them how do you go around it so how do you deal with the framework let me tell you a little little thing about frameworks yeah I love given the camera guy a hard time right he's got attract me and I'm a tough son of a gun to track so that guy sweating more than I am frameworks are wonderful things you can get a lot done with the framework you get a framework like spring or rails or JSF or JPA any of those neat little frameworks wonderful angular lovely things but the author of the framework does not know you and does not care about you the author of the framework does not understand your problems is not writing code to solve your problems they are writing code to solve their problems when you use a framework you make an enormous commitment to that framework you bind your code to it in the very worst case you derive from base classes in the framework the framework author makes no such commitment to you it is a terribly asymmetric relationship and as an architect if you are an architect or if you're a lead designer you must be very skeptical of that relationship because you should be thinking that this framework is out to screw me you should be looking very very carefully at how much you bind to it read the directions you know the documents that the framework author writes read those directions but you do not follow those directions because the directions that the framework author writes will tell you to couple to the framework intimately because he loves the framework you don't have to so then what you do is you put the framework behind in architectural boundary you draw those black lines again then you put the framework behind that boundary and you use it but you use it in a very careful way so that one day when that framework no longer satisfies your needs you will have some options how many of you have used a framework built it into your system coupled it so tightly into your the system only to find out two years later there's a much better framework out there but you have no way to get it in there because you are so inextricably bound how many of you have had this problem you start out with a framework and it's wonderful it's wonderful it lets you go fast but then a few months later you want to do something that the framework doesn't want to let you do and the framework starts to fight you and the more you want to do these secondary and tertiary things that aren't what the framework allows the more it fights you and in the end the framework costs you more than it gives you you have to be careful about frameworks they're lovely things they're wonderful things they are not pure good they are tools and like any tool you better know how to use them and how to keep yourself at some distance from them how many of you doing dependency injection oh look at that everybody's doing the pendency injection that's cool how many of you inject hundreds of things into your applications Oh a couple of year doing that who's using spring well who's got at Auto Wired's everywhere I had Auto wired here and there scattered through your coat I don't want to see that stuff I don't want to see that Auto wired and all your business objects right because if you put those in your business objects you have coupled your business objects to spring why does why do your business objects have to know anything about spring you want to use a dependency injection framework that's fine but inject the dependencies into a safe place behind an architectural boundary and then pass those those objects into the rest of your system through normal means and probably you should not be injecting any more than a dozen things got 42 seconds what is your attitude to point weight it's good about that we don't need integrational test for example with database gel because in that case we just test framework we could mock all all things which come from database I think that's probably a good idea I mock out the database for just about everything I will test the individual queries just to make sure that I've formatted the queries correctly but there's there's not much need to test a database if that database is something you've purchased so I'll just set that aside and mock it out now your answer good okay I think we're done the time is zero on my timer thank you all for your attention [Applause] [Music]

Channel: IT konekt

Views: 134,058

Rating: 4.8338027 out of 5

Keywords:

Id: 2dKZ-dWaCiU

Length: 71min 58sec (4318 seconds)

Published: Mon Jun 10 2019

Reddit Comments

[u/peterbourgon]

Dude is like the archetype of the expert beginner. He learned his one thing (TDD) in one context (dynamically types languages) and just stopped learning after that. Now TDD is his hammer and the entire world is nails. That and defending shitty dudes on Twitter.

[u/sheepdog69]

What a creepy way to begin that talk!

[u/SEgopher]

What notable software projects has Bob Martin actually worked on? I can't seem to find any concrete examples. I'm unsure why we'd listen to someone who doesn't seem to have a clear record of success in software besides selling books and giving talks.

I've seen a few engineers/teams try TDD as an experiment and it significantly slowed down their development cycles. I've also seen papers that suggest that testing has diminishing returns and that good code reviews were just as effective. In any case, teams should iterate to find the sweet spot that works for their type of software. Sometimes correctness is less important than iterating quickly and getting to market. Sometimes correctness is mission critical.

[u/Sauteed]

it'd be nice to see the screen too

[u/Safe-Ruin]

This is an interesting talk but why is he called Uncle Bob ?

[u/[deleted]]

Thank you. I am saving this for later.

Kinda still confused on how to implement this clean architecture concept to Golang.

I have try to write the implementation but still unsure wether it's the right way or not.

[u/[deleted]]

when i started writing unit tests in go i had to double check if the unit tests actually run using the verbose option 😆

[u/JitWeasel]

Ugh and he goes into presenters... The bane of testing. He does realize those are far more abused than the MVC model, right? There's literally zero pattern or rules there.

Precisely where many APIs break too. Not because tests aren't passing or because there's fatal errors or syntax issues, but because the behavior is wrong.