Architecture: The Stuff That's Hard to Change - Dylan Beattie

good morning how are you all doing hands up anyone who's got a party hangover that's great we should celebrate software and coding and development and all being together like this and I'm gonna talk to you now for an hour or so about architecture the stuff that's hard to change this is me Dylan bTW I'm CTO at skills matter in London I'm a Microsoft MVP with the visual studio and developer tools program I run the dotnet user group in London I created a programming language called Rockstar so that everyone can be Rockstar developers and I have some Rockstar swag to give away which I don't want to take home so come and grab some at the end and about five or six years ago the company I was working for hired some consultants to restructure things and one of them had a long series of interviews with me and said oh we think you'd be better suited as the system's architect and so I get this email dear Dylan this is to confirm your change of job role the systems architect you are now responsible for architecture architecting doing architect stuff systems architecture and architecting systems congratulations and so I go back to my desk and I sit down and I go hey I'm an architect now doesn't feel any different visual studio still doesn't load very fast you know and that started me on this this journey of you know it's like if I'm gonna do this job I want to do this job well so I want to understand what is architecture actually all about what does it look like when you do it well what's the role of an architect on a good team and I sort of started off by digging a little bit into the words we use cuz we have all these words we use in software that we kind of steal from elsewhere like you know we talked about pythons and mice and people are like yeah I - I have a Python in my house I feed it my sis like no no Python mice are different and we have buses and we have pipelines and we talked about launching and crashing and you know these are all English words which means something else and we've taken them and gone no no no no no we're gonna use them to mean a completely different thing and this you know language can be a really powerful way of understanding the approach people have to problems there is some controversy in the world so so put your hand up if you're a developer you call salford developer keep your hand up if you call yourself a hacker coder architect front-end back-end engineer the word engineer is this controversial term we have we talk about software engineering now there are some countries in the world where you are not allowed to do that in Canada you are not allowed to be an engineer unless you have an engineering license from the Canadian government in New Zealand you can be an engineer if you have a degree in software engineering but not if you have a degree in software development or computer science you have to have engineer in your degree and I live in the UK and a couple weeks ago the cable TV and my house went out and they sent an engineer to fix it as I I do not think that this person has an engineering degree you know but we have these ideas about architecture and engineering and you know my very first job years ago I worked for Arab who are a big construction company and they have architects and they have engineers and I sort of started digging a little bit into where these roles come from and the distinction that kind of makes sense to me let's think about construction for a minute we have these two things here the little barbecue on the left that's a hack like that probably the person who designed that built it by themselves or they got a couple of friends and helped with them the thing on the right is the Brooklyn Bridge the guy who designed the Brooklyn Bridge did not build it by himself now and this I think is an interesting distinction you know architecture maybe is the point where you're involved in designing software that you're not actually gonna write like you're coming up with the designs and the interfaces and the structures but you're not gonna be the person actually writing the code and in building architecture we have these amazing examples that we celebrate and people travel all over the world to go and looks at look at these kinds of things and in software architecture we kind of were a little bit behind the curve on that what was the last time you showed someone a code base and when look at this it's absolutely beautiful now come and look at it from this side here take a picture you know maybe architecture and software is not quite the same as what the rest of the world thinks of architecture is and it's only software and buildings that have architects they're on aeroplanes Architects there aren't racing-car architects or plumbing architects you know now let's dig a little bit deeper because the first person to use the term software engineering was this lady anybody know who this is you should she's awesome Margaret Hamilton she was the lead engineer on the guidance systems for Apollo 11 and she was the first person to talk about software engineering and what she called taking a systems view of how software works with all of the other components in a system which in this case was the system that took Neil Armstrong and Buzz Aldrin and landed them on the moon she was the first person to talk about software as an engineering discipline it's not something you do after the site by yourself it's something that has to fit in with all of the other processes and machinery and systems that are happening to make this this project work the first person to use the term architecture about computer systems was Fred Brooks he wrote the mythical man-month with from sure some of you have read very very influential engineer and scientist and way back in the 1960s 1962 he was working on a system for IBM called stretch and in the prelude to that he has this quote computer architecture like other architecture is the art of determining the needs of the user of a structure and then designing to meet those needs as effectively as possible within economic and technological constraints first time the word architecture was applied to computer systems now back in the 60s and 70s and 80s computer architecture almost always meant hardware because hardware was the stuff which was expensive to change if you decided to change the way a piece of code worked you just wrote some new code and you shipped it and you were fine but there are fundamental decisions one of the decisions that Fred Brooks made on the IBM stretch project was to use eight bits in a byte instead of six and basically you can trace almost every computer since then the 8-bit byte convention that all of us use you can trace back to this decision now you imagine changing that decision halfway through building a computer system it's like alright that's it throw away the processors throw away the motherboards throw away the busses throw away the factories that we've built to create the motherboards and processors we got to start the whole thing all over again expensive to change software was always cheap as a percentage of the whole cost of building a system two things happened around the 1990s which started changing the way we think about this the first of them was business software went mainstream companies who had you know never ever had computers before suddenly they had computers and they were getting people to build software for them in visual basic and you know little companies florists and little hotels and you know carwash places were hiring somebody to write them a little piece of software and they set it up and run it and it would be great and they could print your invoices and stuff and then a couple of weeks later they'd be like oh it doesn't do this thing we need we need to change it how much and suddenly the cost of making changes to a software system became a significant proportion of the investment for a lot of these businesses and so we started thinking what is it gonna cost to change software we've already created the other thing that happened was the world wide web up until this point pretty much all software ran on one computer and if it had any output at all it was a screen or a printer we didn't really do email we certainly didn't do apps and api's and uploading and stuff the world wide web came along and suddenly instead of building software that would run on one beige box in an office we're building stuff that runs on thousands of machines all over the world clients and servers and gateways and all these components and there are companies now for whom the cost of changing software is almost their entire operating budget you know they have a couple of janitors and some HR people but almost the entire payroll is just people who change code all day every day and for those companies if those changes are cheap they're profitable if those changes are expensive they're in trouble and so we kind of hone in on this idea of architecture as being the stuff that's hard to change because everything else is just coding right the first kind of formal treatment of software architecture was this book Mary Sean David garland it's published in 1995 and one of the things I love about this is they they called it an attempt to formalize a substantial folklore of system design they're all these people all over the world having good ideas about how to build bigger systems faster systems modular computer systems but no one had ever kind of brought all is thinking together and gone let's try and get some some words on this so we can have conversations about how to do all this stuff better I want to dig into Fred Brooks definition a little bit because I like this and I want to break this down into three things so architecture determining the needs of a user designing to meet those needs within economic and technological constraints now one of the big revelations for me in you know my my journey to becoming an architect was realizing who the users were because we build computer systems right and we think that the user is the person on the other end who's dragging and dropping and filling in forms and all this kind of stuff when you're an architect who are your users they are the developers on the team your job is to serve the needs of the people who are gonna be building and shipping the systems that you're working on and it's not just the people who are on your team today it is the people who you are going to be hiring next year or the people who are gonna be working on your open-source projects in three or four years time and so your role is to think about how do I serve people we haven't met yet building software we haven't written yet to sell things we haven't thought of yet to customers that we haven't got yet and make it more happy easy right we're probably going to need a plan now plans have had a bit of a rough time when it comes to software engineering because of this wonderful thing called the agile manifesto which everybody jumped on and read and when this is brilliant now not a lot of people know this but renege they got that the French philosopher was actually an agile consultant on on the side when he wasn't doing philosophy and he's working with a client one day and he says to them you know the agile manifesto says that we should respond to change instead of following a plan and the pliant says oh that's brilliant we haven't got a plan and they cut your salary blah if you don't have a plan how can you choose not to follow it it's not a valid choice there is a difference between saying this is our plan but circumstances have changed the plan is no longer accurate and just going we haven't planned anything let's just make it up as we go along and see what happens what the authors of the agile manifesto had observed time and time again his team's deliberately doing the wrong thing because that's what the plan said and that was what they were trying to address when they wrote this we are uncovering better ways of developing software through this work we've come to value responding to change over following a plan while there is value in the items on the right we value the items on the left more you need a plan but the plan is not guaranteed to be accurate the plan is a living document you as an architect need to have some idea what it is you're going to be trying to do and how those things fit together there is another detail in the agile manifesto or one of the the principles behind it that I just wanted to bring up we follow these principles the best architectures requirements and designs emerge from self-organizing teams now the point that they're trying to make here if you have a team of people who are working well together and are coming up with their own way of doing things step back and let it happen do not hire people and go there you go go over there and self organize and then you come back in a month and you're like where's the project and they're like we self organized into the bar across the street what do you want us to do now you know um be prepared for this to happen because high functioning teams will click and they will start this positive feedback loop but if your team come to you and say we could use some help you know we're not sure how to approach this we don't know how to collaborate on this you need a better answer as an architect than just yet go and self organize it's fine you need to give them the support to allow these things to emerge let's talk about constraints here are the rules no new hires no JavaScript no open source no microsoft no cookies this I believe is the engineering principles of the Oracle Corporation we're not gonna work here this would suck but everything you do there are going to be constraints there are gonna be things you just cannot do you know we cannot yet beam solid objects around the world if someone comes to you and goes can we build a teleport thing for shrimp you'll be like no that's not a thing sometimes they gotta come to you and go can we do high-definition like video calls in real time without using wires and for a long time you'll say no no no no and then one day you'll be like actually maybe yes for a long time everybody believed that a touchscreen phone was going to be a disaster because all the touchscreens anyone had ever seen were those plastic things in the airport that don't work and they click in the wrong place and there was a team of people working at Apple who were working on all these different things different kinds of glass different kinds of touchscreen displays multi-touch and they were the people who everyone else was going a touchscreen phone is impossible the constraints are too big and they weren't actually no now it's possible now we can do it and they launched the iPhone and within a year everyone was going yeah of course everyone wants a touchscreen phone you know we'll do one will do on Google did one Android did one all of these different things converged sometimes as an architect your job is to keep an eye on everything that is happening so you can come to the table and go the thing we thought was impossible maybe now is the time one of the things about this job which is difficult is communicating those constraints to the developers all your developers they go to NDC and they go back to work and go new shiny JavaScript frameworks gonna solve all our problems and you're the one who has to say sorry we have to support Internet Explorer that is one of the constraints of what we're doing because this is a business we have a big client they're still using IE 9 or god help you IE 8 or something and you're the one who has to communicate those constraints and explain to people yeah I know you want to do it this way but that doesn't actually align with the constraints of the system that we're designing here so there we go determine user needs meet those needs within economic and technological constraints now actual doing software architecture is really easy it's got to do three things you got to make a decision communicate it and then reinforce it thank you very much bye-bye now we'll do some more so making decisions how do you choose someone comes to you as the architect and says we want to build a system that does this we want to D cup alike a login system so he can build different micro services that use common authentication how can we do that and your job is to come up with a way of doing it before you can do that you need to understand five things first of all what have you got now a lot of time in organizations and software teams there will be two versions of the truth there will be what people think they have because they paid someone a lot of money for a high performance transaction processing system and there will be what they actually have which is a hundred thousand lines of Ruby that doesn't work properly you know and you need to understand these disparities now the best way I found of doing this is to look at the borders the boundaries between the different systems you want to understand the traffic that is flowing between the components of these systems don't worry too much about the code don't worry about the spec and you know when people tell you what something does take it with a pinch of salt it's useful but it might not be a hundred percent accurate here is the simplest possible systems diagram I can think of now as an architect you don't care whether that website is PHP or dotnet or Perl or Ruby or whatever and the internet there's someone else's problem but this thing in the middle HTTP how many questions can you ask about that how many transactions does it cope with is there HTTP on it when does the certificate expire does it cope with yet put post what if I put a proxy in there how much of this stuff can we cash are we using etags on the traffic that flows across it just by putting a network trace on that HTTP and watching it for half an hour will tell you more about how this website actually works than any number of hours spent reading the code and going oh this module looks interesting oh it's never been run we don't know why it's here you know traffic is real traffic is is the lifeblood of living systems you don't care what anything was designed to do your gene Kranz in Apollo 13 you care what it can do you need to understand the actual capabilities of the systems not what was in the spec and sometimes this will be disappointing like our high performance authentication system can only cope with six users a minute sometimes it'll be pleasantly surprising you'll dig into AWS and be like there's a whole Redis cluster in here that I didn't know we had this is brilliant we can start using this to speed up all kinds of things we can put varnish in front of our web server and boom instant acceleration and we can start rewriting stuff you know there will be pleasant surprises as well as disappointments it's well worth taking the time to do this of course some of you are going well we're working on a greenfield project here we don't have anything there is no such thing as a greenfield project because there are always gonna be a couple of little systems scattered around the landscape maybe there's some sales data in an Excel spreadsheet that you need to work with and some of this you'll be able to work with and some of it you won't for complicated reasons there will be the wreckage of the last attempt to build the Greenfield system because somebody has always tried this before and there'll be some code kicking around there will be politics there will be the person who got fired for choosing to use Microsoft sequel server and so now no one will touch Microsoft sequel server because the last person who did it got fired and you know like now ok I'm really glad we know that because now we know how to approach those kinds of conversations there will be the rabbit hole the thing that looks really really easy and you say to your team you just just integrate Redis with nginx and they come back three months later and they go boss we really need some this is not going how it said it would in the documentation all of these things are out there and only once you understand them you know where they are can you really make informed decisions about how to approach what you're doing you know what you've got what do you need you got a most business stakeholders they say yeah the website needs to be fast website needs to be secure you say brilliant thank you for the briefing you go to your team hey boss wants it to be fast the team go brilliant well build our own HTTP accelerator I saw a conference talk about it and you go back two weeks later and the boss says is the project finished yeah and you go no no the team is working on an HTTP accelerator it's really cool they're using bitwise pattern matching on the incoming network stream to get a top ten of the the fastest web servers ever and the boscoe's not that fast how fast turn these into business conversations because as the architect you're gonna be the conduit between what does it mean in tech and what does it mean in reality how fast is a fast website 500 millisecond response time from 90th percentile some stakeholders will be that sounds brilliant some will be what's the percentile you set him down you got okay what this means is if ten people are on our website one of them is gonna have to wait a little bit and I'll say how long and you say I don't know one second five seconds and now that yeah five seconds is too long no no we don't want that so you can turn this into a conversation and they come back and go make it faster and you say well that's gonna cost and they say how else can we make it faster and you say we could take off the JavaScript that draws the banner adverts and they go we can't do that and you're like well you want it fast or cheap you kind of both you know turn these into business conversations they say it needs to be secure how secure you know go beyond just it needs to be secure what is the threat model that you're worried about here there's a great talk I saw James mikans give here a couple years ago where he was talking about basically three different threat models which are Russian botnets you're angry ex-partner and Mossad and Russian botnets are going to knock on the door for three seconds and if they don't get in they're going to move on to the next account so you know they download these huge tables of passwords from the Pirate Bay and that kind of stuff that's one threat profile the angry ex-partner this is a difficult one because they probably have access to the cellphone that belongs to the account so two-factor authentication might not work so think of mechanisms for working around that Mossad are already in your system just stop worrying about it and enjoy your life and don't piss them off what can you build if you decide right we need to do this this is what we're shipping how dependably and reliably can you hit those targets at your sprints or your iterations or your stand-ups or whatever cycle you're using to do this work and this this one boils down to what I call four p's people patterns packages process who are the people on your team and who are the people you can hire which communities is your project interesting to other people there you can reach out to and get them involved you know how easily what kind of skills have you got your angular people f-sharp people react people back in front and understand all of these and align them with the kind of work that you're doing make technology choices based on who you've got based what you can do who you can hire patents are the kind of conceptual building blocks of modern software and some patent fit very well with some people using some languages and some don't I've been doing a lot of work in Ruby on Rails recently and rails is so tightly coupled to the MVC pattern that we had a 10-minute conversation the other day about something that they called a service object and I was like what is the service even sure how about that talking about classes just you know classes behavior and data and a thing and rails is so coupled to it's a model of view or a controller that something that isn't one of these things has a special name whereas in dotnet we start with everything's a class and then you're like oh yeah and if your class does this then it's probably the controller pattern understand the relationship between the patterns and the technology choices you're making because that can make a huge difference especially if you find yourself fighting the own the language or the frameworks opinions about the patterns you should be working with packages all language is now survived in a sort of ecosystem of components and libraries and modules and you need to encode a transparent PNG or do some crypto or stream some video for most languages you'll be able to find someone who has already solved this problem understand the breadth of the ecosystem that you're choosing to work with how easily can you get those packages are they actively maintained what happens if one of them goes away how easy is it to fork and do your own thing and process is just about reproducibility it's about being able to sit down on a Monday morning and say we're gonna ship this by Friday and on Friday most of the time you've done it it's about how your people collaborate together it's about how well you understand the the pitfalls and the capabilities of the systems that you're working with what can you buy there is a rule Dillons law of software never use PowerShell when you could use MasterCard your team should be building the things that you can sell the things or you know the things you can give away the things that create the value for your organization you know don't create your own cloud system Amazon have done that unless you are trying to compete with Amazon in the cloud infrastructure space that is a waste of your time they solve that problem you solve your problem look at everything you're doing now there is a point here there's a really interesting conversation about using Google Analytics because one of the systems that I'm working on has its own tracking and analytics engine and the reason for that is that we are using those analytics as part of the product and someone said why don't you just use Google Analytics and I said because we don't want to rely on Google for something we are actually providing to our customers this is what we do we're delivering it we own the value stream for that for seeing how many people clicked on an email yeah we have Google Analytics for that that's fine there's this sort of you know yin and yang there's this balance here what can you lose raise your hand if there is a system in your company that sends email keep your hand raised if there are two systems that send email three four five I once did an audit we wanted to rebrand the outgoing email to fit the new corporate color scheme I found fifteen different places in code where a developer had written some code to create an email message template to populate the HTML and send it and the first person who did this was definitely me because nothing existed and I sat down went this the second one was a developer who didn't know that I'd already done the first one the third one was somebody who went we'll two people have done this already so clearly that's how stuff works around here the fourth one was an outsource and you know I had to go and go look making the emails purple is actually going to be really complicated we've turned the color of an email into a software architecture problem because we have this duplication so what we actually did was in the process of turning it purple we consolidated all of our email into an outgoing relay service that applied the branding at the Perimeter before it got sent look for that kind of duplication because there's gonna be opportunities to make things easier streamline them recover some technical debt just you know improve the cost of changing those things once you understand these five things you got to decide what to do now this is kind of the easy part because once you've been through all those motions you've had the conversations you've talked to the teams you understand the landscape you come to things like NDC and you kind of keep up with what's going on in the industry and the community a solution will emerge you'll have a couple of good ideas you'll bounce them off some people on your team you'll be like you know I think we should do it this way then comes the really fun part you have decided what the software is going to look like and in your head is this beautiful crystalline structure of data flowing through systems and interfaces you know data structures and Landers and server lists and cloud and you sit down with the team like let me show you how the software's gonna work and you draw them the diagram and they look at it and they go what is that is that Java I'll talk about the problem with diagrams now in construction engineering you hire someone to build you a building they will start by making some drawings and you'll look at the drawings and you'll be like alright I'm sure you know what you're doing and they'll give you some more drawings and they will produce drawings of end elevations side elevations plan views schematics but sooner or later they will come up with artist's impressions visualizations and you'll look at that and you go okay I can see where we're going with this and I'll come up with slightly higher fidelity versions and you'll think okay and then they will build the building and you will go it's exactly what I thought it was gonna be and you can look at it and you can look at the drawing look at the building and go yeah did it nailed it that's exactly what I expected now we grow up surrounded by drawings one of the earliest earliest things that we learn to do as children is draw to communicate by making marks on pieces of paper and your parents stick them on the fridge and hope that you get better at heart before it gets embarrassing you know and we get good at reading we have picture books as kids with with pictures and we match the pictures to the words so we are visual creatures who grew up surrounded by diagrams and drawings and photographs and illustrations that we kind of intuitively know how to read and get meaning from unfortunately in software this is not necessarily true here is a software architecture diagram who wants to go and build this for me now this is this is a formal notation this is a diagram drawn using a notation you can learn in school I'd studied this at university this is the yordan de março notation for software engineering who can tell me what the things on this diagram are anybody nobody no it's not the slashes are the wrong way around the thing on the right well for starters let's add a key for you know people who might have forgotten a little bit of your than DeMarco since they were in school so there we go there's the key to the Orden demarco notation so the thing that the top-right is not a regular expression it's a database or a file system we don't know which but it's a database or a file system and that the oval things there are functions so now you can look at this and you can be like okay so we've got some you know these two things that have got data in them and then they're going through these two ovals and then there's this bottom right thing what is that okay that's input/output and for some reason it's called mandrill and Hann I think I might know what mandrel is because I remember reading about it in the MailChimp documentation so maybe it's that and this thing with the backslash backslash that maybe that's a Windows file share it's got a dollar on the end you know could be let's start decorating this with some more detail so first of all internal code names let's just put quotes around them to say don't bother googling this you ain't gonna find anything Excelsior is a sequel server full of customer details why is it called excelsior cuz the person who built it got it out of Excel I don't care naming is hard mercutio what is it it's a dotnet service that populates email templates so Norman is a dotnet app that sends emails why Norman Mailer okay thanks Steve really appreciate that this thing over here JK 6 GB 87 that is the Dell asset tag of the server where the marketing department keep all their email templates all these things are real by the way these are actual bits of software that I've worked on and mandrill is the smtp relay service for MailChimp which is an email platform so now we're kind of getting like we're still not sure but you could sit down and go okay I think I understand the kind of people we're gonna need to build this system we're still restricted by the fact we're using the or DeMarko notation because we got it out of a book and that means the only things we can conceptualize our databases functions data flows in for outputs scrapped it let's come up with our own notation nobody can read it anyway we're gonna have to include a key boom now we're getting somewhere we've got some clipart and we've got some Microsoft Office templates and we got some stuff we found on Google we've got this this red line here with the boxes on it which I crypt out of the rabbit and queue documentation so now we can be like okay this is cool we got excelsior that what is the green line here that's a do dotnet okay this makes sense we got Mercutio what's the little gear wheel okay those are Windows services that blue thing there that's an SMB file share okay and we're starting to think okay I understand now like you can look at this diagram see what kind of things are gonna have to do to deliver it let's go even further let's annotate the traffic boundaries a deonette ok windows authentication now we know how to plug these things into each other this Windows file share we're using for text and HTML this rabbitmq okay that's cloud AMQP brilliant we know how to work with that SMTP port 993 TLS enabled that's why I can't send any email I'm using the wrong port and I don't have the certificate now we have enough information here to actually have a meaningful conversation how are we going to build this how long is it gonna take how many instances do we need to provision to run it the one on the left looks like the ones in the textbooks it is neat and it is elegant and it looks like the work of a professional and the people reading it go I'm not gonna ask about that because I don't want to look stupid the one on the right is starting to look like a bit of a mess but it's a mess which has useful information in it and you can actually use to have a conversation one of the big challenges that I find in all kinds of software development is the examples in the books and the tutorials and the conferences are always tiny no one ever shows you how messy the stuff gets when you do it for real this is one part of a systems architecture diagram that I built as part of a outsource and collaboration project about five years ago and this you know is literally broken down there's a key on here like I said this is all the different kinds of traffic there's even a color code in there for some hideous VB cursor based library we knew that existed the network traffic made no sense but we knew it had Visual Basic on one end and sequel server on the other we could never make any any sense of it so we capture on there look this thing's here it's important but abandoned hope you're never gonna find out how this thing works and just stick you know arbitrary notations like there's a comment on here database Inc every five minutes updates user names and passwords you can do it on demand by pressing the big red button put that kind of it because this is something people are like alright it's already messy what if we scribble on it let's start making notes and annotating and moving things around and stuff and this is what diagrams look like when you actually try and convey reality instead of make things that look good like they did in the engineering textbooks final note about diagrams here are two software architecture diagrams one of these is the hexagonal architecture which one I'll give you a clue it's not the one with the hexagons on it a lot of the time there will be you know words like like hexagonal architecture and tear ports and adapters layered interfaces that kind of if you've heard the expression hexagonal architecture and you look at these diagrams you will logically assume that the one with the hexagons is hexagonal architecture now we use the name hexagonal architecture because the first version of the paper about ports and adapters used hexagons and the diagrams in the name kind of stuck it's got nothing to do with hexagons there aren't necessarily six of anything in a hexagonal architecture it's just a name we got the thing on the left with the data ports and the adapters in the fast doors that's hexagonal thing on the right is a honeycomb with clipart pasted into it it's nothing to do with software I made it up never be afraid to ask never be afraid to say hi I'm sorry I'm still not clear can you just run through what that thing on the diagram is one more time what is this line here do what are these things the yellow thing is that a sequel server database you know by having these kinds of conversations one it makes it okay for the other people in the room to be like yeah I don't understand either can we do that again - it gives the architect the context of like okay next I might present something these are the things I want to cover in advance so that we can cut straight to what are we going to do and how we're going to build it it's all about reinforcing decisions so you've come up with your design you understand the constraints you know what it is that you're trying to deliver now you want to help the team build the right thing I did initially call this enforcing decisions but I kind of didn't like the kind of you know the architect as police kind of deal you want to support your team in the work that they're doing and this boils down at two different kinds of things what we call validation and verification validation are we building the right thing this is hard because what you can do is come back in ten years and see whether this company's still in business and the software worked there isn't really a shortcut for this but verification are we building the thing right is the thing we're creating does it match and reflect the ideas and designs and structures that we've talked about now with construction architecture you can take your diagram and you can take your building and you can look at them side-by-side and you can go yeah that looks right to me and there are other dimensions to it engineering stress diagrams and load-bearing things and stuff but this technique we developed as kids of looking at a drawing and telling whether it looks like real life or not that holds us in good stead so let's do the same thing with some software here's our diagram right teams built the code it's done it's shipped we want to have a look at it and see if it looks like the diagram how do we look at the software I know Microsoft Azure has a dashboard let's look at that it was very pretty but it doesn't really look like the diagram maybe teamcity can show us what's going on they're still pretty but it doesn't really help us and these red bits might be bad I know we'll look at the code now I'm just not seeing that the relationship here how can you tell whether the software that has been built matches the design the architecture the concepts and the structures and things now when I first started at doing this I made a classic mistake which is I did I'll do all the code reviews so all the teams can build what they want and then as the architect I'll review the code I'll read it I'll make sure that it's right this really really sucked in it was horrible and I do not recommend you ever try it one you'll burn out you'll have days when you're still at work at 7 o'clock at 8 o'clock at night and you're like are those 15 pull requests still to review but if I don't review these nobody can do any work tomorrow and so you just kind of get sloppy and you're just like yeah that kind of looks right I don't know and the other thing is that restricts the ability of your teams to do stuff that you don't understand and you get this I reviewed your see SHOP code and I still don't know what it does and your lead engineer says dude my team's using Scala the other problem with reviewing the code is have a look at this code here now we've had a meeting about the architecture and the design and we've said we're gonna use a repository pattern with an identity map and everyone has gone on repository identity map we understand mr. architect thank you very much and then they give you this code what can you tell from this code you can tell that the developers knew that the word repository and the identity map were important and they should use these names the names don't tell you what the code does they just tell you what somebody called it reading code and trying to distill what it does on a good team who used name inconsistently and who kind of understand the relationship between patterns and classes and naming and stuff it can work but it could also be a bit of a minefield it's like wandering through a city at night and going who luxury hotel that looks pretty good and then you wake up in the morning and you're like okay maybe following the labels was not the best way of understanding what we were doing here so what can we do to kind of gently guide our teams down the correct path create a pit of success that will help them come up with the right kind of implementation one of the most powerful things you can do is to exploit something called Conway's law teams that build systems the system they build will end up matching the communication and organization structures of the teams who built it so if you have software which is split into components you want tight-knit teams building a component and another tight-knit team building another component and another team building another component and align them with the technology choices bunch of front end people ok you to the front end this sounds obvious but there's places that get this wrong front end team you're over there angular react JavaScript whatever you want to do that's your bit you've over here you're doing F sharp and you know calculations and back-end and price quoting all this kind of stuff you can even exploit geography to make this happen you know say all right well we've got an angular team over here in London we've got a F sharp team over in Belarus where we've outsourced a lot of the heavy lifting and the processing and then our Android team is going to be based down in Porto what you get here is the boundaries between these things will naturally emerge because it's really really cheaper to collaborate and it's really really expend to change something collaborating with a team in another time zone so you get the boundaries between the teams right and this is where you as an architect can get involved with this you have a front-end team the front-end team a building code that plugs into an API server sit with them at the beginning and say right what we gotta do here it's gonna fake up some API responses so you can start work and then the other side you get other the back-end dev team and you say right I'm gonna help you build the specifications the test suite for your API and then you can go away and build the client that satisfies those requirements that's how you communicate the designs you create the data and you create the tests you work with them on this and then you let them go and build the actual code and then you can plug these two pieces into each other and as an architect that's where you look at what's going on in the system and look at the traffic are you seeing the data structures that you expected to see is the latency and the throughput and the volume of traffic the kind of things you were expecting to see one of the questions that comes up often is should architects still code yes absolutely yes because if you stop coding your hair goes pointy and they take away all your def leppard t-shirts but don't work on production systems because what will happen is you'll have a good idea about how to improve the architecture and sooner or later you will give in to the temptation and you'll be like I don't want to update the diagrams and have a meeting I'll just I'll just do it and then the team will go what the hell have you done and you're like it's better now and they're like okay and the first time they'll be like and the second time they'll be like alright fine you do it and you'll become the bottleneck again and suddenly you'll be you know the architecture is falling by the wayside the team will be like well you keep changing things so this is your problem now we're not working with it anymore but you have to keep a handle you have to keep coding otherwise it's really difficult one to just you know keep your tools sharp keep up to date with what's going on in software - it's a great way of establishing credibility and good working relationships best way I've found to do this is the architect builds the monitoring systems you pick people on the teams you like right I want to take your API what it's doing and I want to show that traffic on a big screen on the wall of the office with you know red and green lights and all this kind of stuff let's pair together on building that thing so you as a coder get enough of an ins right into how that system works but you're not responsible for anything that's actually going into production in a customer-facing system you're building the stuff that runs on the wall and one it gives you a sense of ownership you know you're like hey this is cool I got some life code in there too you get a chance to collaborate with people kind of learn from them share some advice and wisdom most architects were pretty good developers before they started specializing in architecture there's value in that stuff but just don't let yourself become the bottleneck the other thing that you can do a lot of the time particularly when you you make a decision say to go and use er off-the-shelf service or host something somewhere or integrate with a payment provider as a business decision it makes a lot of sense because these things work but for developers on the team often that experience sucks because they have these these nasty api's with signed xml crypto signatures and weird sand boxes and stuff and the developers don't see the business value I had a real case a couple years ago where we ported a whole bunch of stuff across from in-house systems to Microsoft Dynamics and Dynamics as a developer it has some interesting capabilities but there was a lot of things about it which are quite difficult to work with and you know the team was getting a bit downtrodden I wonder I said then one day do you remember last year every Monday morning somebody would spend two hours pulling mailing lists out of the database so that the marketing team could send the weekly newsletter and they all went yeah and I went have you noticed we don't do that anymore and they all went oh yeah because you lose sight of this stuff when the business problem gets solved the developers very rarely see the solution your job as an architect is to go look no remember that nasty thing we used to do we don't do that anymore this is why you know communicate the value that is being created through these decisions pair with the other developers on your team build dashboards build monitoring systems look at the network traffic don't make yourself a bottleneck all of these kinds of things are the collaboration patterns that you can use to build high functioning teams to allow these behaviors to emerge to respond to change over following a plan you have the plan but you're not married to it if the plan is wrong you make a better plan and all of this stuff there is this classic xkcd cartoon how to write good code you know and you can okay well we got to do things right or fast if you do it fast does it work yet no no no no almost but it's a massive clutches and spaghetti code throw it out and start over otherwise you do it right code it well are you done no and the requirements have changed do you know how you get good code good code happens when good developers look at code and go I can make this better and they do and almost everything that we talked about at events like this about how to build better systems the frameworks unit testing integration testing architecture what we are trying to do is empower that developer to feel confident that they can try out their idea for improving it architecture is the stuff that is expensive to change you get it right the things will be cheaper to change those little ideas little refinements someone can open a piece of code and go I think I can refactor this to make this work better make this fit better buy good architecture empowers people to do this whole teams to do this over years and years and years like I said it's hard to validate you come back five years later and you see whether it's working or not if it isn't gently steering around a little bit but like everything else it's about giving good developers the ability to look at a piece of code make it better ship it and know that they haven't broken anything by doing that thank you [Applause]

Channel: NDC Conferences

Views: 46,200

Keywords: Dylan Beattie, Tools, Design, DevOps, People, Architecture, REST, API, Software Development, NDC, Oslo, 2019

Id: 3LtQWxhqjqI

Length: 45min 27sec (2727 seconds)

Published: Mon Jul 01 2019