Software architecture as code by Simon Brown

hello good morning demux welcome to the last day Friday so this is me I'm Simon and this talk is going to explore the intersection between software architecture and code and really there are two things that inspired me to put this talk together first of all lots of people have architecture diagrams but they never quite match the code people generally think this is the case as well yeah I see a few people nodding good and how do we create those architecture diagrams in the first place well I can teach people how to draw architecture diagrams on paper and whiteboards but then the obvious follow-up question is this what tool do you use and please don't recommend Visio again so that's really the inspiration behind this talk so I live in Jersey and Channel Islands which is a tiny island off the coast of France even on this gigantic monstrous cinema screen can you see it can I see it no it's it's like three pixels or something now if this was a big iPad pro type thing with the huge screen we could pinch to zoom in gets a bit closer still quite small and we could pinch to zoom in once more and there you go this jetty now why do I live there basically it's a nice place I lived there as a kid moved to London worked in London for about 12 years mostly building Java systems for banks and consulting companies and I moved back to Jersey in 2008 and over the past few years I really been jetting around the world and helping teams understand what software architects is all about from the perspective of technical leadership agility and that kind of thing and really my whole picture is that I think architecture needs to be much more accessible to us as developers sure there are lots and lots of books out there but when I was moving into my first architectural I didn't really understand how the stuff in those books related to what I should do day-to-day as an architect so how do I solve this problem I just write some books I had some different books so that the one on the left is software architecture for developers and on on the right you guys can go going down low for free and some of the stuff in the book on the right will tool back today in addition to doing all this kind of hand wave view stuff I'd still do write code you'll be pleased to know and this is important because I think architecture and code are just two sides of the same thing they shouldn't be separated and that's what I'm really interested in here it's the intersection between architecture and code and the thing I'm specifically interested in of course is how do we communicate architecture how do we visualize architecture how do we share information about how our systems work to other interested parties so let's set the scene imagine you inherit a Java code base let's say it's a couple of million lines of Java code and let's say that this is maybe fifty thousand a hundred thousand two hundred thousand Java classes and somebody asks you to draw a picture to describe that code base would your picture include fifty thousand boxes 100 thousand boxes you know one for every class no even on this big screen if we were to draw it all of the classes will look teeny-weeny and there'd be too much complexity so when we're drawing architecture diagrams we're using abstraction and abstraction is a way to allow us to reduce the cognitive load reduce the amount of detail that we're showing so rather than drawing 50,000 Java classes we comp these things together don't we into bigger things bigger chunks and we have some words to describe these things components or modules or services or subsystems or layers and so on and so forth and the whole purpose of abstractions is that these things allow us to reason about big complex software systems now there are lots of ways we can do this of course there are lots of ways we can think about and describe software systems and this quote here is from Philip cretons 4+1 model and he basically says to describe a software architecture we use a model composed of multiple views or perspectives so we're looking at out a system from different angles essentially and this picture is Philip couch four plus one model it's a logical view of how our system works so logical components for example there's a development view which is more about aligning that for you with the code there's a process for you that talks about concurrency things and there's a physical view that talks about deployment and all of these things brought together by some use cases in the middle this is one approach there are lots and lots of other view catalogs every architecture book will have their own way of doing this and that's cool but over the years what all of the names of these views have kind of lost their meaning so I've been in meetings with architects and they've argued that there's a difference between a conceptual view of a system and a logical view of the system and I'm like what as a native English speaker that doesn't make sense development view design view implementation view sometimes these are the same things sometimes they're different so all of this stuff is just complicated now the thing you will notice about all of these different ways to describe architecture is that you normally have a logical view of the system which is modules or components or services and then you'll have a separate development view which is more aligned with the code and these things are always separate if you go and look up all the different view catalogs there's only a logical view and a separate view that tells you something about the code how its laid out how its structured on disk for example and this makes my head hurt a lot whenever I've been asked to draw architecture diagrams in the past you have to get your head around all of these different viewpoints and views and perspectives in order to figure out what it is you want to draw now being that we're all geeks we could just cheat so how do we cheat well if somebody asks me to go draw a software architecture diagram why don't I just go consult Google or Stack Overflow but let's go Google first and if you've ever done this if you've ever typed in a query software architecture diagram and asked Google you basically get stuff like this it says page after page after page of you know pretty color block diagrams the type of things we see on powerpoints and in Visio and in Word and so on and so forth and they all end up looking eerily like this so if we take this one I've highlighted you know we always talk about layering in architecture being a good thing so we have a diagram showing layers but you'll notice some of the layers of different heights is that important is it relevant is it showing something about size and complexity or did somebody know or not know that in Visio you can multi-select all the layers and make them the same size right taking advantage of the tooling who knows sometimes stuff doesn't fit neatly into those horizontal layers we have the vertical layers it's not the diagram falling over we've got acronyms on here we've got no lines joining things these two have nice gradient shading looks good on a PowerPoint presentation and of course the same patterns repeated all over the place and these are exactly the type of diagrams that I see when I go visit organizations if we if we kind of step back a few steps what is it we are trying to do with these diagrams well if I'm drawing diagrams as part of an upfront design exercise then really it's this question I'm trying to answer so here's the set of diagrams do the diagrams reflect what I think I want to build and of course if we're retrospectively drawing diagrams from an existing code base then the question is is this what we actually have I think we struggle with this a lot I also don't think it helps that we don't have a consistent way to do this across the entire to the industry we lack a common vocabulary with which we can think about talk about draw and describe software architectures think about the word component how many times have you worked have you heard the word components used to mean different things for some people a component is an application in a bigger system and for other people our component is something that runs inside that application so we use the same word for differing levels of abstraction for example now we could say well Simon why don't we just use UML UML the five morally language sorts a lot of this stuff out right I'm going to try an experiment quick show of hands who here uses UML regularly and frequently it's not that many might be ten percent ish so what's everybody else doing I'm drawing boxes lines I've asked this question all around the world and yeah most people don't seem to be using UML anymore there are a couple of exceptions I was in Germany a couple of weeks ago they use UML I don't know why I still not discovered why uh-huh but but mostly you know UML sees be falling out of favor if you look at what you male gives you it's trying to give you a a standard set of things abstractions and a standard set of boxes and lines to represent those abstractions and it's always the notational side we struggle with one of the big objections are here about UML is it's too complex because you have to have all those discussions about open diamond versus shaded diamonds and then you have to go to Wikipedia lookup aggregation versus composition and you just want to write some code so again let's backtrack I think having a common set of abstractions a common set of things is actually way more useful than a common standard notation that's an odd thing to say but it kind of makes sense and it makes sense in a number of other different fields and let me give you a really really good example Maps let's go get two maps of Antwerp and lay them out on the front here the two maps of Antwerp will probably show you the same things won't they the various areas and districts and the sites and the points of interest and the bus routes and the tram lines and the railway stations but the two maps might use differing notations to describe the same things how do we decipher the maps well there's a key in the corner so the maps are a really great example of you know a common understand set of abstractions with a self-describing notation so maybe we should try the first maybe we should try to get the abstractions clear in our mind and I really do like maps and the reason I did my silly can you see Jersey notes - smalls oom Zoom Zoom it's because I like to think of software diagrams as being maps they're maps - navigator a large and complex software system if you open up your smartphone and go to Google Maps especially and do a search for Jersey it will zoom you straight in and it shows you all the detail in Jersey and where the places are and what the names aren't things like this if you've never heard of Jersey before it's completely useless so the first thing you do is you zoom out to get to the map of Europe to get some context the software diagrams the same sort of things sometimes as developers we want that zoomed in view of the world but actually depending on who we're talking to we might want to zoom out a few notches so so how do I do this well for me a software system is made up of a number of containers a container is simply something that executes code or hosts data in real terms it's something like a mobile app a web app a database a file system a batch process a standalone java process that sort of thing if we zoom into the containers for me they contain components so I'm going to use the word component to mean something that runs inside a process essentially now this is a massively overloaded term of course so for me a component is nothing more than a bunch of related stuff with a nice clean interface and some encapsulation around that and because I mostly build Java systems my components if you zoom into them that well they're just made up of classes and that's it it's a simple hierarchical model to describe the static structure of a software system this works for Java if you're using a functional language there maybe it's not components in classes maybe it's modules and functions if you're using a database technology it's not classes maybe it's functions and sub procedures so you might have to modify this that the the things in the structure to map onto your technology why am I doing this well what I'm trying to do is I'm trying to create a set of abstractions that I can use to model the static structure of a system and for me this is our a static model first approach and you might say well that's fine but I want to represent other things I want to understand how a user story is implemented or a use case is realized and that's fine if you want to draw things like sequence and collaboration diagrams to show these aspects well the box is on the sequence and collaboration diagrams needs to come from your static model so for example here's a feature here's a user story the sequence diagram is simply the collaboration between components in your static model if you want to show the deployment mapping well the deployment is very simply how you map your containers onto Hardware so in terms of visualizing stuff in terms of pictures well I call this my c4 model and basically I will start by drawing a system context diagram and this basically shows your system in the middle with stuff around it and I'll show you some examples in a second I'll then pinch to zoom in to get to my container diagram this shows my mobile app sending Jason to my web apps throwing stuff in the database and then might zoom into my web app and show my components and then if I really want some low-level detail I'll optionally draw some class diagrams I'm just gonna repeat that again optionally I don't normally do the class level stuff because we can get that easily from the code and of course I'm not saying you should only draw these types of diagrams because there are a number of other ways you might want to describe something about your software system wireframes UI mock-ups two main models deployment diagrams there are a whole bunch of other things you might want to use to describe your system I think we as software developers are actually the most important stakeholders for software architecture I don't think software architecture is one of these things are only ivory tower architects do I think it's for all of us and that's why I've focused this static model on essentially modeling the system from the system-level right down to the code with a couple of levels in between so we give some quick examples when I moved back to Jersey in 2008 I wanted to find other tech people and I wanted to keep a list of them particularly those people who were blogging or tweeting or speaking of conferences so I built this thing here tech tribes je and it's a really super simple super basic content aggregator for our local tech industry it basically lists local people businesses blog posts tweets events and so on so this is a context diagram of my tech tribe's system the Box in the middle with the monkey that's my system that's the thing I've built it has users at the top of this diagram so these are the various types of users roles in this case if you go onto the site right now you'll be an anonymous user and you can view all the content because my my content is aggregated into the site I can sign in with my twitter ID and manage my user profile and there's a super-secret admin user so that's my user community roles act as real people personas however you want to model people at the bottom are my key system dependencies so these are the things that tech tribes needs to be available for it to run properly we're pulling profile information and tweets from Twitter if something has registered a github ID we pull back the list of public repos and we're subscribing to people's blogs and that's it it's just a really simple high-level picture to show tech tribes and how it fits into the bigger context if we want more detail well now we do the pinch to zoom in movement and we get to level 2 and this is the container diagram for tech tribes it still has people and it still has my external systems and basically all I've done is I've exploded that monkey at the top there's an Apache Tomcat web application a java web application when you want to patch your Tomcat it's the single point of access for all of the users the bottom box is a content of data and it's the thing that reaches out to Twitter github blogs and pulls the data back and the data is basically installed somewhere in that middle trio of boxes so there's a my sequel database for storing most of the entities there's a MongoDB store on the other side for storing the tweets and the blog posts and there's a file system in the middle for the search indexes this diagram does not tell you anything about clustering availability sharding deployment that's what we keep for a separate deployment picture so again this is just the the high level static structure it's level 2 if we were interested in how that thing works the contact my data we could choose it pinch to zoom in and now we get to level 3 and this is a component diagram for the content of data the content of data talks to the outside world those the boxes on the bottom and the content of data stores data in those other containers at the top and again very simply in the middle there's a spring schedule task it gets kicked every 15 minutes it uses the various connectors at the bottom a diagram to talk to the outside world and there are some other components at the top for pushing stuff in and out of the data stores and that's it if we wanted to know more information about their tweets component here and the tweet component basically allows you to store tweets into a datastore and get them out again we picture to zoom in and we get to level 4 and this is a class diagram of the internals of that tweet component public interface a couple of package protected implementation classes and a public exception in this case so that's a really really easy set of diagrams it's a really easy way to describe a software system tools so in order to draw some diagrams to explain how this worked I've needed to use some tools and the tools I've use here are basically just OmniGraffle from the Mac it's essentially a Visio style application for the Mac it's a general-purpose diagramming tool I have two draw boxes make them the right size put some text in make the text bigger Center the text make sure all the details fit draw the arrows it's a very very manual process now sure we can build templates to help us do this so we can drag things off a pallet onto our diagram but come on it's 2015 why are we doing this not even the building industry does this anymore if you ask a building architect to draw your architectural diagrams they don't get Visio out they get a modeling tool out like AutoCAD so they are much more sophisticated than we are ironically the other thing of course here is imagine I'm doing an upfront design process well I'm going to draw some sketches on a whiteboard and then I'm going to write some code and what happens about 20 seconds after you start writing your first line of code are your diagrams get out of date so why don't we also generate the diagrams from the code has anybody tried this yeah I can see people snickering it's just a horrible horrible thing to do there are lots of ways you can do this of course you can use plugins for your IDs there are static analysis and dependency tools structure 101 Latics and depends sonar there are a bunch of them and there are some humor modeling tools which lay to reverse engineer and the problem with those tools is basically they see everything so you point these tools at your code base and they just visualize your code base and they basically give you a mess and sometimes your code is not even a mess but it's it's just because they're having too much detail and the problem with most of these tools is that they're showing code they're basically just reverse engineering code and presenting it in a diagrammatic form and the java terms these diagramming tools are basically seeing packages classes interfaces that's why these diagrams look so complex now of course you can teach some of these diagramming tools to abstract a bit but that's hard to do this is a really great example of what George Fairbanks in his book just enough software architecture cause the model code gap so when we draw an architecture diagram we're drawing modules and we're drawing components and we're drawing services and we're drawing layers and all these sorts of abstract concepts but in our code we don't have these concepts so Java does not have a component keyword does it you can't type public components X in Java and make it create a component for you there's no layer keyword in Java we normally create this concept of layers by dumping stuff in a package you know we dump all our repositories into one package all of our services into another controllers into another that's the problem here our code has one set of things we use and our after two diagrams has a completely different set and they're mapping between the tiers is sometimes impossible to determine George's book came out in a 2010 ish maybe a little bit full up this is not a new problem so far I've managed to traces back to the 90s this is a paper from 1995 I think it was and basically it talks about reverse engineering diagrams from code and even in the first couple of paragraphs it basically says as engineers we like drawing pictures on whiteboards with boxes and arrows and we can accurately very accurately reverse-engineer diagrams from code but the diagrams we reverse engineer never quite match the whiteboard pictures again because their concepts don't meet so I guess we have to step back a second say well we're drawing components on an architecture diagram what is a component from a code level perspective and there are lots ways we can do this but let me give you one simple example so I'm sure you're all aware of spring if you don't know spring spring is a framework in the java weather does everything if you've asked the right people and part of spring is a framework called spring MVC to build MVC style web apps and there's a sample application on github called the spring pet clinic and it's basically a showcase of how to create spring MVC style web apps this diagram down here at the bottom corner is I basically stole it from their slides and this is an architecture diagram for the spring pet clinic system so there's a bunch of stuff in the browser at the top there's a layer of controllers web app controllers there's a layer of services and there's a layer of repositories when you pull down the code and throw it into your ID of choice you get something like the picture on the right essentially there's a bunch of model entity domain classes they don't do much in terms of business logic there's a repository package in this case with a JDBC implementation there are two others but I've excluded them for the purpose of this discussion there's a services package and that corresponds to the service layer on the diagram there's a Utah package because well why not we all have utils admit it and there's a web package containing all the web app controllers so imagine this is your 1 million line codes Java system that you've inherited and somebody asks you to draw a diagram the easiest way to get a diagram out of this code base is just to generate one from your tooling if you use a UML class diagram approach you get that don't forget this is only a tiny code base yeah it's like 30 classes a couple interfaces whatever you could say Simon come on you could have spent more than 30 seconds organizing the diagram to make sure the lines in overlap I tried it's impossible it's impossible because basically we're showing everything we're showing all of the low-level code level constructs and all of the relationships and fences between them and this is a small code base don't forget imagine if this was a hundred thousand classes you've got absolutely zero chance of understanding the diagram so this is a naive approach and it's not going to work so again let's step back and let's take a more thoughtful approach so instead of showing everything why don't we just restrict what we show on the diagram why don't we exclude some stuff that's not relevant to our architecture discussion so let's ask the question of the code base well code base what are the architecture significant elements within you now I'm going to make some assumptions here I'm going to assume that my model class is my domain classes are not architectural e significant I don't really care about them when I have it when I'm having an architecture discussion I don't think they're structurally important I'm going to say my repository stuff is because that was the bottom layer on the architecture diagram the spring guys used the service well once you see the service you'll understand it's really structurally significant I really really hope myutils are not significant I'm going to exclude those and I'm going to say my web stuff is architecture e significant if we take this approach and regenerate our picture we get that that's a bit better we can now move the boxes around and you'll see I've tried to recreate the layers that we had before so we have a bunch of web controllers at the top we have the service in the middle and we have the repositories at the bottom this diagram is still showing code this diagram is still showing classes and interfaces and that's why we can now see the true picture of the dependencies between these things so if you look at the cloak service well the clinic service has no dependencies as an interface but the implementation class does you see there's some funny tangling in terms of the repositories at the bottom if this was a Google map this could be Street View and I don't want Street View no Street View is great if I want detail but if I want to figure out how to get from A to B it's a bit hard so I need to zoom out a bit if we zoomed out a tiny bit from this diagram what I really like to get something looks more like this so what I want to do is I want to bundle together the interface and the implementation class and I'm going to call that a component so I'm going to make the assumption here that all of my controllers are separate and therefore they're separate components I'm gonna say my clinic service thing is its own component and I'm going to treat all of the repository implementations as separate components again this is a sample app in the real world you might have a whole bunch of helper classes within your clinic service you might have a whole bunch of row mappers and things like that with in your repository implementations but again I want to bundle all of that stuff up in essence what I really want to do is draw that diagram so this is the same diagram with that level of detail just removed we have the controller's at the top they use the service the service uses the various repositories and I want to draw this automatically now the code is a single point of truth isn't it if you talk to many agile extreme people they'll say you don't need documentation it's all in the code that's not true but the architecture the code you know does embody the architecture the code is the final realization and implementation of the architecture of the architectural ideas but here's an interesting question can we get that back out again so can we extract information about the architecture from the codebase hmm maybe maybe not this is a paper from 1996 by Paul Clements and he talks about architecture description languages if you've not heard of the term architecture description language essentially it's a a textual way to describe an architecture typically the static structural qualities of an architecture so rather than drawing boxes and lines diagrams in Visio you use a language to describe the same stuff is anybody here using an ADL no so ADL's have been around for a number of years now the most popular one I'm going to put popular in air quotes the most popular one that I've come across is called Darwin I know of two people using it around the world and if you do search for Darwin architecture description language you basically end up on the University website and that sort of tells you only need to know that you can download the PDF of the language grammar it's horrible now why should I learn this really complex language syntax just to describe my software system and yeah I hardly see anybody using around anybody using these things around the world but I like the concept you know the concept of an ADL is great because then we have something that's textual we have to link to support texts we confer some control at different and so on but anyway back to this paper Paul Paul Clement says that in practice architecture is embodied and therefore recoverable from code this is in 1996 and he's making a claim that we can get architectural information out of a code base I don't think this is true anymore you know we've come a long way in the past 20 years look at the amount of frameworks that were using that that are kind of hiding knowledge look at the amount of convention over configuration I don't think we can extract as much architectural information from the code as we cut could perhaps in 96 so let's try this you know given your code bases you're working on today if I asks you to draw me a context type diagram to show how your system fits into the bigger wider world could you do it from just scraping your codebase for information so let's take this piece by piece can you get a list of people or users or roles from your codebase now maybe you know if you're building something with a user interface and there are security roles in there maybe you have a configuration file that Maps role type to URL mapping for example so we could scrape that information out and imply a list of users maybe have a nice enumeration somewhere in your code base that says here are my different types of users the roll name is for example maybe that's in a database table but I think this is pretty hard to get actually you certainly have to write a bunch of custom code what about the software systems that your system interacts with can you get that just by scraping a code base the answers may be you know maybe what we do is we we scan the classpath of your systems and we write a bunch of rules and the rules go something like this if we see Twitter for J on the classpath and make the assumption that this system is talking to Twitter and that's fine we lose all the intent you know why is it talking to Twitter what information we're getting out of the system we could scrape configuration files for known API endpoints and again make some assumptions but the intent tends to get lost what about those other situations where we're just dropping a document on the message bus and it's disappearing well that's definitely hard to kind of capture and what about those really nasty situations that we say we're never going to do where our system generates a file it drops it on a network share and another system picks it up and although this sounds horribly our cake there are still lots of places doing this so these top levels of information kind of hard to get if we pinch to zoom in containers could redraw a container diagram just by scraping the codebase this is slightly easier because we could take your Eclipse workspace or your IntelliJ project file or if there any dotnet developers here your visual studio solution file and we could pause the metadata and we can look for web applications we can look for spring boot apps we can look for drop wizard we can look for the text public static void main we can look forward database connection strings so again we could extract a list of containers from our code base but it's going to be a very very hard manual process pinch to zoom in can we get a list of components for a given application by scraping the code base and the answer is actually yes but there's a caveat and a caveat is do you accurately have a way of extracting and therefore identifying what those components are so what rules could you create on top of your code base to describe how you build components and this is where George Fairbanks talks about an architectural e evident coding style so this is one of the examples he uses to bridge that model code gap he says what we should do is we should adopt an architectural encoding style and basically this is simply dropping hints and metadata into your code base so that things in your code reflect things on your diagram we're trying to make a simpler mapping between the code and the stuff on the diagrams and there are lots and lots of concrete techniques we can use to adopt an architectural everything coding style and and here are some of them so naming conventions is a really really simple one so if we have a a logging component on our architecture diagram let's make sure that something somewhere in the code is called logging component again so we can match the two things up maybe it's about packaging or namespace and so maybe we bundle all of the stuff related to a component in its own package maybe we throw annotations on things to signify that they are somehow structurally significant at module act component at service for example if you're using maven modules or OSGi modules or hopefully in the future Java 9 6 or style modules that's a great example of an architect for everything coding style because you'll probably have a bunch of modules on your architecture diagram and you can find those modules in your codebase javascript is just a pain there are lots of module patterns in JavaScript required jason and whatever equi script 6 is also getting support for basic modularity so again hopefully that will help clean up javascript that and of course micro services Microsoft is a really really good example of an architect for levelling Cody saw if anybody here is doing micro services and you have an architecture diagram describing all Mike reverses architecture I'm pretty sure your diagram has one box per micro service on it and then there's a really nice mapping between a box on a diagram to something in the code we can probably find those micro as it's really easily I'm going to go back to my tweet component for my tech tribes example so I showed you the little class diagram on the right hand side it's a public interface couple of an implementation classes that are package protected and an interface and this is how I've organized my code base so all of that stuff is in its own package and I've added my own act component annotation to signify that this is how about architectural important it's a nice clean mapping that's why when I have my tweet component on my component diagram if i zoom in there's one package in this case that has everything inside it if you look carefully you'll notice that there's a tweet component in pull and a MongoDB tweets Dao previously this was actually a layered architecture and I refactored my layered architecture to make it more architectural II evident there's a whole interesting discussion around layered architectures and why potentially layered architectures are bad and that's too much for this talk so if you want more information on that I did a talk recently called modular monoliths and that talks about how we typically organize Java code inside layers and why that might not be a good idea so now we have a way to identify components what was my initial goal in all of this like half an hour ago well it's to draw pictures and ideally I want to draw pictures automatically so my strategy is therefore very simply this let's get as much information as I possibly can from the code base by extracting it and simply supplement the suffix that's hard to find I think this is a good approach certainly for now until we start embedding more architecture information into our code bases and I really like the architected description language thing I really like the concept of using text to somehow describe my system but I'm not gonna force the architected description language stuff on you guys here because you'll all leave so let's take all of these concepts together throw them in a pot and let's create an architecture description language using Java code because we will know Java and we like Java so here's some code for you this code is is enough code to describe the architecture of the spring pet clinic system so what is the spring pet clinic system we'll basically imagine you work for a vet now at an animal hospital and you are an employee of the clinic basically you use the system to do stuff that's it so we have some Java code from right lasso program and we're going to create a model of our spring pet clinic system and then what I'm going to do is I'm going to create a software system and a person because that's basically all my system is it's a user using the system so we can do that in Java code we create a bunch of nice little domain objects a software system in a person and we say that the person the clinic employee uses the system so that's essentially enough Java code to to represent the context of our system isn't it if we pinch to zoom in well what's the container model all about well essentially it's just a java web app talking to a relational database right so let's describe that again in code so I'm going to create a couple of containers one for my web app one for my database I'm going to give them some properties and I'm going to say that the clinic employee uses the web app and that the web app uses the database so again all I'm doing is I'm very very simply defining manually the static structure of this particular system so now I have the context level and the container level if I pinch to zoom in I don't want to have to take this same approach to manually specify all of the components in the code base yeah especially if it's a really large code base so you know this is code so we can just scrape stuff out the code base and that's exactly what I do so I've got this component find a thing and you plug in different strategies and in this example because it's a spring app I can throw in the spring component finder strategy now how do we identify components in spring well conveniently we annotate classes if are important we annotate controllers with app controller we annotate services with at service and we annotate repositories with yeah you guessed it out repository so let's just use that existing knowledge finds the annotated things extract them from codebase and call them components and that's all my spring component finder strategy does it goes and finds all the annotated classes using static analysis and reflection this runs against the compiled version of the app and there's some logic in there that bundles together the interface and the implementation class I want to get as much useful information as I possibly can from this codebase and that includes commentary and documentation and sometimes that's in the Javadoc so I'm also going to use this Java doc component finder strategy to pull out the class level or interface level commentary and answer that to supplement my component model so that gets me a list of components of a code base and their dependencies we can then use some more Java code because this is just Java code to do something if some additional wiring so we can say go and find me all of the spring MVC controllers and make the clinic employee the user use them and go and find through all of the spring repositories and make them use the database so we're just kind of wiring up the top and bottom of our picture that creates us an in-memory model of our architecture we can then visualize it in a bunch of different ways so in this example I'm going to create a context view and the context view is simply the people and the system's I'm going to create a container view which is my people my systems and my containers and I'm going to create a component view for the web application all of that stuff is called structure Iser for Java it's all open sources all on github and there are a bunch of different strategies that you can use to find your own components naming conventions that sort of thing I will build a Java EE version eventually and the reason this is open-source is because I don't know how you've structured your code so I want people to plug in component finders that reflects their organization of their code once you have this model of your code you can do some interesting stuff with it really really easily so this is a really simple Java Script d3 visualization of that hierarchical model so it shows the spring pet clinic the employee the two containers and then how the web app is broken up into a bunch of components again that's like 20 lines of JavaScript and d3 code also included in the structure Iser for Java open source library is a dot writer so if anyone here is a graph this fan this is all done for you so all you do is you point this putting the dot writer angle model it generates you a bunch of graphs and you can then paste them into graph is and you can get a really really simple basic architecture model out for your spring web application using graphics it's a bit messy a bit ugly but it works really well Michael hunger from the foj took my took my architects wall and throw it into neo4j it's a graph and he has a bunch of graphed just so you can go look at and he's written some cipher queries so again you can start querying the model looking for relationships looking for things that shouldn't be related there's another tool JQ assistant which there's a similar sort of thing with java code for example but I want some pretty diagrams so vendor alert it turns out as of four weeks ago I'm now officially a product vendor and I do apologize for this in advance but all of the stuff I'm about to show you is is all available for free so I felt a tool could structure Iser and it's a web based diagramming tool it's a software service you can think of it as software architecture diagrams as a service which has an unfortunate acronym of sad-ass and that sort of describes me because every time I go to conferences I talk about diagrams and basically what this tool does it's exactly what I showed you so you write a Java program or wall or a number of Java programs to describe and extract architectural information from your codebase and then you throw it up via an API using JSON and these are the diagrams that generates that caveat again it doesn't do automatic layout so you have to drag boxes around that's all you need to do so this is the context diagram for the spring pet clinic system it's the employee using the system this is all from the code you saw this is the container diagram it's the employee using the web app web app using the database and that's the diagram I showed you before so this is the component diagram for the spring pet clinic web application all automatically generated you just drag the boxes around the the commentary the text in the boxes that's the Java coin I've added some styling I've added some shapes you might say well what are the shapes mean well we we have because it's all in the model we can also automatically generate keys out of it so this makes it really really simple to you know create some nice diagrams and and make them understandable the other thing we can do is we can go back to our Maps concept and we can start linking our components to our source code again because it's just Java code and what that means is if you go to the diagrams on the web you can double-click in component and it basically takes you to the source code and get up and that's kind of what I wanted to do I wanted to create a set of maps for a software system at a high level right down to the code so you get your million lines of Java code so when he gives you a set of maps and you can navigate the map so you can jump straight into the thing you need to need to change so again it's that hierarchical model its contacts down to containers down to components down to essentially the code you can also do things like create dynamic diagrams so if you want to show how a particular feature or user story or use cases implemented then we can create some dynamic style diagrams and this is basically just choose a path through the application in this example I'm basically hard-coding the the routes through the application for the view list of pets feature but of course if you were to run your java system and output some stack trace information you could assemble this or reassemble this dynamically and since this is a web base we can just animate it so you know we can have nice simple animated diagrams now this is a small app the spring pet clinic system is not very big at all it's like 30 classes 10 components does this approach scale does that look as bad as it does down here yep now I can explain this diagram to you but I shouldn't have to it's too complex to showing way too much stuff this is the component diagram for my tech tribes web app all of the web app the spring MVC web app controllers are on the top and all of the boxes in the middle are my components but again it's showing all of them we're falling into exactly the same trap here as the other diagramming tools I don't think this code base is a mess but the diagram is when you might say well you could just get a larger piece of paper I know that doesn't solve the problem it doesn't reduce the amount of complexity on the diagram but of course this isn't a static diagram is that this is not a Visio diagram it's a visualization of a model and because it's a visualization of a model it's really really easy to write some more code to slice up the model so in this case we can say right rather than showing all of the components in my web app why don't we find all of the spring web app controllers and create one component diagram per controller slice through the system so we're sticking with the same levels of abstraction but we're just slicing the diagram up and essentially what you get is a larger number of much much simpler pictures and I think this is the better way to approach scalability of complex systems particularly when you're kind of documenting them and of course you don't have to do by controller you do by functional area or aggregate route or bounded context or feature set entirely up to you that's the beauty of having all of this information code throw that sort of concept into your build pipeline regardless of whether you use my tooling or not and you get an architectural model that's continuously kept up to date you don't get that horrible situation where the stuff on the whiteboards never reflects the code you have these big a three or a zero plotted printouts the last updated date says 2008 because it's just too hard to change somebody new joins your team you point them at the maps they're up to date they can find what they're looking for easily so again I'm trying to increase the amount of architecture documentation we're producing but actually making useful and relevant to us as developers and that's really with the entire point of this talk the entire point of this talk is if the Software Architect as a model is in the code ideally by using things like architecture level encoding stars and creating a much modular code base you can get it out of the code thank you very much so that the clock is saying we have seven and a half minutes left that seems wrong five minutes left for questions any questions it's one here in the middle so that the question is this approach goes exactly the opposite way for model driven development where we create a model of our system we tag in some code to make excuse phul and we for generate code and the question was what's my view of model driven development it's not something I particularly like is the honest answer we tried it 10 12 15 years ago it has a couple of interesting applications it's it might be useful for product companies but as a as a technique for building software no I don't think it works not yet anyway it's one down here in front so it's so that the questions am I also able to generate sequence diagrams so we so with my structure Iser turning my as a product vendor no I don't think sequence diagrams show you enough information given the size of the diagrams so this is why I prefer the collaboration style approach but again you could you could very very simply write your own tooling to extract things from your model and throw it up to something like web sequence diagrams comm so again once you have that model you can do a bunch of interesting stuff with it it's yeah it's eating sensible and we know all of that core library that does all the hard stuff is all open source so you can plug new outputs and exporters into it quest over here so what about yeah so the question is what about more complex interactions between your system and other systems like Twitter and you've got a couple of options here I guess one is to is to put some information into the code so that when that code is executed you log out stuff that you can then parse and create a model from the other way is that is to do it manually so if you if you have a small number of integrations you might find it's quicker just to create a view of that scenario manually using code if you have lots of them say in a micro services or messaging style environment then you might want to dump some stuff into your logs and then post-processor post process that and create some diagrams and models a bit later there's no real easy way to do that a couple of questions here so first one so the question is how strong is the dependency on spring so there is a structure Iser spring module in the open-source project and your if you need that then you pull in spring otherwise you don't need spring at all so it's independent of a spring that the core libraries independent of spring and behind so the question is together je from 15 years ago tried something similar so I was a big fan of together day if you don't know together day basically it was a it was a tool and you would point it at your code and it would draw diagrams for you and the two things would remain completely in sync all the time so you could modify your system by adding code and new boxes and things would appear in a diagram or you could drag a class as a box onto a diagram and wire up and magically code would be generated so it was a complete round-trip engineering tool for building systems it tended to be quite slow on systems of any scale and again it comes back to that same thing it was showing you classes it was fundamentally showing you a visualization of the code and I think that's too complex so my approach is as I said is to kind of step one level up so that we are reducing the mantle of complexity that we're showing on these diagrams so that's why I'm really excited that in this as a slightly different approach this is also why I'm really excited about the Java 9 modularity stuff because I think the Java line modularity stuff will fix a bunch of problems that we have now let's go down here right so the questions have I looked at using RDF as a way to represent the architect model and then we can use inference and throw the kind of tax on tax on myself on top the short answer is no but again it's all open source so because you have an in-memory model you can dump it out as any format you want to interesting idea over here yeah so the question is Microsoft video is never going to die right because we still need to do architecture before we write any code and and that's really it's a two-pronged approach for me so if I'm doing an initial upfront design exercise yeah sure I'm going to draw pictures on whiteboards and pieces of paper and I might use tooling like Visio once we have the code that's what I want to start also generating the diagrams of course people need to do presentations as well before they write code especially if you're a consulting company doing proposals so there's always a little need for Visio but I'm trying to get people away from vision as much as possible we have one minute one more there right so so the comment was this seems like code comments on steroids is there not to danger that the the code comments in the code will diverge from the rule architecture and against the answer is if you're using lots of annotations and you're putting this metadata to supplement your code base then that's definitely a risk if on the other hand we find a much better way to extract automatically useful information so let's say we all adopt Java 9 modules and we write a component finances right I'm going to find all Java 9 modules and their dependencies and that's a much stronger model it's um it's a much richer much more accurate model with less risk of floating out today but you're definitely right there are inherent risks and all this superb thanks very much you

Channel: Devoxx

Views: 16,554

Rating: 4.9130435 out of 5

Keywords: Software Architecture, DevoxxBE15, Devoxx, DevoxxBE

Id: oDpdaXt0HQI

Length: 61min 14sec (3674 seconds)

Published: Sat Nov 14 2015