How serverless impacts design - Gojko Adzic - DDD Europe 2020

[Music] [Applause] excellent well thank you very much for coming I hope you enjoyed the conference so far I'm going to talk about how a emerging architectural pattern with a horrible buzz word called Savalas is impacting the way we design software now kind of in this community we are very very very precious and precise about the language we use so I'm going to have to introduce a few words how I use them in this talk first so we don't misunderstand each other and I think kind of if we look at the whole idea of domain driven design most of what people actually do is not designed most of what we tend to do when we talk about the main driven design is mostly at least in my terminology my view about modeling and domain driven modeling wouldn't sell nearly as well as domain driven design so Eric decided to kind of you know use the branding better now I I do mean an impact on design not on modernity because I think modeling is much more bound to a particular business a particular market opportunity a particular way we want to capture the world so I want to talk about first a bit difference between modeling and design so in in the context of this talk modeling is kind of figuring out how we are going to build a theory about the world we want to represent peter naur in the 80s published a very influential paper on how programming is actually theory building we're building a theory about the world around us how it works we're trying to kind of capture that and in the 80s kind of basically everywhere everything was called programming anyway but then we started splitting this thing out so this building the theory about the world the joy of discovering that there is such a thing as a shipping container and that you know for some reason it has legs or whatever that's modeling and then how do we capture that and how do we translate that morning to software that's what I call design so although these two things of course are not exclusive and there's a lot of overlap and we do modeling and we do design at the same time and things like that it's important to kind of make a distinction so kind of modeling is building the theory about the world design is where we stake that theory and we say look you know this is actually going to be a service this is going to be a repository these are going to be you know classes now we can talk about whether deciding whether something is an entity or a value object is modeling or design up until we go blow in the face I don't really care about that but there's kind of you know somewhere on this line we started building a theory somewhere on that line we end with software and in terms of what serverless influence is the severest influence is the second part a lot more than the first one but that's important now there is a third part in this that typical design tends to ignore and the third part of this that's really interesting is to think about deployment now in the context of this talk when I talk about deployment I mean figuring out well this is going to talk to a database and this repository is going to sit on you know a scaled cluster of services this repository is going to talk to a queue and things like that so traditional design tends to kind of imply deployment once we figure out the design we can say we want to deploy this on a cluster of Windows servers or windows is never going to touch this because we want it to work now one other kind of interesting thing about this is most of what we talk about when we do modeling ignores deployment as well I'll come back to this later now the the last term that I just want to introduce before boring everybody do that with terminology is what do we mean by server because when we talk about several s it's a question of what is server and that's not as easy to define as people would think because there's a 1 million blog posts online how service still has servers in so it's not only service and and things like that so in the context of this talk when I talk about a server and we look at the deployment there's usually two types of servers people think about one type of servers are physical machines they're boxes they have lights they blink they hum you pull out electricity they disappear and things like that you pull in electricity they kind of restart the other type of what's called the server is a program that operates a socket it's a web server it's a message server it's a system where you know it waits for some network requests to come in now the context of server less is very very confusing because if you do server less there's a lot more of one type of service and a lot less of another type of service so when we talk about server less generally that means your app is going to run on many many more things like this but you don't care about this stuff anymore that the whole idea of service is that the provider of the platform terminates the socket and the provider of the platform owns the socket your app no longer starts a web server or a message server or something like that somebody else does it for you so services sever less the same way wireless is wireless we all know that somewhere in this room there is a Rooter probably more than one and there is a wire an effort why are stuck into that Rooter but you know we can browse the internet without caring about that wire somebody else manages that so that's kind of service and and all these complaints that yes service in several small of course there's wires in Wireless and you know things like that so um now we're done with the terminology I'm not gonna bore you with the terminology anymore but what's kind of interesting here is this buzzword is horribly stuck with us now for some insane reason you know marketing was good about that and now it's everywhere so it's a lot technically better to think about not surveillance but socket 'less where most of these platforms have a similar programming model the programming model is basically you implement a handler for a request this is how you do a WS lambda in Java there's kind of handlers for different types of languages they're very very similar so you have a request Handler and pretty much you do something interesting when a request comes in now how a request comes in is not up to your code it's how you configure and how you deploy to the platform and deployment becomes really really important the traditional modeling design spectrum kind of implies deployment after design or once we have a design we can figure out the deployment in the service world the deployment becomes really really critical we have to think about the deployment when we do modeling and I'll talk about this thing later because kind of this do interesting stuff later becomes really really critical how we deploy stuff um now an event in this case in 2014 was actually you pushing something on Tokyo and you were able to do that all if you work for Amazon in 2015 you started being able to do that even if you don't work for Amazon in 2016 we started being able to receive events from HTTPS we started being able to receive events from message queues and 2019 you can receive an event into this for pretty much anything that happens on the cloud a user registering a user deregister in a payment coming in a service going up a service going down technical stuff like metric thresholds being exceeded this stuff like your sales dropping incoming emails database records being written kind of pretty much anything that happens on cloud providers you can now act on in this way which opens up some really really interesting possibilities so for example when we look at the different types of clouds and and different types of providers pretty much all of them have started adopting this is a universal integration strategy and and generally almost all cloud providers started copying Amazon now currently this is not currently this is data from about a year and three months ago I don't have any more updated data but about a year ago Amazon held about 70% of the market here but Google had their own cloud functions as your had functions you know Huawei had functions IBM had functions meanwhile IBM decided to open source their functions and donate to the Apache foundation which pretty much means they've given up on it when something goes to the Apache foundation you know you know they failed as a business and decided to kind of just offload it somewhere and and you know Twilio has functions and and and who knows what else and and what's interesting about this is not only is it becoming an execution mechanism where you can design the apps it's also becoming an integration mechanism cloud providers and other providers are allowing people to customize their platform using this stuff what's really interesting about this is because they they own the socket once they on the socket they can do interesting things about this as scaling as securing it as retrying things as monitoring it and things like that and they can hold everything within a boundary that they like to hold with typical installation patterns that's better being online for the last ten years like web hooks what happens is you run a bit of code somewhere the platform provider runs a bit of code somewhere like a payment provider the payment provider sends your web who could then first you need to validate did the web who come from the payment provider or is somebody else forging this and then you have to say okay is this web who consistent with what I expect and then you have to do something with that and then the payment provider has their own policy saying you have to give us a confirmation in less than three seconds otherwise we're trying it again so you're crazily trying to kind of figure out what you can do within these three seconds and you don't know how much the message is going to travel back and then if it fails they doe they don't know if they are safe to retry or not and you need to build the complex system here once the provider on the socket they can deal with all of that and that's what's really really interesting so apart from just an execution platform all of these providers have started opening hooks into their products for example the picture behind me is a configuration screen for kognito kognito is Amazon's username and password management system for users and you can just take that and build an app on top of it so it will support secure authentication it will support integration with various integration mechanisms and you know a bunch of other things but you can also customize it by telling it to run this service function when something happens interesting for example you can see preauth entick ation or there is configured the product time building is used a lot at schools and every now and then we have a clever teacher who registers for a personal account and then decides to share a password with everybody at the school now okay you know we don't want to support horrible abuse but we want to support teachers sharing with you know five or six children or sharing at the class that's work that's okay so we tolerate that so rather than building my own user management system just because I have that tiny business function business requirement that's different from the standard what I can do is I can say look I'm going to use this you know generic thing almost in the domain driven design sense but I will add this teeny tiny bit of functionality that's specific for me and instead of creating a web cook and making this horribly complicated amazon is going to run it for me amazon knows that it's my function amazon knows that it's in my security context amazon knows when to retry a ton of stuff becomes simpler so um this function basically says if we had more than 20 logins in the last 24 hours from the same account kind of reset the password and send them an email that's my specific case everything else is generic so um what that kind of allows us to do is allows us to maybe not think about several s but web hopeless it's a different integration pattern where the provider will run the code and you pay for utilization you pay a tiny bit not for running a service on your end but maybe you know how much the rendered function and what's also kind of really really important here is the provider own security availability recovery in all this other stuff that's kind of embedded in it Simon Worley joked about surveillance and he said that kind of if you've not done devops yet just don't bother at all skip the whole generation by going service because you get DevOps included in the price the provider owns that problem it's not a hundred percent correct but there is you know some truth in it now what is an interesting trend now is we're seeing kind of second level providers not bottom level cloud providers like Google Amazon and Microsoft we think second level providers of applications implementing this is an idea for example Atlassian started a service called Forge it's still in beta which allows you to kind of give them a bit of code to run to customize their products so you know you JIRA is perfect for you but you want to make it more perfect if that's the word you know you just do the tiny tiny bit that's really different at is there Atlassian will run it you don't have to kind of worry too much about that they were worried about all of this stuff Twilio does something similar between the functions metal if I does something similar with 95 functions in fact they're actually reselling lambda they're just adding a bit on top of the price so everybody's happy but this is a integration pattern that is going to be more and more interesting and the reason why this is going to be more and more interesting is kind of we can start building only the bit of stuff we need I don't have to build the user management platform just because I have some you know 5% different functionality from the existing one I can kind of pretty much get all the generic stuff from Amazon or Google whatever I can integrate with different providers for supporting and provide these kind of small bits and pieces that we need and I can really focus on what's core for my stuff so domain-driven design really talks a lot about focusing on the core but this days you know core is a small part of the system there's a ton of other we need to build where surveillance is really where domain-driven design becomes show me the money that's where can really good design starts becoming important so what's kind of interesting here is the pricing model for service is where things become really really interesting technically it's just a container management system that works fast but Amazon Google and Microsoft had grown so big in their services that they can bet that enough people will use something like this for them to keep a lot of reserved capacity Tim Wagner who headed the Amazon run the project in 2016 defined the pricing model as kind of never pay for idle only pay for what you use and they're going to kind of deal with this to make sure it's available so for example I have a small test website that allows me to run kind of performance tests on this it is see if I'm online so I can at the moment I've deployed a bunch of things there I'm not paying for anything because nobody's using it now Amazon Google and Microsoft bet on enough people using this stuff so I kind of get a thousand users to use it at the batch of 10 now they're going to kind of start spinning up new instances now the starting new containers like crazy serving my requests and charging me for what's being served and at this point you know I've kind of gotten between 24 and 43 milliseconds and and 24 to 17 milliseconds 1995 95% usage it spin up 11 instances to handle this stuff by this time it probably killed these instances and charged me for what I used and that's the really really interesting thing about this because I don't have to plan how much capacity I need one of the biggest architectural constraints over the last 20 years figure out how much is the right amount cup of capacity for this reserving the capacity for that paying for the reserve capacity where with this stuff now we are not really paying for reserve capacity Amazon is paying for reserve capacity or Google or Microsoft we are paying for when this thing is used and there's a couple of kind of the pricing model is almost the same for all the providers at the moment they charge you a certain amount of money for a per request and at Amazon it works out to about zero point two dollars for a million requests it's actually prorated per request so what executed was like a thousand so they're going to charge me two P two cents for what I just did and they charge for time spent in the processor based on the maximum amount of memory now these two things are really important for what comes later so you don't need to worry about individual prices but kind of more or less everybody charge is the same Google is slightly more expensive than Amazon Microsoft but you get a bludger free tier per month and kind of the pricing is relatively comparable what what priced is the number of requests and the time spent in the processor times memory utilization what that means is that we remove a architectural constraint that's been with us for the last 60 years there is reserving capacity when we're planning from immortal mainframes to applications deployed to Google App Engine or Heroku we were paying for reserved capacity and what that means in is in deployment not in design in deployment we are bundling things together we have two expensive production machines and we have 500 different services we designed but we deploy them on these two machines because we have too expensive production machines that are properly secured or we've rented 10 virtual machine instances from heroic or Google App Engine we build lots of these different services we designed them to be decoupled with this I need them to be isolated but we deployed them together for example with the product I'm building we do a lot of file conversions we have about 20 different file converters one file converter is converting from the source format to PDF it's used all the time like everybody's using it all the time probably this very moment about 10-15 instances are processing that then we have markdown that I built for myself I use it three times a month probably nobody else uses it and we have everything else in between now if we wanted to kind of do this properly and you know we designed these things as isolated we'd need to kind of keep about I don't know 200-300 instance is ready for failovers for kind of plant capacity we'd have to isolate every one of them and plant and that would cost a lot more than the actual capacity we need because I can serve everything with about 10 15 instances so what we did when we were on Heroku where you pay for instances you pay for itself capacities we designed them to be isolated but we couple them together in deployment and then the SVG exporter we had a bug and it did not clean up temp space correctly when you fill up the temp space on Linux it starts misbehaving crazily so all these ten instances we had basically like dominoes fell because of that bug now the point I'm trying to make is this we design things by implying deployment and we design things to be nice and d-cup of them brilliant but then we deploy them to be coupled because we are paying for reserve capacity and then we couple them without intending to do that and and this is kind of the problem because so far deployment architecture was rewarding that type of work now with Amazon and with Google and and and with as your functions what you pay for is the number of requests so the number of requests if you have 50,000 people asking for PDFs 5,000 people asking for images and one person asking for markdown you pay exactly the same whether it's deployed completely isolated every single thing into its own application or as a single application even better if they have different memory constraints if the PDF one that's using ghost script is a memory Hagen needs a lot of memory the SVG one is working on text files by breaking them apart you pay less by deploying them individually you pay less because the SVG can work with less memory so um because we are paying for utilization not capacity not instances not environments and things like that lots of interesting things open up and one of the really interesting things that opens up is basically several starts financially rewarding good design decoupling things what we've been preaching for the last thirty years designing things to be nicely isolated having nice boundaries and things like that can recently be rewarded when you bundle everything together the reward for that is all when the business changes six years in the future we'll be able to kind of easily change this stuff that's why there's so much technical debt around the world because business people don't really care about that sever less immediately rewards good design and that's why I'm so obsessed about this so I'm here's some hard data for the product I'm building in 2016 we started moving from Heroku that's reserved instances to lambda in 2017 we cannot finish the migration during that time the number of the users increased about 50 percent our hosting costs actually dropped about 50 percent which means that we had about 66 percent saving there on our costs and not only that we were much much much faster deploying the stuff we wanted to deploy in a sense of taking business functionality to the users because we could focus on individual things we want to do we could focus on kind of the core not the generic or the supporting part IDC did some research last year and they claimed that based on the companies they surveyed the average company saves about 60% on operating costs that's kind of consistent with what we did but also they're about 89% fast they're taking business features to the clients and I think this is this is kind of this is the money shot this is the you know give me the money version of show me the money version of severance because this is what's going to put pressure on people to adopt this kind of stuff um now some people go totally crazy with this a big UK charity called Comic Relief usually has one big auction per year now that big auction per year is massively massively spiky if you don't have enough capacity for those two hours you're losing money if you have the enough capacity for the rest of the year you really don't care things fall over all the time they're in panic mode before that and they did this talk you can you can watch this video my slides are online now so you can just download this and finally Shira so their campaign management they're kind of processing payments and things like that went down from 83,000 dollars for that kind of two hours to about five thousand dollars that's ninety four percent saving just because they don't have to worry about planning for capacity and predicting how much they need they don't have to figure out in what parts they need this Amazon or Google takes care of that to Microsoft for and so that's kind of the interesting thing about this now when we look at how to get here you can't just take an app that you've designed to work with containers or servers and and and take you there there are libraries for that but you're not going to get anywhere close to these numbers in order to actually get this benefit we need to start thinking completely differently about modeling and design and that's kind of what I want I really want to talk about today so one of the things that kind of is starting to happen in this world because we are no longer paying for you to you no longer paying for reserve capacity there is no benefit to think about big applications anymore there is no benefit to think about how I'm going to kind of put these three services together and this thing and this thing and there's this unbundling happening from applications to tasks and then each of these tasks can scale independently when you bundle something in a big up and then you say okay we need more capacity then you get more capacity for everything you don't really know whether you need the same amount of capacity for invoicing as you need for payment processing right maybe there are two completely different things but the way we were deploying stuff was rewarding bundling it together so it all had to scale together with give coupling into tasks then the cloud provider can kind of figure a tow for this payment processing thing you need actually more capacity now I will give you more capacity for the other stuff you really not using it now so don't worry about that and that's why I think you know when you look at what's actually used in an app at any certain point you know who knows I mean it's very very difficult to know with bundled apps because Amazon or Google or Microsoft are geing per request that's really interesting they need to track what requests are going on how much they cost so you get like micro level cost of the flows through your application if you decouple it properly if it's all bundled then you just get a single price when you get micro level costing you can make better decisions around well this thing is costing us a lot is it actually that valuable should we offer a simpler version of it should we stop offering it or should we invest in it or this thing is costing that's quite a lot it's making a lot of money for us hey excellent so this is kind of where I think part of the design changes is designing more around tasks than designing around apps and another part of that it becomes a little interesting is rather than thinking about one single core domain that's kind of you know this wonderful thing that we're going to build and protect and design and things like that we actually start thinking lots more about well what is the core piece of this task that we want to do in user registration what is different for my app from all the other apps what is the teeny-tiny business kernel of that and then we don't have to think about well is user registration really core for us that's how do I decide don't know like you know Nick talked about this morning well you know maybe this is caught maybe that it's called maybe search if you talk about these searches that really not caught for us but maybe a little tiny piece of it it actually is thinking about big cores doesn't allow us to do it thinking about small tasks and small kernels of these tasks allows us to kind of start thinking about that in a in a much much better way so rather than thinking about core supporting genetic and things like that for kind of the whole thing we can start thinking about core supporting genetic for teeny-tiny workflows and thinking about them and optimizing them and designing them so kind of the good news about this is because it's so if you if you decouple an Optima you know lots of small tasks pretty much you get isolation out of the box you get blast doors or fire doors everywhere you get mistakes that are very very easy to fix because they don't propagate and the whole approach is very very tolerant to making a stupid design mistake because hey we made a stupid mistake in this service we'll just kind of you know rewrite that and everything else will continue working as it is and we very often look at things that the product I'm building there's only two of us building it we do everything from sales and marketing to support so about a month ago we were looking at some piece of code we wrote three years ago so two hundred lines of code like we wrote it together pair programming neither of us could understand how it works just staring at it and staring at it and you know scratching our heads like so instead of you know spending hours to figure out how it works and then doing characterization tests doing lots just which is threw it away and wrote a new one which kind of we kept the knowledge we throw away the code we fix the design mistake so the whole approach is very very forgiving to design mistakes that's the good news about this and if you start thinking about the you know decoupling from apps to tasks the only reason why we didn't do that before is it was too expensive to operate thinking about I'm gonna operate every single task of my upon a separate infrastructure was like insanely expensive now it costs exactly the same as operated on a single infrastructure so the benefit is no longer that so I think I'm with traditional domain-driven design people spend a lot of time thinking about where are my bounded contexts er my bounded contexts are on teams are my bounded context around products should this be inside this context or outside this context with debugging you know D bundling this stuff to tasks and decoupling it to tasks well hey you know each task is potentially context like who cares I mean if we made a mistake we'll just fix it not a big problem at all making a mistake about kind of contexts in a single app is usually expensive making a mistake about context boundaries in a decoupled app is no problem at all and I think that's why this is so powerful and empowering for people who design because yes you can be a sloppy designer like me and still make a successful product makes a lot of money but it doesn't have to cost a lot of maintenance so um another thing that's kind of really interesting here is with traditional design around applications we tend to look at conceptual consistency as the primary motivator for boundaries like a payment means something in this its words its names its structure a payment means something completely different here we're going to put an anti-corruption layer we're going to kind of make it complicate your gonna translate but kind of a lot more I find myself doing boundaries around well this thing needs security access to this database therefore when it loads a user the user has this meaning this thing doesn't ever access the database when it gets a user really like you know that can mean a completely different thing I'm not saying you should do that every time but there's an option of kind of structuring boundaries around security knox and I'll talk about I'll talk about this later and kind of the most important thing here is when you decouple this because the deployment is decoupled the blast radius for change is relatively small the blast radius for mistakes is relatively small because they're all decouple into different tasks so that's the good news the bad news is the basic level hello world is a highly distributed up and you need to know how to design highly distributed apps from the start and I'm really talking about design fear not modeling but designing highly distributed apps is something that not a lot of people have experience with because you know you don't need a high throughput transaction processing system usually for most of the work you need to do um so this is where the problem becomes we cannot really ignore distribution traditional modeling and design imply distribution once you to design the distribution is implied we design for service if you really want to get the benefit out of this you need to consider distribution all the way through you cannot ignore it anymore now kind of one of the I said several s deployments are very very very tolerant towards mistakes in design there is one particular mistake that it's not tolerant for and that's Martin Fowler's first law of distributed computing now in 1995 or something like that Martin Fowler published a book called the patterns of enterprise architecture and in patterns of enterprise architecture he defined the first law of distributed computing and lots of people misquote that law and misremember it online because you can find 1 million blog post they talk about how martin fowler's first love distributed computing is this it is basically you know if you are deployed on a distributed platform you cannot really obey that now martin didn't say that what martin said is don't distribute your objects that's the full first law but that tends to be too complicated to remember and the even understanding objects and that's a really good point that's what i want to talk about you know what do we mean by object what martin meant is you know don't distribute in the domain driven design sense don't have a single aggregate on multiple machines at the same time because you cannot guarantee consistency it's going to be really difficult to work it so domain driven design has you know a bit more to say about this of course but so one of the things that becomes really really important is how do we design stuff to obey this but how do we actually not make the system a distributed monolith where everything is tied to everything else because we're not distributing objects and that's one of the biggest challenges we gonna have so can with seven as basically time is money really you pay for utilization so if your application when it loads up needs to talk to three databases and need to kind of save something to four database that task is gonna cost a lot if the task doesn't need to do that then it's gonna be much much cheaper and that's how people save money by kind of considering deployment so I'm a is a really stupid example of this say you have a user uploading a file and the user has some limitations what you know they're allowed to upload and what they're allowed to do in the system and they have an intent to do something with the file they want to you know convert if they want to save it they want to send it I that you know so a not a good or bad but let's say a way of designing this would be to create an aggregate around the user the intent and the limits that's where the boundaries of consistency are and then this is file upload that has a reference to the intent now in a database that would be a foreign key in a kind of Java program that would be in memory reference or something like that but the kind of designing something like this and then unbundling into multiple tasks becomes interesting what does a task that does this processes this file upload need to do well that one will need access to the user database to figure out who the user is that one will need access to the intent system that's probably kind of some file system of change system somewhere it will need to connect to kind of ten different things this is an excellent way of paying amazon more money so if we look at kind of you know minimizing the chattiness because chattiness costs a lot then we're going to say well you know yes this is a nice little aggregate on paper but what I actually just give me the you know give me the file id give me the intent type give me the user directory where the files are I'm gonna break this aggregate completely and this is the problem that this is the fundamental problem that we will have designing for several subs this thing saves a lot of money but is it is it clean I don't know you know what and conceptually what is good design it's good design something that saves money now is good design something where you know you can change it to some potential future later how do we balance these two things so I think one of the critical challenges here is figuring out what do we even design and I mentioned in a kind of if you think about applications you know deployment usually kind of we model and then from the model we infer the design of course that's kind of you know that happens at the same time but kind of deployment is an afterthought deployment is something that happens after you have a design if we really want to benefit from this we need to know this is going to be deployed separately like this so I can design an aggregate around this stuff we need to understand the deployment as well so I think kind of in the service world the intended deployment has to influence the model and that becomes a challenge and and kind of the model influence deployment and the design comes after that the design is Co influenced by this another thing that kind of that is really big differences with the traditional design because we don't consider deployment we don't consider the element of time as in time in memory time in processor now that's ignored you design a service will the service be transient start run appeal I don't know with this we kind of tend to design much much more shorter live tasks and maybe that's one way of reconciling this because the problem with breaking aggregates and consistency and things that is the danger it happens when people start modifying stuff if I design a smaller task that I know is not modifying this and it only lives for 200 milliseconds and we're not going to keep it around for long then oh you know a lot of these constraints are no longer necessary to kind of put there and one of the things that is really really interesting is in traditional application based design data transfer is a complete afterthought data transfer is something used a genetic library to serialize it through JSON or serialize it into XML if you're really crazy or you know if you're so inclined see there's some kind of binary protocol like Google protobuf errs that's completely synthetic we designer objects and then we're gonna see realize this you know god forbid user normal something like that please don't but it's kind of enough to thought of that where with several design data transfer is actually the thing we need to focus on it's kind of figuring out what bits need to travel through the network so that individual tasks have enough information without chatting too much to other tasks and that's really kind of completely different from how we are thinking about design now in this case what I think is going to happen as we evolve practices around these events will become kind of mini aggregates and mini aggregates in a sense that there are regrets for a specific purpose there are tickets for a specific task for a specific group of tasks they're not aggregates in a kind of deployment deployment kind of ignorant way and one of the other things that can really becomes interesting fear is when we think about design of applications we usually talk about well focus on the core and design the core well all the ugliness just pushed the boundaries we will do an anti-corruption layer like we'll unit test it well well you know translate there don't worry about that and Nick was talking about this morning how you know people are obsessed with boundaries I don't think so i think people ignore boundaries when the design and I think kind of the problem there is the problem here is with kind of several as being tolerant to design mistakes apart from the protocol really the boundaries are the things we need to be designing and everything else is fixable later if your service has a good protocol in and out you've made a huge design mistake in it it works but it's okay just replace it the protocol is the same no worries if you've not designed the protocol well nothing is going to save you so I think it's kind of this is a bit turning the design on its head we need to design different things and again this is coming from highly distributed systems I remember talking about this with somebody who used to work for the election for about 30-40 years and so haha well you know Erlang systems 30 years ago we had this rule designed the protocol everything else doesn't matter highly distributed system that's why I think you know people don't really think about this the right way so um these two ways of designing protocols for network that I know about this kind of some the rivets of this this kind of remote procedure calls remote procedure calls are pretty much we have a tight coupling between the caller and the callee they share an interface it's kind of usually a shared kernel or some library or something like that and the remote procedure call tends to pretend that the network doesn't exist usually they're done by proxies which is you know you magically create proxies these were web service proxies or there were javabean proxies or wisdom proxies and and it's every 5 or 10 years somebody invents another technology that's magically gonna solve all those problems through this and we have events or messaging events the messaging our kind of model per host events our messaging a saying look a user registered like if anybody needs to do something about this do it I am NOT going to imply send an email or you know confirm this or like whatever you wanted to do it so events a messaging don't kind of they communicate maybe an intent or a user or a fact that happened they don't tie with the interface on the other end now the big difference between these two is about two years after people implement RPC they kind of go and cry and suffer badly because the reality about network not existing when it actually exists hits them and then everything falls apart and we've learned that kind of event based systems are a lot better and of course you know the stuff in between you can send kind of commands and things that but the really interesting question becomes how do we design and I think this is kind of one of the core problems that we're going to have in the future how do we design these events so that they are not coupled to the receiver interface they're not coupled to the receiver implementation like hey I use a registered but they still send enough information to these things so they can finish without chattiness it's very easy to say he'll a file upload happened with this file ID an intent ID but then who needs to work well you know go and read ten databases about this but if we say hey this file ID happened with this intent in this limit how are we not really implying an interface of what needs to read it and balancing this stuff becomes really really challenging that's why I think these boundaries having meaning and in language and events around the single receiver is bad because then we're tying ourselves too much to this but maybe what's going to happen is kind of we need to start thinking about decoupling these things into groups of services so that additional application based design will say okay we have this kind of payment service we're gonna you know it's payment so it's important so we're going to put the best security people on it they're gonna design it like crazy it's going to be secured it's going to be scalable it's going to be wonderful it's going to kind of you know use cues to a synchronize things and then talk to payment providers and one of the things becomes kind of you know how do we how do we decouple this into tasks and how do we decouple this into what is the right context for this so we had a relatively interesting design discussion thinking about deployments when we were thinking about this and one of the things that you usually have to use for every single payment provider is you have to use their own library for this now when you have to use their own library it comes with five million dependencies because you do maven install it the Downloads the whole internet you do npm install in dollars the whole internet and we kind of had a problem where one of the payment providers we were using had a security vulnerability in their dependencies but like so there's a trade-off like do we want to use that payment channel we're getting a lot of money from it from a business perspective yes we want from a security perspective like oil they have a like vulnerable dependency there which is deployed in my payment service so all my securing stuff around it doesn't really matter we know it's has holes in it and because it's a single payment service it has full access to this database okay from a security perspective no and what we started realizing is kind of maybe thinking about these boundaries around security needs is interesting like what does this task actually need access to this task that has access to their library well it needs to kind of talk to the network and it needs to talk to this thing here it doesn't need to talk to my database if I start thinking about tasks not service your applications I can start thinking about well you know I'm going to have a function or a task that talks to this payment either we're gonna put their library in it it has access to the network it doesn't have access to anything else it can post a message on to the queue and that's fine and this is where kind of things became really interesting because what we realized is kind of you know there is a context around this more generic stuff that we have and the in that context we used our models we used our language we pulled out everything that has dependencies on third-party libraries outside the payment service and then allowed every single provider to use its own language we had a stupid amount of complexity when everything was together on Heroku because we in the same memory process we had to translate from all of these different concepts that people have and figure out what what needs to do and now we can actually have well in this context here it uses the paper language in this context here it uses the stripe language in this context here well you know it uses our stuff and of course this is kind of a you know with hindsight this is a good solution but what I'm trying to say is when we started considering security needs we started seeing boundaries that we didn't see before and this allowed us to kind of think about well what library what third-party library do we want to load here does that actually require a different context different language does it need access to the database if it needs access to the database maybe it's inside this context maybe it's not and and we start to think about so I think kind of one of the really interesting things is kind of week we started looping tasks as bounded contexts around the runtime security needs and that's why again why I think ignoring deployment where runtime security needs come in play while we're designing is kind of a wrong thing to do so another thing that is interesting kind of to think about in all I'll finish and this is when we look at infrastructure the infrastructure really when we are designing applications it tends to be stateful or stateless you know we know that this service is stateless we don't care this service has a cache we care and things like that where because a provider controls the infrastructure the provider basically can just throw away your service at any given point they can drain them they can send stuff to they can spin them up and a lovely about that is you get a lot for free for example in 2018 there was this problem with Intel processors and you know everybody went crazy about that and a month later there was another security issue there that was kind of less publicized but we got an email I woke up eight o'clock in the morning I got an email from a school administrator in the u.s. saying that you know this vulnerability an Intel processor has been discovered they want all their vendors to kind of immediately say what they're doing about this I'm still waking up you know my head's not in the right place I'm drinking coffee so I copied the CV identified I put it into Google to find what it is the first result on Google said Amazon and has already been touched and that that's why you know this is amazing you kind of you know security is a wonderful thing but it's not my core domain I've hired that from Amazon and because they want infrastructure they could just kind of drain stuff replace it and nobody is kind of you know interested about anything so can we're moving away from this idea of result capacity to utilize capacity and not even that but we're getting crazy functionality now like you can do a gradual deployment this comes out of the box you can do while you're deploying stuff you can say well give 10% of the requests sent 10% of the requests to the new version of a up send 90% of the quest to the old version of my up or kind of gradually move that thing around monitor for something weird happening if something weird happens send everybody back to the old version of the app center you know kill the new version or if nothing weird happens just keep giving it users five years ago you could only do this if your Amazon or Google or Microsoft or Facebook maybe doing this would cost twice as much as operating your production doing this now doesn't cost anything more than operating a single version because it's priced based the number of requests and stuff like this becomes amazing because you can also I you know in a weird Inception way define a lambda function that tells this thing what's something weird is so you can monitor for example sales conversions on the new and the old version that's your unique thing deployment exceptions are not unique your unique thing is how your truck sales deploy that then you know send a new version and for some weird reason the new version of the app is making people buy less well there is a bug it might not be a bug that was caught by unit test integration test there is a bug it's you know something's weird like rollback and again that's allowing us to do kind of interesting things about this so kind of um to conclude with surveillance there's a ton of ton of benefits from considering deployment when we're designing and usual application based designs they tend to model and in design thinking that kind of okay we know that this model is going to evolve over time but at any given point the model will be consistent like at this particular point the model for a user in my application is this but if I do this well I might have two versions of application running at the same time what is the model for a user like what do we do if this first tasks the safe stuff to the database has a new version that saves a different model to the same database and the previous one can you know load that during this transition and I think this is where can we need to start assuming that model no longer just evolves over time but kind of evolves over space as well and that at the same time in a deployment we might have an inconsistency between the models and what I think this guy is going to lead to is kind of it's going to lead us to think about well model not as a universe but you know this comic multiverse stuff where everything is connected to everything else and it's weirdly similar but different in some ways but if we can achieve that the benefits are amazing what we can do if we can design an app for this is yes we can do gradual deployments but we can also do stuff like oh we have a really important customer they have a specific need we can implement the need for them in one week but in order to make that consistent for all our other customers we need about three moments usually what would happen is you finish that then you make it consistent for three more months then you give it your customers this way hey we finished enough for this customer completely deploy a new production version of the app for this customer give it to them let them use it let them benefit then clean this other stuff up and then move people gradually over to the old version it costs exactly the same as the other approach just three months faster because it's priced based on utilization not priced based on environments so can what I think we need to start doing is think about version tolerant design think about think you know how does something upgrade and downgrade and regrade and who knows what else with the aggregates that are coming in and we started putting version numbers on everything we send out that allows us to say okay you know now we are replacing the service and we recently had to replace the stripe integration no problem at all we say okay you know we are keeping pup keep pumping messages version 1 into the queue the old one is picking them up we pump 5% messages version 2 into the queue have both versions running at the same time see what's happening and stuff like that is really really interesting to consider because I don't think we think about versions in our model enough when we design so that's pretty much it for me for today I hope I've kind of tickled your imagination at least a bit I wrote a book about several s deployments and and design it's called running service and as a small present for suffering with me through our you can get like 50% off if you go to this URL this week thank you very much [Applause] [Music] you

Channel: Domain-Driven Design Europe

Views: 1,644

Rating: 5 out of 5

Keywords: ddd, dddeu, ddd europe, domain-driven design, software, software architecture, cqrs, event sourcing, modelling, microservices, messaging, software design, design patterns, sociotechnical

Id: jZoWAQOaxt8

Length: 54min 56sec (3296 seconds)

Published: Wed Oct 14 2020