Udi Dahan - If (domain logic) then CQRS, or Saga?

for those of you who are seeing it for the first time my name is Eddie Dodd I'm a I don't know what you could say DDD CQRS microservices formerly SOA recovering addict this is the place a high OD we love you that's an American thing I guess so I tried to come up with the most cryptic title for my talk while using enough of the buzzwords that ultimately kind of say well there's something there domain models yeah you know that's close and CQRS we've heard about that and sagas who's heard of sagas before okay wow a lot of you good so we're going to be coming at these concepts sort of in a roundabout way the idea here stems from a very simple statement that has been the staple of almost all programming languages from the very early days and that's the if statement now most of us we don't think twice about typing in those two letters whether it's in the form of validation logic or business logic or calculation or any type of place where we need to do branching you know the ifs go in there and only in sort of really rare situations where you have sort of an if and then another if and then an if and another if okay we'll switch to a switch case but really beyond that we don't spend too much time considering what are the the risks around using this construct and the realities are not really programming risks they're domain logic risks where that if statement sort of lets us off the hook fairly easily so that's what we're going to be spending our time talking about them my hope here is that I give you some new ideas and new questions to ask your domain experts when you come back from this talk when you come back from this conference and hopefully that will be the kind of thing that lead you to a a deeper understanding of why the domain works the way that it does or in many cases to uncover a lot of rich domain logic that you didn't even know needed to be taken care of so the if statement that that we use kind of has two major roles the first one I'm going to call it if the protector where it's coming in there and all of the validation logic and all the constraints and making sure that strings aren't null and all that kind of stuff this is the kind of stuff that we rely on in order to do our traditional defensive programming and for the most part this is how we think about the if statement it's the thing that protects our code from all of the evils of the outside world but I'd like you to see the other side a different kind of if statement if the deceiver you might know this guy if you've seen the movie the devil's advocates and at Al Pacino portrays this partner at a very large law firm who later on we find out is actually the devil but he kind of insinuates his way into the protagonists life tricking him and laying all sorts of little traps for him to end up on a path that isn't so good for all of humanity and I'm going to try to help you see this dark side of the if statements that we're using and it's not going to be the kind of thing that's immediately apparent I'm going to have to start kind of upstream in terms of a given scenario or use case or problem domain but as we go forwards I hope you'll see how that deceiver pops up for us so I'm going to start with the example that if you've seen any of my talks you know that I go on and on and on about it's my friend Amazon who's bought something online with amazon everybody anybody not bought something online with Anders Verne oh we got their hands out okay what's wrong with you people so in terms of buying stuff online we've got our search screen which allows us to type in a couple of words you know domain modeling domain driven design and it finds all of the wonderful books for us about domain driven design and then after we see a book that we like let's just say it's Eric's book and say any okay great look at that and see that it's in stock and all that kind of stuff and then come along it's all right I want to add that to my shopping cart right now this is the point in time where we move from the query side of things which is what we've been doing so far right doing the search is the kind of query looking at the details of a given product is also a kind of query you know what other related products there are all of the type of information that pops up is a query when we go to click that button add to cart is when we switch from the query side to the command side so if you've heard about CQRS that's really what we're doing here linguistically is starting to evaluate what's the responsibility of each part the command side in the query side so one of the important questions to ask ourselves when dealing with any type of command is what would cause that thing to fail now here I want to distinguish the technocratic contributions for failure of well the web server that I talked to crashed or I couldn't connect to the database or we got a deserialization exception all of those technocratic reasons should be handled by some kind of infrastructure whose name rhymes with and service bus so we're not going to get into that now we're going to focus more on the logical contributions for failure why might a command fail now one of the answers that I get repeatedly every time that I kind of ask this question well why would something like this fail well what if somebody deleted that item right if the item is deleted obviously you can't add it to your cart so that would be a logical reason for failure I couldn't find the item in the database now I want to spend just a couple of minutes talking about deletion because in in all of my work with customers and developers all over the world this this example this what if the thing was deleted is brought up again and again as to why something would fail so I want to spend just just a minute talking about deletions developers end up still doing delete delete delete but one of the questions that pops up is well if I delete a product what about all the other things in my domain that are connected to that product should we cascade the delete onwards so if I colita product does that mean that every single purchase for that product in all of history should also be deleted quick question all right poll again who has done cascade deletes once in their career okay now you know where the second question is going to be who's down cascade delete twice in their career right it's the kind of thing you do at once and all of a sudden you realize just how far those dominoes go right it's video you go from the product to the orders and then the orders you know ultimately the customer and then from the customer to the account rep room from the account rep and before you know it half your database is gone and Twitter is ablaze and saying what's wrong don't you have backups have you heard that one and you're like yes we've backed it up five different ways but none of them can be restored those poor poor guys we have all of our stuff on github by the way not to say that it couldn't wouldn't happen to them but it's one of those things that that really drives the point home you got to be careful with the deletes so alright we know we don't want to Cascade delayed it's really dangerous it's definitely not the kind of thing that should be applied sort of at a at a raw database level without any sort of domain concerns around it so once we start talking about delete invariably developers are saying well delete is risky let's move to a soft deletes who's who's done a soft delete before in their system yes it's kind of the pendulum swinging back the other way delete was so traumatic that we're not willing to delete anything anymore and then we soft delete and also I have a kind of theory that that was probably a contributing factor to why events or saying got so popular so quickly because it's based on that same philosophy if they don't delete don't overwrite don't put yourself in a position that you can't roll back from and that's something that yo we all like to have a safety net now in terms of that soft delete my issue with it is that it's still very much a database up perspective it's in that sense of okay we did a delete we probably shouldn't have let smart do sort of a soft delete thing on top of it and then that way we can unsought the leat the previous soft delete and then we haven't lost data but soft delete is is kind of a kludge when you try to apply it across an entire system so imagine an environment where a user has just added a new item to the product catalog all right so they type something in they look at it they're like oh it's a duplicate and now they want to delete the thing that they just added now of course you say oh no problem I'll do a soft delete on that and then there's the whole you know they click delete you're like okay soft delete and then you put that in sort of a special status kind of place now you know get rid of it and then they have to go into the soft delete area and then they click delete you're like are you sure now we're really going to delete it this time in it like yes believe it doesn't provide for a really good user experience when you have users that are in sort of data entry mode where they're doing a lot of you know get something in merge this and they kind of want a sandbox in which they can work without having to worry about all of the Cascade delete effects or to have to deal with soft deletes or all that kind of stuff in essence one of the things that I think is missing from a lot of the systems that we're building it's not the soft delete feature but rather that distinction between the system as a whole that all the users can operate on and a sandbox the difference between private data and public data now when we think about private data and it's that sort of data entry scenario somebody's at saying okay I want to create a product and create another product okay change that set the price put this image no I don't like that image put it in this category put it in that code the kind of stuff where they're that they're kind of making a mess for those of you took part in Alberto's event storming session you know what that's like right sticky notes all over the place sometimes you got to make a mess in order to figure out what's necessary what's essential and to distill things down we if you have an experienced event storming I strongly recommend it but a lot of times our users operate in domains where they have to go through a similar process they kind of have to make a mess but it would be nice if they had a little private sandbox that that mess could be consolidated in and they can put sticky notes up and throw them away without the system call silly asking them are you sure you want to totally delete this previously soft deleted thing gets really annoying really quickly so that's the private side of data that again I think is missing from a lot of systems that we tend to treat data as if you know there was only one state once you create it at that point in time it appears for everybody else and they can do whatever they want with it and that starts to create problems for users so if we have this type of staging area we have this kind of sandbox where we allow users to create read update delete without having work to worry too much about applying all of the domain logic and all of the constraints and all of the invariants and all of the important stuff that's so relevant in the public domain it can give our users a much better experience and solves the problem for us of delete versus off delete and then the one final piece of the puzzle would be to include a kind of final publish step so for those of you who have used content management systems or blogs or things of that nature you've probably seen that button there you write the blog post and you can save it but it's not published yet and then there's that final step of saying are you ready to take your data from the private domain where only you can see it out into the public domain and sometimes it's just those two little pieces of having a sandbox and having an explicit publish step that solves a lot of the problems around delete so that we can have true deletes in private but then the question becomes what about the public side of things that's where we were before that's why we created the soft elite to begin with so anytime you're building a type of multi-user system where you have the that you feel the need for a soft delete oftentimes what you're missing is a private data sandbox that will enable your users to create read update delete the data that they need and just get on with things now what do we do about the public side of things what do we do about the product that already has been published and now is available for people to buy and they've bought it and we have all that history at that point in time we need to replace soft delete with something a little bit more business centric so anytime you get a requirement from a domain expert saying I want to delete this your first question is okay so sort of a private sandbox delete in which case absolutely you get to delete it versus the is this a public domain entity that can have relationships to a whole lot of other things in which case delete is the wrong word tell me why you want to delete this product that question the why do you want to delete is going to give you the more meaningful business oriented command in which case in our domain of retail they can say well I just don't want that product to be for sale anymore in which case we can turn around and say oh okay maybe let's not use delete maybe let's create an explicit command that says mark item not for sale so anytime you're in a public domain type of area you get a requirement for delete figure out why have the discussion you'll uncover some sort of business meaningful command behind the scenes and then that's going to start to lead you on a very interesting path of more domain logic that's related to that specific command and to distinguish all sorts of additional states that can happen for your entity as a result of that so now that we have the situation where the business has told us it's not a question of delete versus soft delete but whether the items for sale or not now we can say great in the add to cart' command processing we can go and implement the check there's that if statement if the item is still for sale great you can add it to your cart but if the item is not for sale will return some kind of error so it could be as simple as returning false it could be throwing an exception there are all sorts of ways that we can tell the user why they can't buy that item now here's that if statement that we were looking at before all right that's where it came from start with a delete move to a more business centric State and now we have if the protector at play telling us well you can't buy that product it's not for sale anymore now the thing about that if clause is that when we get the requirement we're like yeah it makes sense a person should not be able to buy an item that's not for sale to put it into their shopping basket but now we need to start to look at things on a larger continuum so not just to Add to Cart button by itself but to go back a screen to the search results right before we got to the point that we were going to add an item to our shopping basket we saw the item in search results arguably all of the items on the search results screen should have been for sale right the query should have taken into account that state the same way so now we're in this kind of interesting situation the user comes in they do a search they see a bunch of items and they're all for sale they click through to the next screen they see the details of the item it's still for sale they go to click that button that says add to cart all of a sudden it's not for sale how did that happen those are the questions that we need to start asking ourselves and to realize that what we thought was if the protector in our domain logic is actually if the deceiver saying pay no attention to the search screen that just two seconds ago the user saw that item on pay no attention to the fact that it used to be in a different state pretend like there is no such thing of history and only what's happening right now matters only look at the code right in front of you and that's where in essence we're making that same sin as before of the delete of looking at things through a very narrow data centric point in time perspective and once we start opening up our view and we start realizing wait a minute it used to be for sale now it's not I'm adding it to the shopping basket somebody else must have changed its state to being not for sale in essence what we have here is multiple actors working at the same time on the same record right in essence the if statement is almost hiding that kind of race condition from us we assume that it's the kind of thing that must always be true you might have heard the term invariant at least once or twice in this conference the kind of thing that we assume yeah you know you should only able to buy items that are for sale that should always be true but the way to evaluate that is to say well if the item previously was for sale and I'm going to add it to the cart and somebody else is marking it not for sale now that we have a race condition we need to pick things apart a little bit more after those you've heard some of my talks or have been reading my blog you know what my stances on race conditions in any race you need to be prepared to lose when I say you I don't mean the person that's adding the item to the cart I'm talking about that race condition on the other side of marking the item is not for sale right because our assumption was that when I'm going to add item to the cart the other user already came in and marked it as not for sale but what happens if things get switched around what happens if I added the item to my cart first and then 20 milliseconds after that the mark item not for sale got evaluated doesn't that mean that I now have an item that's not for sale in my cart that is unless we fit tell the other user in the back office I'm sorry you cannot stop selling that item people keep putting it in their carts thinking yeah that's that's a pretty dumb idea right there was that situation with the call of the hoverboards those those things with wheels that the teenagers ride on and almost kill themselves with they started blowing up no I'm not talking about the Samsung phones that the other thing was blowing up it's getting really dangerous if you ever noticed that all sorts of things they used to be kind of toy fun technology type things starting to blow up and come dangerous yeah so in any case you wouldn't want there to be a situation where we discover an item is blowing up and it's injuring people they keep buying it we say stop selling this thing say sorry people keep wanting to buy more and more of it in other words the mark item is not for sale command should be successful in any case regardless of how many carts that item already exists in so when we have this race condition where we can lose in essence it means the fact that we had the if-statement in there it doesn't really help we can still end up with carts that have items that aren't for sale in them and that's the thing that we need to take away from this if statement is that we think because we put the if statement in our domain logic because we use the magical incantation called invariant that somehow that would automatically apply and be true across this what I call collaborative domain a collaborative domain is one in which you have multiple actors that are operating on the same data in parallel and again bunch of you have heard me say this before if you're in a collaborative domain you need to start thinking CQRS but collaborative domains they don't always sort of jump out at you right in your face as all yeah this is a collaborative domain most people would have thought what do you mean I'm going on Amazon and putting stuff in my shopping basket there's no collaboration there a lot of collaboration isn't the kind of thing that's immediately visible at the top level of your domain but if you have an if statement that means that the state can be set by some other actor that means you have other actors that can and will be operating on the same data at the same time as other actors are and in that case you need to start thinking much more seriously about the command processing now I have a short saying about how to apply CQRS into these collaborative domains try to keep things as simple and as distilled as possible command's don't fail if you're in a collaborative domain you need to figure out a way to design your solution so that almost never will a command fail if the command has made its way to the server and it's gone through all of the necessary preliminary validation if it's hitting your domain the command should be successful so all of the statements of but what if the items not for sale anymore as well that could happen 20 milliseconds after your command was successful so you need to handle it handling it does not mean how do I prevent the command from succeeding handling it is saying given the fact that I need to be prepared to lose the race condition every single time how can I still fulfill the overall business objective that was originally implemented with that if and usually that means going beyond the boundaries of just that command processing and looking more holistically at what's happening in the domain in many ways what comes out of this thing commands don't fail means you're getting rid of the if statement it means that you're going to have to figure out other ways to adapt the behavior of the business in order to do what they have statement trying to do in the beginning so sometimes it can feel a little bit weird to say well if I'm not going to be checking if the item is for sale then what am I going to be doing in its place and when so the result of saying commands don't fail means all right we're not going to be putting the if statement in there that means we need to start finding which other use cases need to change I'm going to say that again making commands not fail in one use case or one page or one command often requires us to make changes to other commands or to other queries in our domain because all of those things are linked together as far as the end user can see so a lot of times when you start going down this path it can feel somewhat I don't know uncomfortable like you know I I don't think the human body was meant to do that and sometimes it can kind of feel that way when you're having these conversations with domain experts they're like couldn't you just put another if statement in there like well okay so I get it you don't want to kind of twist your business around alright so we'll check if the items not for sale when you add it to the cart but because you can lose that race what we'll do is when the user clicks the button proceed to checkout then we'll check it again right but you know there's another race that happens when the user goes to click the proceed to checkout button because at exactly that same time the other user could be marking the item is not for sale all right well we'll go to the final step where the user clicks place your order right sort of the final final step and in there we'll go through the shopping basket and we'll check our olives items still for sale and then we can still lose the race again because 20 milliseconds after that items will still not will be marked as not for sale so it doesn't really matter you can take that same if statement and you're going sort of button by button by button saying I'll check it here and if that doesn't work I'll check it over then if that doesn't work we'll check it somewhere else all so that I can sort of keep doing things the traditional way but it doesn't really work it's not bulletproof it's the kind of thing that I want you to understand it's not an invariant because of this collaborative domain so the very act of us saying I'm going to try to invoke the same rule at multiple stages of the process should already start giving you an inkling that the way that we're going to have to address this is through some kind of long-running process so this whole hey we need to make the business a little bit more relaxed and limber and able to contort itself in all sorts of weird and wonderful ways that leads us to eventual consistency and I guess has all of you heard that have heard about secured us have heard about eventual consistency but the issue at hand here is not the eventual consistency of the reed model it's not the fact that we're publishing events from these commands and then we have an event handler which is doing some sort of denormalization and putting that into a reed model that's technical eventual consistency the problem that we have here is the business oriented eventual consistency it's the how do we handle the situation of people buying stuff that's not for sale knowing that okay we can't enforce that in the millisecond so we need to do something a little bit broader and longer-term so here's an idea given that the system needs to be prepared to lose these race conditions every single time maybe we don't need to check so much but we can leverage some other maybe let's not call it a feature you as an end-user that's buying stuff on Amazon you may not view this as a feature it's got a behavior of the system that shopping carts timeout people stop their purchasing session so if somebody puts something in their shopping basket that we just saw on the search results screen is for sale then we can say look as long as your shopping cart is active you can continue on and place your order and all that far but if you don't complete your order if your shopping basket times out now what we've done is we're no longer in a collaborative domain anymore that user user number one who is adding item to their cart is now gone collaboration over and now we have the ability to do something with all of those shopping baskets that are in an inactive state so you might have noticed if for those who have used Amazon for those you that haven't get with the program already that sometimes if you have items left in your shopping basket and you come back to the website a day later a week later a month later it'll have this message on there the important messages about items in your cart who's seen that message before in the system yeah okay but your hands are gone up but is it having your probably the kinds of people that use the wishlist feature properly right the rest of us can't be bothered the shopping basket is our wish list and then there's that nice feature it's safe for later right so important messages about your shopping basket usually it's telling us telling us the price of the item has changed but also it's a great mechanism for us to in essence tell the user we fiddled with your shopping basket while you weren't here just to let you know so if an item was not for sale then we could remove it from their shopping basket once the shopping basket became idle so I hope you're starting to notice this type of long-running process that's beginning to manifest itself here there's the first trigger where the user is saying I'd liked for us to stop selling this item and then there's this sort of gradual percolation of shopping carts first of all all the ones that are currently idle we can immediately eject the items from those but every other shopping basket that becomes idle we'd want to verify is that item in there and then remove it so in essence this item is not for sale is not just a single action where we go into a database and set a boolean flag to true and that's it and it's not sufficient of what we did before of verifying add to cart or proceed to checkout but rather this process happens behind the scenes in many cases when the users aren't there when they're not really looking and it's this eventually consistent state that eventually all of the shopping baskets either they buy that item and then the item is shipped and we're done or the shopping baskets become inactive and then the item gets ejected in essence the end result of this is that what started off as a very simple if statement in one part of our domain is turned into a longer running process and I'd like you to start thinking about all the other if statements that you have in your domain logic whether it's on a boolean state like an is for sale is not for sale whether it's an enumerated value we're saying is it in state number one state number two state number three whether it's a number we're saying is this number greater than X or less than Y or things like that every single piece of data in your domain logic that you are using as a part of an if statement I want you to ask yourself what other actor in the system could have invoked something to change this state and once you are able to identify that then you've uncovered some of the elements of collaboration that were hiding in your domain and once you're there once you have those if statements once you've identified those commands once you can see the collaboration happening then start looking for long-running processes and it's kind of dangerous because once you start looking you start finding more and more of them and then you're at risk to kind of move into the oh my god they're long learning processes everywhere like you see an if-statement oh my god another long-running process the concern is that in many cases that we actually go too far and say while everything is going to be a long-running process and it becomes yet another silver bullet alright so what I want you to be careful in going down this path right now most of the DDD community is way on one side of the pendulum of thinking if statements and invariants in my domain model are good and normal and proper and that's just fine on the other side of the spectrum is the domain model is all long-running processes and all of the if statements are happening in that context and there's no longer any of that sort of static point in time state everything gets modeled as long-running processes and the the best situation is somewhere in the middle as any consultant will tell you it depends right any consultants in the room yes our favorite thing so is it so should I do X or Y I mean kind of sit there and the more we get paid the longer we can draw out the right you're not paid very much like it depends if you've been doing this for a while you can come on mmm and people will wait right it's crazy it's fun I got to tell you I've been doing been doing this if you haven't consultant you got to try it just to kind of see how far you can go before people kind of it depends right maybe well kind of draw them out at the end oh you think no actually in this case it is abundantly clear that we should have used a long running screw with their minds a little bit keep them guessing but the reality is and I've been doing this domain modeling thing for quite some time that when you start looking beyond the if statements once you start analyzing and asking these questions and you will need a real domain expert it's not the kind of business analysts that parrots the things that they heard from the real domain experts not good enough you need to have someone who can actually make decisions and speak knowledgeably about the domain just a small note sometimes people think oh you can act and get access to the real domain expert to get answers to these questions you know that's some vice-president in charge of whatever in the company I can't get access to their time sometimes the best domain expert I found them company is the mainframe programmer that they don't let out of the basement if there's a meeting going on and you'll see you know all of these kind of really highly paid important people arguing with each other about you know the business does this and the business shouldn't do that and you bring the mainframe programmer in the room kind of listen to all that is but it doesn't work like any of you said this is what it actually here I'll show you the cobalt that doesn't like no it's okay I don't need to see the COBOL I trust you sometimes that's your domain expert it's that person that has the entire history of the company in their head how it worked how it changed all that kind of stuff you know in the time that they were there the organization changed three CEOs fifteen senior vice presidents and I don't know 100 directors they have the organizational knowledge so sometimes when you're looking for who can I ask about this stuff sometimes it's that person in IT that is kind of overlooked by everybody else because they're still writing COBOL poor soul not like the rest of us that are doing node or go or Haskell or dotnet core or whatever the new hip technology that everybody wants to talk about look for the answers there you'll find a lot of them and as you uncover more and more of these long-running processes the trick or the challenge is not to make them too big now this is something I don't have the time to get into in the in the course of this talk but I do spend a great deal of my time talking about service boundaries and bounded contexts and things of that nature that is the kind of thing that you should keep an eye out that when you're building these long-running processes that you don't build them to run marathons all right you don't need the kind of long-running process that starts with well a user signed up in my system and then they clicked an item and then they added something to their shopping basket and then they bought something and then we shipped it and then they cancelled their order if you have too many and ends in your business process you're going too far too long you need to slice that up into different founded contexts so when looking to model your long-running processes understand that's going to happen within a specific sub domain of the problem within a specific bounded context and getting those boundaries right is really hard but it is super important so for those of you that want to learn more about first of all how to build these types of long-running processes yes I am somewhat biased towards a specific implementation technology part of the reason why I created a stack like in-service bus was that I ended up having to build long-running processes over and over again and I found out that there's a whole lot of repetitive grunt work in terms of handling concurrency and handling retries and handling all of the types of scenarios of what happens if you have two instances of the same long-running process that are trying to start at the same time how do you get them sort of merge back in together these were the types of technical problems that I didn't want to have to deal with every single time I was doing domain analysis so I went and built something like this so in an service bus there the feature is called sagas I think is a much shorter name than long-running process every single time to say that this is a pretty good job of illustrating what the idea is for those who are unmet you know go to town play with it see what you think for those you that aren't on net well now you have a reason to come back so if you want to learn more about how to design these type of long-running domain models how to take an if statement and extract out something that looks like a long-running process and to see how that can be event-driven we have a bunch of videos available online just need to go to this URL go to particular net slash DD deu and you'll be able to get some videos where I talk more and give more examples also from this retail domain that hopefully a lot of you are already familiar with I'm showing how something starts as a very simple if statement and how that turns into a multi message sometimes a multi actor type of correlated business process so if you want to hear more about that kind of stuff take a look at that for those of you that are on the dotnet stack you'll see how it gets implemented with sagas and in-service bus and with that thank you all very much and have a great rest of your conference [Applause]

Channel: Domain-Driven Design Europe

Views: 28,778

Rating: 4.8037734 out of 5

Keywords: ddd, dddesign, domain-driven design, cqrs, event sourcing

Id: fWU8ZK0Dmxs

Length: 46min 58sec (2818 seconds)

Published: Fri Jun 02 2017