On .NET Live - Commands, Queries, and Clean Architecture

[Music] [Music] [Music] do [Music] [Applause] [Music] [Music] and we're live hi everyone welcome to another on.net live my name is tessa phillip and today we're going to be learning about implementing clean architecture inside of our.net applications with my friend here ian cooper so ian before we get started why don't you go ahead and introduce yourself to everyone and let folks know who you are and what do you do hey i'm ian keeper i'm based out of london in uk uh where i am uh which i'm talking about she's a principal engineer they just take jeffy takeaway and i focus a lot on uh kind of messaging uh event space um mentioning earlier just eat in the uk is a is a.net shop um and uh i work on an open source project called brighter which we can talk about later tonight which um forms some of the role that says something like a media to perform is basically uh but also actually acts to work with messaging as well and the other thing i guess i'm known for is um talking about test driven development but that's a whole different story nice and ian you mentioned that you're calling in from the uk um i'm in florida and i know we have tons of folks inside of the chat right now that are coming in from all over the world so i want to make sure that we give everyone a moment really quickly um as i'm looking over i see we have some folks from south africa some folks from norway um turkey the ukraine um let's see like we have some folks from south america from india here's someone from colombia so definitely shout out to everyone that's here really appreciate you all joining us again for another on.net live here we go here he goes now here comes the the floodgates there's algeria there's there's all kind of folks here brazil shout out to brazil again thank you for everyone that's here joining us again we always want to make sure that we take a moment to just acknowledge all the folks and countries and everyone that's interested in learning about.net now before we jump i'm sorry folks in the ukraine are up late then yeah that that's what i'm thinking about we have someone here from plantation florida that's that's that's interesting um who else do we got yeah we got folks from all over the place right this is this is amazing someone here from jordan um so again always definitely always appreciate folks whenever they they join the show now before we really start diving into clean architecture as always i want to share some links and share some of the announcements and highlights of things that have been going on so i'm going to share my screen really quickly and here we go so first thing i want to talk about i i know this has been like a big topic on social media lately everyone has been talking about it and sharing little clips and videos and stuff like that but github announced this thing called copilot which seems to be like like so it's not autocomplete but it is kind of autocomplete it's not you know it's not um intellisense but it seems like it's it's literally like ai peer programming with you which is really really cool and interesting so if you head over to copilot.github.com now you can sign up for the preview i think some folks have already gotten in i haven't gotten in yet i'm still trying to get in um but again this definitely looks like something pretty cool ian have you have you gotten into the preview no i i i signed up but i haven't yet actually i'm not actually in but uh i i joined everybody else so i wouldn't be left out um you mentioned in codespaces earlier i just haven't got into that so i'm loading on the list of whatever um handouts come yeah i noticed every time github announces something like the floodgates for getting into it like code spaces for this for github actions like people just like oh let me let me you know let me dive into it so i'm hoping i get into this soon uh maybe at some point in time we can do a show to talk about like some of the technology behind co-pilot if folks would be interested in that type of stuff all right now moving on so next blog post that i want to share with folks this one is from redbox so red box is a company if you're unfamiliar they do movie rentals but like you know actual physical disk rentals still um but you know what i didn't know like redbox actually uses.net and they wrote a really interesting blog post about how they're moving over to graphql using the hot chocolate project that our friend michael works on and so if you're interested in learning about like their story behind why they adopted graphql like some of the tools they're using in addition to um hot chocolate and some of these types of things um definitely check out this blog post i'll make sure i share the link inside of the chat too um so folks could take a look at it um again i i know a little bit about graphql i haven't deployed anything to production with it but i don't it feels like graphql is like again all the rage now for folks just like we had like a bunch of folks talking about rest um ian have you have you wrote and written anything in graphql or tried it out before so we played with it a bit um we we haven't a lot of our services are still um classic api json appropriate json services and we do do composition quite often and we were wondering whether we might start trying to use it to provide effectively the kind of you know bff layer um by composing various um classic open api style micro service interfaces but that's still we're still kind of on the edges of figuring out um whether the tech's kind of mature enough to do that yet but some people are definitely playing in that space um yeah yeah awesome and then again if there's any folks in the chat that if you're using graphql as well again let us know um if you're interested in us having some more coverage about that on the show and lastly that i'm going to share before we start to dive into our topic today um dot net conf focus event so dona com has a few events throughout the year um in addition to like the main conference that usually happens in november and for this particular focus event is going to be happening this month july 29th and it's going to be focused on f sharp now like this was just recently announced like all of the speakers haven't been announced yet and all the sessions haven't been announced yet but you could definitely come here you could save the date and you could see some of the folks that you know you may or may not see like show up on the on that day so if you're interested in learning about f sharp seeing what people build with it and you know just interesting use cases and projects definitely make sure you head over to focus.net.net wow that's that's an interesting url and um you know make sure you check out the focus event that's going to be on july 29th 2020 2021. all right now that being said my screen is going to come off and and this is your show now so i'm going to have you dive into a little bit about this thing i keep hearing about which is called clean architecture like i've been wondering like what is it exactly like what what are the components of clean architecture what do i need to do to implement it so i'm hoping you could help us a little bit with like answering some of these questions cool all right so let me share oh yeah and then whenever you're ready i'll go ahead and put up your screen you just get the and then we can start talking rather than number one otherwise we'll be staring into infinity all right does that look good yep that looks good all right let's get to it all right so what i'll do we'll do is is i will get to clean architecture but um i find it's actually quite useful to start somewhere else and that's to kind of assemble understand the building blocks that build up towards where clean architecture is so i want to talk about modules um so forgive me if you remember all of your you know computer science you've done in the past and you know modules backwards but let me just kind of level the playing field for everybody and also i want to talk about it a bit because sometimes i feel in kind of the in dot net land and it's only twitter sphere some people are a bit down on modules and layers i wanted to kind of like just about what they really are so really this is this diagram represents my kind of concept of generically what a module is right so it's something where we say we have some kind of facade which provides the functionality of the module and it's generally sort of coarse-grained and then we have the implementation details which provide the actual how the module works and what we're trying to do with a module is say rather than you having to know anything about the implementation details you just have to know about the facade and that principle is known as implementation hiding and we can think about the fact that lots of things do how can i have this characteristic of being a module right a class has the characteristics of being a module right you hide away the implementation details we have a public kind of set of uh members and uh methods that effectively are narrower than the overall implementation details of the class we have we have value implementation hiding is kind of central to what object orientation is doing so objects in a way are kind of modules generally we that when we talk about modules uh as a design idea we tend to be thinking about something slightly bigger normally than just just a module suggest a class so the idea is that my module depends on another module and what happens is my details my implementation details depend on another module's facade it's basically the abstraction over the details it contains and other modules might depend on my facade right so you get this model of details depending on the facade actually and the facade itself depends on its own implementation details right so we can say a module's detail should depend upon another module's abstraction or facade but the abstraction of a module should not depend on another module's details only its own details and we can slice the system up into a set of modules where each module represents some part of our functionality such that the people working on that individual module can say to everybody else hey you only need to really understand my facade provided you can understand that you don't need to know the details you can kind of focus on your module you can focus on the things that you care about doing and when you need to use the stuff i provide you can just focus on the facade if you think about the way we use say for example the the base class library.net framework we're essentially doing this right we're using all those classes we don't necessarily i need to understand the implementation details sure we can kind of trace through nowadays in a debugger and kind of see what's going on but actually we don't need to to do that right this is just really saying we could actually do that at a slightly higher level a level of a module where we could have say objects that provide implementation details and objects that provide the facades we deal with a narrow number of objects and their internal details and indeed you know if you if you drill far enough into an object sometimes in the in the base class library you'll find objects inside that essentially are not ones you deal with right so that object outside actually has some some inside that you that you don't deal with um so as you're talking about that in my head i just think about like dependency graphs right like my my application or my my code or my sdk or whatever the cases is dependent upon something that's on the outside but like you're saying what i i should be dependent on the abstraction versus like the actual fine grains implementation of that right because you don't want to know about the details you just let somebody else worry about the details you just use the functionality that is exposed with the facade that the details effectively help implement right um and so the idea is this gives us what we call good high cohesion and low coupling and that's the thing we want right so cohesion is the property that everything inside a module tends to have a similar purpose and we often quite often say tends to need to change together for the same set of reasons okay and coupling is the property that one a change to one module results in a change to another module what we really want is low coupling in other words a lot of time when i change my module i'm changing implementation details the facade remains remains constant and nobody else has to make any changes to their code right so that's low coupling and it tends to have become from high cohesion because all those implementations just interchange that particular feature or part of my code base is contained inside that single module and generally speaking what you don't want to do is think about modules in terms necessary just of the flow of the system but you want to think about modules more in terms of errors of responsibility because then they have high cohesion okay we can also think about high level modules and low level modules right so low level modules don't depend on anything else and high level modules depend on other things and we tend to find that um the more reusable kind of uh things that are basically um uh the key to our system tend to be things people tend to want them to have few dependencies and the things that are at the top tend to be the bits that are kind of gluing it all together so generally when this model you might think well actually something like my domain model i'd like to put down here with having no dependencies i'd like to basically be something that effectively then i can use from the glue bits like controllers etcetera my code the other interesting thing about modules is if they've just got a facade and some details in theory i can replace one module with another one that has exactly the same facade or a different set of implementation details that may help us basically um uh with uh making change right and sometimes we may find we we want different paths through our code so i activate a different module so typical examples are kind of large order paths small order piles here for kind of room booking modules you could have group bookings and you could have individual bookings it may have a similar facade to the system but the logic implemented is quite different okay and change grade substitution is is pretty similar right the other thing of course with modules is this classic thing of testing in theory effectively i can swap a module for something that can stand in for during testing and that is really how this notion of unit integration and system tests comes about it's the idea of the unit as a module effectively you get rid of its dependencies and essentially you just test it in isolation but note that a module here is not necessarily not to be a class in that class anymore got it and so really quickly we got one question coming in from youtube and uh this person is saying uh can we say that an aggregate and he has ddd for domain driven design are types of uh modules can we could we look at them from that perspective you could yeah i mean aggregate's a good candidate to be something that you might want to choose to implement as a module it has a route especially as a kind of you know thing that you effectively would perform some operations on the top and then it has um a tree underneath where effectively you don't really want to access that so directly so your repository should tend to bring back an aggregate route if you have multiple things in a tree below you just want to deal with you want to lock and deal with the route the only thing would be a little bit is that aggregates uh are the the the the concept behind them is a lot more about locking right i can just lock this root object i don't have to lock everything and and the idea is that reduces the number of locks that i have to apply and say for example a relational database system which has scales better that's really what a lot of what the kind of aggregate idea is about postage to the entity coming out of my uh store i have an aggregate because i can i lock this one route i lock the whole whole trick when we think about it also kind of it kind of reminds me of like the solid principles that you know i know i remember before everyone was talking about these solid principles and there's the the d in solid like the event dependency inversion principle where pretty much just says like you have to depend on abstractions versus like the concrete implementation of that abstraction yeah we're going to come to dependency inversion again in a little bit that's going to come up in just a second so we're going to come and explain exactly how that relates to the modules um but what on modbusion.net right so there is i mean i've seen some people like jubilee that don't argue that hey the objects are modular.net but actually we do have a first-class uh structure which we can use for modules now interestingly enough.net confuses the issue slightly there's actually a technical module in in.net which is just supposed to compile file which you can kind of link together but we actually need a slightly from the computer science console module one something about a bit higher level where we can say hey i want to hide all these details and expose this facade and that is really just an assembly right you think about an assembly what an assembly does for you is it says hey anything that i declare as public can be seen outside the assembly anything that i declare as internal things inside the assembly can see each other anything that's private is hidden from other things even within the side of the the assembly so at a very simple starting place you take an assembly and say what do i need to expose from this assembly to other assemblies in order to use the functionality it contains that's public what do i need to share as part with as part of cooperating objects that implement this assembly they're internal and what do those little objects themselves need to hide from other things so they want if they want implementation hiding within side those items all those things can be private so so assemblies are really of this first class citizen.net for actually using modules so one of the best ways to use assemblies is to think about hey i've got a set of things that change together because they all share some kind of responsibility or things that i can reason about having a similar role right um you know what i can make them an assembly i could give you the public interface that gets you to the um parts that i need you to basically interact with to use it and i can hide all implementation details and that means if i then want to go and change its implementation details inside that particular assembly no one effectively using the module has to worry unless i change that public interface right and that gets us to the next concept which is the layers all right so one of the problems you could get is let's say you have a module a right and let's say you change its facade and let's say that a module c has implementation details that taken dependency on a by depending on its facade that means that if i change that facade on a as well as rebuilding a to basically build that functionality and i'm going to have to rebuild c because c has a dependency on that changed interface now imagine that effectively that change that i made to a that changed the implementation details to c for some reason that bubbles up and it hits basically the interface of c so c is now going to change its interface in response to this change but imagine that b has implementation details that depend on c's interface right then i've got to basically well build a and c and then b right but if i then was defined that a he had implementation details depending on b's interface i've conformed the complete cycle in that this change to a's implementation details ripples forces c to rebuild which changes interface forces b to rebuild which in turn would force a to rebuild but that's what i'm trying to rebuild to get mine changed in the first place and cycle dependencies stop you building right um people don't build these these these problems quite so much anymore but i can i mean years ago when i was a c plus plus developer and we used to have the you know battle world of the daily build it would take four hours to run in c plus plus we would regularly encounter this problem of people who created a psychic dependency that we had to then basically fish out and break so there's a way to stop you getting psychic dependencies right and that is layers and so one of the key things that layers will do for you is stop you getting psychic dependencies and that's because a liar says you can only depend in a given direction so here this is a classic three-layer structure right well we'll talk about the three layers because some people basically are down on it and we'll talk about why they are and why why maybe that's misunderstanding but here's a classic three-letter structure i've got a ui layer right some kind of presentation layer if this was an api a http api that'll be my http api there's some kind of presentation like i've got domain layer right that's where effectively i've got the rules effectively that my business cares about and then classy below got an infrastructure layer which is probably talking to the database but might also be things like talking to say a payment provider and classically this idea is that i can i depend downwards so the things in my presentation layer a and d they can depend on the domain layer and b but b as you can see by that kind of like stop sign can't depend upwards on the presentation layer right and a can also depend on what's called a relaxed model it can skip the intervening domain layer and go straight to infrastructure which is very useful that's what kind of start thinking about cqrs or that one of the things it's doing is skipping the layer in between um strict models don't let you do that models you have to pass all the way through the individual layers we'll talk about why that that is a bit later one of the principal ideas behind the layers though is just breaking that psychic dependency chain right i can't get circuit dependencies now because i don't let you depend upwards right simple rule stops it happening the other thing that's useful about it though is it helps me reason about the responsibilities that those modules have i've got modules as responsibilities presentation i've got modulation responsibilities the rules basically my application and i've got modules which tend to be doing some kind of i o with the outside world for the lord where i want to get stuff from in response to the requests the presentation layer and that can actually be helpful i know where to look for stuff and i know i can reason about hey that these things are appropriate different layers one of the reasons people are a little bit down on this classic presentation domain infrastructure model though is uh people assume that is all that layering offers you and really i could have any layers i wanted right as long as basically i have this kind of like downward dependency model sometimes referred to as the layer cake model because you know the the the layers stand on each other right then i won't get a cyclic dependency i can have whatever layers i want and in fact you know if you look at some like domain driven design one of the things it encourages is you to have modules in your domain layer which represent portions of your domain right um so so don't get fixated on the idea that you have to have these three layers just recognize that layering is a way of stopping circuit dependencies and modules and also giving you a way to navigate your modules by saying hey things that have this kind of role are all going to live at this kind of layer so when i think about layers one of the first things that comes to mind is you know how i used to build applications a long time ago there was always like a database layer and then there's a business logic layer you know whatever whatever your organization thinks it makes sense for layering and then one of the things that i've seen come up more recently conversations is the onion architecture where you know instead of looking at you know vertical layers we're looking at concentric circles of of things right like there's a core layer in the middle and then yeah you know you're like the things that depend on it like now encompass it in the outside um you know in the outside circle sort of yeah so we're going to come to that because that is a form of clean architecture so onion is classed as a form of an architecture clean architecture is just a way of saying hey a whole lot of people independently realized that there's a similar model of porsche adapters onion an old one called boundaries controllers entities and they're just that their former layering right so we're definitely about to come to that so the first thing i want to cover off first though is dependency inversion because it's important to understand where that comes from so if i've got this layers model right what happens if in my domain model i want actually to get access to something that basically is a type which is basically belongs to say d in the upper layer so imagine for example that there was some useful information on the uh http post and i wanted to create a type to encapsulate that in d um and i wanted to pass down that kind of information into my domain model but then my domain model if it then uses that type has a dependency on something that belongs to d right uh potentially an implementation detail as well we say we can't do that so how can i pass something then down from d as a kind of you know into into b and b can take a dependency on that information that seems problematic right so the odd this is where dependency inversion comes in right the dependency inversion just says b declares at its layer an interface that says hey i need to take a dependency on something that has this behavior right so whatever the data is effectively except for the the d is going to put into that type hey i need to basically express an interface that describes what i can get from that right so my i suppose my requirements as an interface right i say i need these things and then i simply ask d whoops give me an implementation of that so d creates a concrete object that implements that interface it can then pass them into me and a really simple level i don't need an ioc container to do this i just new up something and pass it to the object below but because it implements that interface right the dependency of the b has is at the interface defined at its own level d is allowed to depend on things below it so it can say i implement that interface and a or d whatever can create a simple a an instance that d and pass it down d can do it itself right um but i can pass an instance of that item down the stack to be that's all dependency inversion is and that's what it was originally built for any idea in a layered architecture you can't create basically this thing that lies above you something has to get passed into you so you describe your interface which is the requirements you need and something in the other layer can create a concrete instance of it and pass it in and you just get the interface you don't really care what the concrete type is it's been preventing it you just call the methods that you need the requirements of okay you know one of the things that has come up with me and questions that folks have asked me in the past is where do those interfaces live like where do i define the interfaces um and you know i know some folks believe well the interfaces should be defined in the layer that they come from but then some folks i know are like well the interface should be in some interfaces assembly like something that is not within earlier and then implement there as well and so now i know there's a little bit of a conversation that's always happening regarding well should i have a whole project just for interfaces and then implement them in concrete assemblies or whatever the cases so i don't know if you have any opinions yeah so i would say define the interface with the module that essentially says hey i need to get one of these right and if that interface if you have a number of those where essentially shared right a number of modules at that layer all need some service from above and you want to share a common interface amongst them yeah it's okay to define basically a assembly at that layer that says hey these interfaces this layer tends to use from above um sometimes that's a little bit of a smell i i find that basically multiple things in that layer are too commonly using things from above it's kind of like why do they all need that is it is there are they genuinely separate modules um but yeah i would i would start with the module that needs it defines the interface at that layer if multiple things do need it you could consider sharing them across the modules of that layer okay okay but yeah um interfaces are belong to the consumer and then the consumers abstraction are saying give me something that satisfies the contract that i'm going to describe to you okay right okay let's talk about domain models a bit right so inside that kind of domain layer there tend to be two types of business logic we think we think about one basically says hey this thing is solely focused on a thing we need to do in the domain generally some kind of rule often one of the things to think about here is i have a rule that requires some facts so some state and it operates on those facts and it gives us some kind of result when we exercise that rule against those facts right and that's our classic uh hey i've got an object it's got some state and it has some methods on it because those methods basically work on the encapsulated state right um you know great that nowadays we can and i hope you will consider that basically hide those implementation data some of those class with things like you have called nowadays don't have to expose it all as public properties leaking your implementation details out right the other is some kind of application logic which tends to represent some kind of workflow that's right so i've got this entity that says i've got some states and facts and i've got some rules that basically work with those facts but actually what i need to do did you get anything something useful done i actually need to check two or three rules uh you know and that effectively bits of logic and apply them together and that will basically represent something useful or the user wants to do right and that logic we tend to think was application workflow logic it it doesn't necessarily always um work against state and rules of state it just coordinates the activity of those things so if i have a number of classes in the above group which represent we can go as entities that's basically state and the behavior that's coupled to that state then i might have a class that says hey i know how to orchestrate a number of those entities to get your functionality and typically if we look at the layer diagram we often think of a split happening like this we have in the domain layer a service layer an entity layer well a service module so i mean whether it's a layer or not is a bit of a tricky question but i mean people sometimes think of it as a split layer with a service layer and entity layer you have a modules for the service you have modules for the entities and the service is how you basically call and use the entities entities of course don't talk back to the service right so the entity layer i've got these classes that effectively have state shapes headed states and above a service layer where i do some orchestration and kind of workflow logic and once upon a time it was not uncommon in.net land to represent these as say a class in b there was all static static methods on it that essentially orchestrated your actual objects that had both state and behavior yeah i've seen some different implementations of that i've seen i've seen like you said like different projects i'll also see different solution folders with multiple projects representing like those different layers as well yeah okay so there are a couple of different sort of implementation techniques one is the logical lives in my entities and i have a very thin layer basically effectively just coordinates those items right and generally basically the idea is your facade over those in the service layer should do the minimum it just coordinates then it's all the real behavior of your domain model should live with inside those entities and that's what probably you and i classically think of as a domain model right the other one basically is what's called an operation script and that's where you say well actually i don't really have many rules associated with the state it's really i store some state i get some state back out it's a very very crud-based application and there in some cases it may actually be better to have your service have the logic of you know saying hey you know what i've just got some sort of uh data structure which basically has some public fields inside it i get i i save those i get them out right and i've got it in the service flower i've got the script that does most of that work so it's a question of whether you want a thin facade and a rich domain layer because a rich to main model or effectively just some objects that effectively are very thin representing state and a richer application workflow because really your application's much more about application workflow based stuff neither one of those is right or wrong it just depends on what you're actually dealing with um in terms of your application right um and it's actually you know though some people don't i'm not fans of it it's actually okay within a well-stretched application that has good modular structure to say hey over here i've got basically kind of operation strip style modules dealing with some data structures you know that basically are very the the state and no behavior and over here i've got a module that's actually a rich um domain model with a thin facade dealing with proper entities and that's actually okay uh within some applications provided you clearly have those in separate modules so you're not mixing the two types together which is confusing let's talk about ports and adapters supports adapters is probably one of the most famous the clean architecture styles right and all we'll tell you a couple of poor samson bce so the reason for showing all that stuff beforehand is just to help you understand that really that everything you you need to know you already know now we're talking about modules and letters um so it's a so basically what it has is these layers it has an application layer which is for you and either the main layer um nothing has caused probably more confusion than the fact that alistar copeland decided to call the domain layer the application layer um because he talks about basically port so the application and uh i might think the application actually is the running software right uh he doesn't mean that he means the domain layer but nothing has caused more confusion than that the adapters which essentially do i o in and out basically of our our overall system and ports which are essentially what sits between the adapters and the domain model now this is a kind of uh dynamic representation the reason for it's called the model portion system often called hexagonal architecture is because the typical diagram representation is a hexagon why well because alistar wanted you to think in terms of these flat faces as things getting exposed so in the middle those circles represent entities that represent classes they've got state behavior the kind of hexagon that surrounds them are the ports it represents effectively a pointer with a facade over the domain so think about basically what you're talking about basically some kind of service level being a facade over our domain model right and it's a it's a facade over the domain where effectively we do application workflow like ring a bell um and the adapters then effectively call the ports to exercise the system behaviors and the adapters are stuff like if you look over here on the left hand side we've got stuff like http adapter or a gui adapter so you know it could be um you could be thinking about basically an http api we might be thinking about razer pages feeds i can have an atom feed so it's exposed out um over on the right hand side um the left by the way something's called prime in the right secondary i've got something like a database where effectively i'm reaching out to go and store my data or retrieve data but there are other ports for example if i raise a an event i might do so to a zero or a mq with an mkp adapter right or i might send an email or i might turn out something out by signalr all right so there are other kinds of i o adapters our application can use and they all sit around the outside okay so if i think but this is a layer diagram for what we saw earlier right it's kind of it has the adapter layer which fulfills the functions of both the infrastructure and ui layers that we saw before and other models so all of our io basically goes in that layer so we're saying all the modules in there have some kind of i o responsibility all right and then i do a portion of the application layers right where the port layer is effective we talked about before it's a facade that basically does our kind of workflow and the layer down below that is the application layer is the kind of entity layer that contains our classes basically in state right and you can see again if actually we use the interface model dependency inversion in order to basically take dependencies on things that live outside right so if for example up in my port layer i want to get access to the database how do i do that now in the three layer model we saw before it was not uncommon for your domain to say hey give me some access to the database and i'll go and save myself right here i can't do that because the database layer is not below the domain it's above so i need to pass in through my port into my port something that says well here's access the database and typically this is where ideas like the repository or a dio interface come from because what i'm saying is hey at my port layer what i need is something that i can use to get data out of a data store right because in my workflow i'm going to start that by getting something out of the data store exercising it which will be basically an entity that from my application layer that's believe me i can depend on it exercises this time howard has more than one of them somehow and maybe save the state at the end of it right and to do that i'm going to need some way of getting hold of that data okay i'll define an interface like ida or repository or a dd gateway whatever you want to call it right um folks get fussed about people you know saying hey well look at the repository pattern again right just you can call it entity gateway right and no one has to have to fight you over whatever what you've used for it um which is an interface that lets you put the actual concrete implementation in that layer above okay why is it valuable well i've the main layer layers clean completely of any technology concerns right remember we talked about earlier on about the idea that these low-level modules had no dependencies right well with the our domain layer then it has no dependencies so it's not subject to change around technology concerns um and that's what we can therefore mean that our business investments pretty secure because it's if we decide we're going to change the way we represent that we want to use a different uh you know um we want to switch to a different web framework that's fine the main model doesn't really care that we've done that once switched to a different database the main model doesn't care that we've done that right it's where i know this is a european-american thing i mean i i quite often end up in projects switching databases over the lifetime of an application everyone in america thoughts on twitter never does it i don't know whether there's a whole european-american thing going on here um uh whether you just will rewrite stuff because you've got more money who knows uh i always hate it all the time too i always say why do we need abstraction we never change the database and i've been on tons of projects with the database changes so i don't know yeah i don't get it i don't know where that comes from the other thing actually the changes in much the database though is the technology you use to access the database so you decide to throw away you know and hibernate and take in dapper or whatever right so those change quite a lot and that's the same thing because you that adapt that lives at the adapter layer um so it gives you that ability to change the frameworks you're using to do some of that work as well quite easily and that's that can is often more of a pressure on us as devs to go hey this new hotness is far better than the old thing you're using and lord knows you've been a microsoft developer long enough you have seen a lot of ways of getting data out of the database yeah for sure okay so interestingly enough right uh ivory jakobson uh some of you who are have grey in your beards uh may know who iv acupuncture is some of the youngsters may not so ivy ackerson basically is famous for writing a lot doing lots of the pioneering work but object to orientation and particularly the idea called a use case a use case for those who have not seen it before is a kind of a heavyweight version of the user story this is what a use case might look like for ordering basically food in a restaurant so if you look over on the left hand side i come in as a client and there's a use case called basically order food at the top right so i place my order i order the food in that use case a waiter takes that order and he he may also let us order wine right because you know we like to have wine uh and then basically the waiter will then effectively send the water through to the chef who cooks the food and the use case when it's when the food is cooked the waiter can then serve the food and the wine they're there more use cases and then i will then eat the food and drink the wine and pay for the poop and paint the white all these use cases represent effectively things that somebody involved in this process wants to do a kind of little mini workflow i feel like right yeah i remember creating a lot of these back when back when i was still in college and i was like wow like this must be what real folks do in in enterprise and then i started working and no one ever use one of these but for me i would create them from my personal understanding because for me it makes it very clear about the direction which information is flowing and like what the expectations are yeah one of the interesting things is that um if you look at some of the more sophisticated techniques for doing user stories like story mapping et cetera they they lean quite heavily on the kind of history of use cases and indeed aleister coburn who was a big fan of use cases wrote a book i think aggar agile use cases which effectively blur the line between use cases and user stories so it's almost like you can think of uh use cases as kind of the grand parent of user stories that come a bit later and some of the more uh useful techniques around user stories lean on some of the learnings we had from use cases but they're over detailed probably although the average jira ticket looks worse than the use case um okay so he got he wanted to express this idea of the use cases clearly in his architecture so he got this article boundary controller and entity and he said hey when i think about basically my system i'm going to have a boundary and the boundary is going to be an object that interfaces with basically actors so things outside my system like users or databases or payment gateways right some of this stuff should be done it's starting to sound familiar and hey there's going to be entities and they're going to be state and the rules that depend upon that state okay we've talked about that before for sure and the controllers that later change namely change to interactors because controllers is like service or event is one of those words that we can't find enough things to attach meaning to for in software development for some reason um but they mediate between boundaries and entities so the idea was that abound she would never talk to the entry director it would talk to an interactor and the interactor would effectively be a the kind of execution of a command that the boundary had created right and so for jakobson this is where the use case is expressed in the interactors where the workflow happens right some of you may begin to see the pattern we're talking about now so my actor talks to a boundary object which talks to the control object which basically exercises the entities great so if i begin to think about this diagram we drew early it becomes pretty clear that that model is the same right the boundary is our adapter layer it's the i o concern layer the control object is that port layer it's the domain workflow layer and the entity object basically is the application it's where the entities live in my application so the point about clean architecture is there has been if you like consistent re-implementation quite independently we think of this idea um and onion architecture is another one and it appears that people people just keep coming back to the same principle we're working because it keeps the domain really clean of any kind of technology concerns um the interesting thing is that basically jakobson saying controllers are transferring objects that implement use cases right so they're saying hey what you should do in encapsulating their interaction as a use case this like little workflow you could name it after the use case you could have a cook food interactor right everybody think about modules we're thinking about modules as being things that effectively let us reason about the system and encapsulated things that basically had a cohesive kind of role in the system hey and actually that would make some sense we think about those modules around use cases right because hey that's a cohesive kind of thing it's just the same forces of change whenever i want to change how we cook food maybe cook it let me that's the thing i could target and a port important adapters is exactly the same thing for cobin it's a use case boundary remember astor coburn also wrote some stuff about use cases later and he was saying hey one of the reasons i want to do is expose basically use cases clearly in the architecture so both of them had the same idea that the use case which you can think of maybe as the user story right is exposed somewhere in my architecture and it's at this layer this model is effectively the same right and so what that tends to basically also help out with i find is testing so i don't want to much about testing because i do lots of other talks and testing but one of things to bear in mind is that you can actually think about driving your tests against that port layer because that use case is a useful piece of behavior that you want to get under test and then you treat your your tests basically as another adapter that is driving your domain through the mechanism of the port now you can do what we call basically shifting gears and shift the gear and testing practice to actually address the entities directly if you need to but it's often quite useful to think a little bit more about actually testing against that port layer okay and then really when we talk about more about say integration tests we're really focused on that in this model more on that adapter layer testing that adapter layer works right with some system tests on the outside and nowadays i'm pretty sure that you can for example the average kind of open api style http i'm pretty sure there's a good model for bash scripts and jq being perfectly adequate system tests but i'll get shot for that uh okay um honestly i keep thinking i could just write lots of things in in jq and bash scripts okay or powershell uh putting your preference so this is actually what the clean architecture looks like this is the kind of canonical picture of it right i've got entities at the center i've got use cases surrounding them around that i've got like a controllers gateways and presenters around and i've got the actual um physical things that provided their services to me right so the blue layer presents things like um uh a database you know network interface adapters etc the green layer tends to not be stuff you you write tensorflow represent things like sp.net core entity framework right stuff that you're using the use case layer is really where you want to write stuff so you just write a controller um to to plug into effectively asp.net core but you put most of what you're actually your logic into your use case and that drives your entities right so you have this kind of model of skinny controllers with a fat kind of use case model in fact entities inside it with all the behavior and you you don't put all of your logic then into your controller same way effectively you know your your gateway to the database your entity framework core and your db context right actually you just you wrap that up and you just use that from within your use case and we'll show you some a little bit of code towards the end should give you clearer picture of that um so the model is depends inwards right remember with our layers which is to avoid circuit dependencies we're always going to have a dependency in one direction so you depend inwards right and if you want something from above you use dependency inversion so if i want to get hold basically or something of uh say entity framework caused you'd be constantly to wrap that right and you'll see actually the code i've got tonight um has got a mistake in it i'm trying to fix that up although uh i was saying to cecil earlier i managed to break my demo code version so i'm running an older one trying to fix that up for you but i will explain to what the problem is and then in a couple of weeks time i'll let you all know when i've got the fix in for the demo code for this actual talk um so so ian really quickly one thing i've noticed that a lot of folks in the chat are talking about is ddd right like domain driven design and so i mean we're not going to dive into too deep into what domain driven design is in this particular stream but i'm curious is domain driven design something you think that is required or just something that will be helpful when implementing a clean architecture i think it's the other way around i think if i want to implement domain driven design one of the things that helps that i will find useful is to have a domain model that is cleanly separated from the technology and uh a clean architecture helps me do that what lives in my kind of entity layer and a little bit in my interactive layer is the stuff ddd cares about right whereas everything else it doesn't want to really understand so it moves them away so it sends the repository pattern you know what is it what was it originally for in in ddd the idea effectively is that my domain code doesn't want to really know about database interactions so here is this abstraction of a collection or an abstraction of querying the collection in some way and it's going to hide away from you for the fact that that collection is not necessarily a lovely memory thing that you've got nearby but in fact requires you to talk to the database and get stuff out that's what that's what that really meant and i find the clean architecture is a very good way of giving you that domain model that's free of all those concerns and gives you just really this narrow interactive layer that says hey i'll do the business for you of taking in a request from outside maybe talking to the gateway marshalling all the entity pieces and making sure they work and making it happen right there's a little bit of this application work for a lot it has to happen but yeah you can focus on your domain model it helps with testing too but really it's an enabler this model of those kind of domain focused practices i would see tdd and dd is really being domain focused practices right and in fact um really so what was what was eric's war cry with domain drum design eric's concern was that um uh the really smart developers in your organization are spending all their time dealing with tech stuff not with the domain model that can that basically is your business and that's actually where the value lies right and that we've we've somehow shifted the focus of our attention and everything else to hey um what database om am i using and how slick is etcetera library and what does the uh yeah yeah well but the domain is really the heart of it and so what eric's book is really was about more than anything was um getting developers to understand the main models how to re-engage with them etc and indeed actually you know venture design is a funny book in the sense that it's strategically about making it servicing your domain per se it doesn't go into a lot of details beyond ubiquitous language all those kind of things etcetera on good domain modeling and you need to look also things like responsibility driven design from rebecca worst brock effect reception to really grop that whole picture because he was building on that assuming at that point you probably read that book anyway um but yeah i think clean architecture is an enabling strategy to make that easy for things like ddddntd it makes them really easy okay that makes sense so here is kind of a laid out flat model rather than a concentric rings model and you can see the idea here is i've got some kind of delivery mechanism so let's say for example i get an http request so i've got some kind of web server a boundary layer which is where i uh interface with that so i'm writing my controller in say sp.net core um it isn't allowed to touch entities it has to go through interactor so it's gonna receive in a request model so you might think of that as a dto right some kind of data structure that represents say the body of a post request or something it's going to deal with an interactor so it's going to go to basically this layer in between and say hey can you run that use case form you can run that use case to basically order food the interactor's going to go away and say okay i need to actually go and get some the menu out of the system so i'm going to go to the entity gateway and load that up for you and that's going to be one of my entities right and then i get some other entries i might just new up myself back to etcetera and then i'll do the work and then i've done the work i'll give you a response back and again i don't want to give you the response in the shape of an entity because that's something you can't touch it's a different layer so i give you back some kind of data structure which you can then represent right and that's actually quite useful when you have these at the interface layer when you have facades that um deal with primitives and don't deal with basically types that are basically part of your implementation details be very wary sometimes people expose implementation leaders out of their facade by putting them on the facade itself right so the way we think about that is this right a web controller api framework question response model now people talk about mvc sometimes they go is that nvc nvc lives down over here right web controller effectively and its response model all deal with that nvc thing that's nothing to do with this basically interact or an entity right different similar models they're similar shapes right and the idea effectively i'm separating say a model and presentation effect etc but they are designed to fulfill separate separate goals um and down here we're just showing basically the entity gateway is talking potential in some database okay so i'll i'll move through these slides a bit quickly but how do i implement their interactive layers one question then comes up right because that's because this gets into the other parts which is brighter so one way of implementing a use case an interactive layer is to think about the command pattern right so what's the command pattern basically it's an interface that has a methodology called execute you call you basically call its constructor you pass any parameter state from the outside in the constructor you call it execute and it runs the domain the enterprise so the command panel it turns out is a really good model for being that kind of interactive layer and people often know the original pattern talk a lot about the command pattern being a kind of transactional scope right hey i'm going to do some work and at the end of that work there'll be a result right so command patterns are really to work doing that and you can see that model here saying okay i've got some kind of i command um of an uh interface so generally speaking you you you can effectively create concrete instances of a command but i can mine it doesn't execute on it right and my boundary there just exercises that across the entity right i've got a concrete type of exercise right very simple now so i just got a couple of questions like if you could go back to that previous slide like it looks like we had a couple of questions about where things are supposed to be so this first one is um from youtube so our entities just the data structure without logic that's the first one no the entity is a data structure and a logic so the split is entity to contains domain logic it contains state effect in other words of facts that basically make rules that we have about our domain uh operate on so i've got some enter so effectively i might have something the classic example i always use is in an entity might be something like um an insurance policy and the facts are things like you're a smoker you're overweight um and you go skydiving at weekends for kicks right and the rules are how much money am i going to charge you for your life insurance that would live in an entity it's state plus rules that operate on them the workflow is that and you probably wouldn't actually do this workflow in routing insurance but just for an example the workflow is hey somebody gets in this uh policy to begin with and the first thing they do want to do is validate it they want to validate all the fields are there that etc then the second thing they may want to do is um figure out what the possible premiums are for coverage so they run some algorithm that tells them etc so those things are potentially different commands that you could exercise one would be basically validate the submission one would be basically effectively underwrite the policy et cetera right another one would be basically effectively that you know when the customer chooses it would be we select the policy option those things would be commands living application workflow logic against the entities and then last one i'm going to ask you before we move on um also coming in from youtube so our view models are part of the presenter or the interactor all right go back to that we got it so uh boundary layer so basically those from ports that's what all the adapters layer right cont is basically this webcam is bounded as the web controller and the view model is essentially the response model remember we said basically when i go to the when i come in so the parameters to my web controller effectively etcetera the objects i use there that's in my request model i then call the interactor i may pass them in to basically be the parameter values that get set on that command or whatever when it's finished and does its work it gives you back it it updates basically a response model which is effectively your it can be your view model from mvc that makes sense yeah and then that's one that's um should we use rich or anemic entities right depends on your on your model use whatever you're appropriate to the main model so rich model says hey i've got some rules and i have state which drives those rules right that's what i often tend to think about risk domain models is if you've ever seen rules engines and how they work that's what a rich domain model should be like right i write a rule and that rule depends on some variables which are basically state and so what you do with root classic rules engines is i load the facts in other words i load state into the rules engine represent the value of those variables then i run the rules and it spits out a value and rich demon models are very similar right in a sense that it says in this object i encapsulate some rules and i encapsulate some state that effectively is the variables that go into this the decision moments in that rule right so something that's something that prices something effectively etc would have you know prices discounts that kind of etc right as facts and then essentially the rule says what you do is you add up all the items you see the discounts apply you give loyalty bonuses etc effectively and he had the price at the end of it right but if what i'm doing is just saying hey i want to store the birthday of every employee that's an anemic domain model right there's no real rules you're applying to you're just store in the information so you can get in and out and the application workflow logic just says hey you what you want to do is get everyone's birthdays out great there you go so it is literally do you have a domain more any rules that apply to states and you pick the appropriate one and then um so and this one just came in as well so whether to use rich or anemic entities also depends on the programming paradigm whether it's object oriented or functional and that's a question yeah i mean to some extent there is this basically uh if i have a paradigm that's not oh i i'm not necessarily encapsulating my state in the same way for sure um the uh that the functional version of this is looks pretty similar in a sense though that your your interactor really is much more your kind of uh functions basically effectively over your kind of railway syntax down there but yeah that's a bit more complex um someone like scott ashton is basically probably the person to go and look at his stuff because he i think gets this model as well i think he talks about it somewhere so i've got a command object so the other word doing commands which is the kind of pattern you'll see in mediator and brighter and libraries like that says well i could i could separate the parameters from the command from the thing that executes some code using those parameters why am i why do i want to do that when i separate the two things because i can insert effectively middleware and arbitrary things in between the point effectively where you asked where you send the command and the point where you receive it and those middleware things are quite often the thought of concerns like logging um bright usc uses it to run poly for example so that you can have policies around retries and stuff like that basically associated with it um there is also a level which basically you don't have to know as the sender what is the handler that basically receives this and you can in theory have more than one hand to receive it right so actually it's a bro it's a fan out it's a broadcast or in theory you could say well actually at some point i can swap that over for something else um and the control code never needs to know that's what dispatch is for and we'll see that um i see how bright does that a little bit later she's in code does that but the idea is they're very similar i'm going to show you with the test here all right yeah uh we're doing a test driving it here in this case so basically you've got an actual framework a test again request model basically a setup of the test right and i call effectively directly into my um request handler which basically has an interface usually in a concrete implementation which drives my entities and effects returns a response model otherwise the things i'd assert on the test right so that's just showing you a little bit two things already i should really should probably shouldn't show you two things at once but it's showing you basically how that varies around a handler it's also showing you how you can use tests to drive the same model right let me show you some code time and as i say so this this 12-factor tutorial is on my um uh github reference everything else but actually the one you're going to want in a long time is ca tutorial which takes you through it a bit it has a so the way that both of these work is when you go get to the repo you'll find the repo's master branch looks decidedly empty and you'll be like ian's ian's ripped me off man there's nothing here so the branches represent steps through basically the transition um this one has a number of branches um so there's basically app only and app and work we're pretty gonna show you only app and worker tonight but the key one the ca tutorial takes you through um breaking down an app that effectively has no use of layering um to one that has layering to different strategies that start with a very simple kind of like service layer to then go to commands then to dispatch them up so you can see the different variations um uh and when you pick it up i'll probably improve the repeat to explain how to use it okay and so before you move forward too much so could you bump up the font size it's oh absolutely really small so what may happen is if i switch this over to presentation mode it doesn't it's doing nothing why are you doing nothing ryder you've let me down in front of all these people that is a lot better isn't it yeah that's much better yeah i have to remember to do basically is bring back the tool window on that side okay so i can now that looks perfect um okay so this is running actually in the background i'm running some docker with basically uh my sequel in it it's the world's most over engineered hello world application so the idea is basically i can so this is basically writer has the same thing that code has if you use rest client inside it which is you can create a file with some simple http statements kind of inside it like curl and you can actually just execute them interactively so it's great for writing those kind of like end-to-end level tests to go with your app but rest client in in code does the same thing um and here what we can see is i can see have i got any greetings i think i've got one i prepared earlier which will pop up in a second so you'll see the windows which hopefully i can just change the layout slightly so you can see that so it's going to give me a response back it's going to say to me oh 200 okay oh yeah i've got already put one in hello mary right so can i add another one i can add hello world so just post the greeting in come back why you know come back okay i mean hello world have gone in i don't know why it decided to there we go hello world and i can then do a get i should have a few in there now right i'm stacking up greetings um obviously it's completely useless and i'm not going to make my fortune from it but it's a fairly simple kind of app which doesn't do too much now because of that there isn't actually a rich domain model here um but what it's going to give you is an idea of how we might structure things so if we're going to in the greetings app right so greetings app has some adapters inside it which basically just me identifying for myself basically what kind of layer that is um and it has a greetings controller and i'm sure you all know how controllers work but what you can see in here is we've got a couple of things going on right this query processor seems to be doing some things from us and we seem to have this command process of doing some things so we'll break that down and i'll explain to you what these two are we'll start with the command processor which is brighter and then we'll go to the query processor which is called darker no naming obviously you know i i really end this of hot chocolate as a name i'm like why can't i come with better names like that okay right now um okay so what the command processor does is just that dispatcher model we saw earlier so here there are parameters we're saying i want to add a new greeting right it's going to have a message so this effectively coming in here is my request object i'm going to turn that into basically parameters into my add greeting command then i'm going to say to the command processor um send this to someone to deal with so what the way brighter works we have two things send and publish send says one person gets it published says hey found this out a lot of people want this right and you'll see it as a query after to go and get some state so i'll come back to that in a second so what happens when i do this well brighter will search for a registered handler for this now i'll withdraw registration a bit later but what happens is in this model over here in my ports kind of layer um what i've got are a set of handlers and so and greeting command will go to this aggregating command handler now the keyboard you care about for the minute is you have to kind of basically write this right which is you have to write you have to implement the handle method and then says here's a command right and because async is a cancellation token and in there what we're just doing is we're saying okay we'll get a hold of a entity framework context um sit in a repository uh and then we're gonna add basically effectively uh our grouping into database and the thing that's i'm fixing is wrong here is really this should be a unit of work because it means we've got a dependency in here on the layer above us which basically is the entity framework uh db context so what you'll see when i'm gonna get the ca tutorial version up is replace that something called unit work um because we we just want to we just want the behavior to express the behaviors that we're getting out of context right um search the greeting card it's going to implement that for us okay uh you'll see a command processor post here we'll come back to that later the other thing that's more interesting perhaps is these attributes that we put on it what are they for so remember we said we can have middleware in a dispatcher between the point you send and the point where you actually do your work like the data's the data sync in your pipeline so what we're doing here is declaring some of the things we want to happen first so the steps represent the sequence in which they will happen it's a russian dole model so in other words each one encapsulates the next so this one here is the one that occurs immediately before your handler what happens is we're saying we're going to use a policy and the policy is retry policy async what that means is we provide these policy acing as a you can write these yourself these basically middleware components and attributes that are with them we provide a few out of the box for you and one of them is basically to use poly so this is saying go and look at what this the poly policy with this identifier and wrap handle async in that poly policy so in other words if this fails we will retry according to this policy that's then wrapped in another poly policy which is a circuit breaker some others say if the retries fail we may choose if we can see enough failures to break the circuit to this particular handler which means that when anything gets sent in future it would just basically get a broken circuit exception back out and then effectively on the outside we have a logger which basically says a log or attempts to basically run this pipeline so as you can see that's i guess it's almost like this is pretty much middleware like if we if we compare it to um you know like that russian doll you know request and response out type model like this is pretty much you doing it but within the context of brighter yeah exactly just the same thing it's just middleware yeah um okay and uh again we'll come back to post a little bit later so the other thing we had when we look back at the controller is we said hey here we can do a query press we said it was darker so we're doing here i mean it's pretty obvious we're saying hey go and get back basically the greeting that we just wrote into the data store right um the thing is what our command is not doing is giving you return value so we don't give return values from commands there's a trick which basically is if if you want to do that to work around that problem uh you can just put basically an ad creating command you put the out parameters because you can then update them in your handler and then read them here but that's really if you do things like generate identity based ids in a database you need to get them back to retrieve the item um here because we're just using a guide and to do that we can basically just go and effectively um pull the items straight back out with the id that we we gave and we generated in the first place and this basically is using darker which is kind of the flip side it's the thing doing queries rather than commands so again it just basically says hey go over find basically the relevant handler for this and execute that so we're doing greeting by id which is over here and again you just write basically this bit so you're saying okay i'm going to go and read stuff out and again it's got some middleware um two different groups of people slightly you have written bits of brighter and darker over time so you might find that the names a little bit different um and at some point we we have a kind of backlog item to try and sync up some of the provided middleware names they use retriable query for running a poly policy whereas we we actually use use policy at some point we will sync that all up um but it's the same idea right you've got middleware like prologuing rechargeable query that you're basically using to go and effectively run um the query okay now the other and nothing i like that you said that so the question for me now would be is brighter middleware and darker middleware compatible like can i share them between the two or would i have to write different ones for them i think at the minute they are close but not the same so um yeah one of the things i think for us to do is to basically uh make that much more compatible okay and then uh we also have some folks in the chat talking about mediator um right could you could you give us like a two seconds comparison a lot like brighter and darker versus mediator sure so uh so we came first uh and uh when so i know you mean really well so mediator is more lightweight right so what jimmy wanted was something that basically didn't come with any any other dependencies so we take dependencies on things like poly and stuff like that etc and provide a lot of that functionality media is very lightweight it comes with first dependencies so it does less out of the box for you you have to add more but if you've just got a very simple use case to me it is basically a pretty good option i sometimes said people if you you know one thing to do because they are so similar and the way they work is you can honestly start with mediator and then say hey bryce has got some interesting functionality maybe i'll get i'll take him brighter and it's not a huge job to then swap media to the brighter um but they're similar kind of ideas yeah and we mediated doesn't have say the split that we have between brighter and darker as a kind of query uh model and a command model it kind of does it all out of the box on one thing i i think it's don't think in the middleware works or anything like the same way as that's not kind of attribute-based i think it's a later edition one thing we do do that mediator doesn't do is was given is the kind of the hint of is here which is a post okay so a post says rather than using an internal bus if you like basically to send between or a sender and receiver use an external one okay so send this over messaging based middleware so you can use it to basically do a couple of things one simple one is to communicate between two microservices over basically messaging middleware the other is which is kind of related is to say hey within my actual service i've got like a web and a worker because i want to offload i get a request in i want to actually just stick it on the queue chunk my way through that work more more slowly and just respond back to the user and say hey i got your request i'm working on it right typically a patent called a task queue um in python land they use cellular a lot for that particular model you see django sites using celery for that particular piece of work right so we tend to use them here used in both those two circumstances brighter although increasingly a lot more recent it's been used a lot more microsoft as well so what you do is you can accept a variety of transports rabbit kafka as your as yours from nine basically you've got decent zero transport so nine is currently in beta um aws ask us an sns we've got redis buyer basically service stack uh be aware that that service stack is commercial at certain level usage we're also probably gonna do streams via um uh uh the um uh other regis redis implementation which is from the stack overflow team but i come at the point where they call theirs um uh and we probably get a few more transports out over time i i keep being semi-persuaded that we'll do gnats for basically folks around running in kubernetes um i guess we keep looking over at dapper and seeing what what what transports we make we may have to compete on um and i guess it folks too if folks if folks that are watching that they want to have us transport that you don't support like it's open source right so people can always contribute there you can add transports and transports are relatively straightforward to write that's one of the biggest errors of contribution we get so one thing we prefer so why don't we didn't have an azure transport for ages y because none of the core team at the time were in an azure environment we were either data centers or aws but we but we had people who who were on azure environments using it um and the trouble we had was they all ended up with these internal policies can i say no you can't raise the software we finally managed to get a company that said oh yeah you can have ours so we got a decent contribution when we know that works at high scale which is what we wanted before we basically said everybody else yeah um so that's coming out in nine basically um but why can switch and show you probably is i can show you this command processor post working if i switch over to this now probably what i have to do is do a call kind of like um let's see can i get it to scroll up the screen that's going to be interesting isn't it well i've never understood why that is the pain point of all demos with console applications is that they live at the bottom of your screen um and so this is this is basically just i'm running uh docker compose i've got a number of docker containers running the actual app you've just seen but if i uh what you should see here as you can see actually we've got one already before you can see there's basically the web and a worker so the worker effectively is listening to that post request so over here this worker you'll see actually and there's not much doesn't pretty much going on um we can talk about brighter configuration this is an older school version of configuration now this is a host builder one i can show you a quick glimpse of that later on does it all this is long this is the long version so under the hood host builder just does all this for you now um but the key thing to understand here is we're talking to robert mq we create basically a connection to it uh and then we build basically what we call dispatch which runs basic message pump that does all the kind of hard work for you talk about that it looks much simpler now i can show you basically how it looks basically with a modern host builder version a bit later as i said this is an order example because i crashed the new one um but the key thing to understand is uh what we'll do is we will say in a subscriber registry when we receive this command run this handler now this actually happens to be over here in the handlers for this this project um what is this going to say when i get that handle i'm going to pump something out to the console right so it gets sent by the web app and then this worker app picks it up and just pumps something out just so you can see messaging kind of working and what will happen is we'll get another message come out like here so we go over to http um tests we run a post let's run one it's from allison we've seen alice before one lat okay and then we should see when we go over here hello alice now appears basically in the and you can actually see if you look up a bit further though it looks little bit kind of like um uh complex you can see worker here is spitting out something so this is what's coming from that request logging piece of middleware that we added basically is one of the attributes it's telling you receive the message from this queue with this routine key via this exchange and rabbit and it tells you basically what that message looked like on the wire and then basically he's also going to tell you some other stuff about building pipelines et cetera you can you can probably see if i switch over to this new my email reminder from cecil this is rabbit so basically rabbit has let me give you a bit more zoom in a little bit so you can actually see what the hell's going on here that's good q here and you should better see let me just send one we should better see hopefully uh the queue kind of reacting sometimes we can capture it and sometimes we don't we don't see the message right there yeah so you can see the message kind of going through to that green line as it waits instead of rabbit all right um just to show you because i broke my world of demos uh in writer itself i should better show you what a modern uh let's show you this one and then inside of the solution like what you're showing us is like that brighter command repo or this is a code from the repo so like if you went to the github repo and you go to samples everyone will be able to see the code that you're showing right now absolutely so this is kind of uh how we configure with host builder which is a little bit simpler um which is kind of what we do nowadays in the version nine so effectively uh a subscription this is this is the work of the consumer says hey i want to listen basically to this given channel so that's basically uh um from point of view of rabbit this is naming it naming it my friend's naming your channels especially the zero subscription so we're saying listen to this channel right um and then effectively it's saying um here's some additional parameters like when we're gonna time out um do you want you to create the infrastructure so in some cases we will create the infrastructure on the messaging framework if you want you create validate says check that the messaging infrastructure exists before you run and there's a kind of assume that says hell i'm just going to go for it um re-cue and there's an is async flag so each one of these sets up an independent what we call perform which is a message pump which reads from a queue you can have multiple message pumps reading from the same queue you can effectively uh increase incredible number of performers uh but you can see one of ours is async pipeline and one is a one is a non-async pipeline um when you're writing workers listening to message queues sometimes the is async async isn't necessarily as big a performance again as you might think because we have a thread reading and message pump that also essentially runs the handler etcetera it's a kind of single threaded performer and you introduce more threads to read from the depth predictability because then you know exactly how many threads are reading the queue and you can effectively predict what your rate of consumption should be you can use things like queuing through to predict what's going on so we don't tend to basically use like a thread pool and uh explode the number of threads that you're actually using you get to tell us how many threads you want to use to run worker processes to read a given cue and because of that effectively you tend to find that if you turn on is async you're not getting a huge benefit because we're not going away and reading more work while it's currently working flight because the problem with that is we have to basically spin off threads to basically pick up that work right right um and that's the problem what problem is called is probably more complex than than this particular session i don't hesitate to think that you know we have ever scheduled a third session so but this idea called back pressure which is basically if i consume a stream one of the important things for me is to be able to say how fast i want that stream to come at me otherwise i have end up drowning right now i'll be totally down for doing another session in the future about back pressure and some of these other problems and folks that are here that are still here with us if that'll be something that you're interested in as well go ahead and you know just send us a message give us a thumbs up or something because if this is something that you'd be interested in doing we could always talk about having ian come back and and discussing that topic with us okay um so uh yeah and then we basically go and this stuff a serp activator is effectively just saying something that actually listens uh outboxes uh we don't have time for outboxes tonight but an outbox essentially says when you send something i will store it and i will mark when it's actually been sent on the messaging platform some messaging platforms will essentially let you send and then later on basically via a callback tell you what's actually sent by what it means is copy to all the nodes in the broker um what we'll do is we'll watch for basically that failing and then we'll resend it for you um uh use external baskets basically says effectively we're building brighter here but we want to use an external bus for doing posts so please configure all that stuff for us um okay uh but yeah i didn't know um i didn't know you had that feature inside of of brighter like i've seen some folks do it with other other things like i've seen yeah folks do i'm gonna send him thing to rabbitmq for instance and it'll store and push it to another broker like something in the cloud or whatever the case is i've seen those types of things happen i didn't realize you have like a book functionality and brighter we we yeah it's better from nine we had it before but it was difficult to use unless you really knew what you were doing um we've kind of basically said that's a that was okay for us but nobody was using it because we didn't make it clear so with nine it's much clearer how to use it and we also provide some nice in-memory alternatives if you don't want to basically use one that's actually tied to a kind of kind of persistent store um and it hooks in to say sorry mq for example i think a publisher confirms which basically says uh hey i'll acknowledge when basically you begin sending so you can move on to the next thing but only when i've actually copied it to all the nodes will i give you the confirmation back and that's when we basically update the outbox to say it's been it's been actually sent for example um and it's also useful effectively if you say for example uh write something to the store but you effectively then can't send it and rather than effectively abort that operation what you want to do is hey save that i've got the message that i want to send saved uh at some point when that's thought when the broker comes back up you'll send it for me right so there's a tweet for process you can run the background the background service that basically says keep looking for failed sends from the outbox so yeah that exists for us so this looks like a pretty pretty cool setup i know we're running a little late on time if folks have any other questions they want to send to ian feel free to go ahead and drop them in um and then ian do you have any last like any last demos or anything like that that you want to show us no i think that's that's cool i think that's enough for one night i'd rather answer any final questions people have um i'm i cooper on twitter and my dms are open i'm always happy to answer questions about this if you want to ask stuff around brighter we basically recommend there's a just we have a on on brighter command brighter on github maybe we should show i should show with that url so that would be a good thing to do next yeah that'd be cool and then uh we do have one question that's here it's a landing page you go paramore i o you can pretty much get to anything from there if you want to go direct to github um the org is brought to command and we do have a discussions so if you've just got questions you want to ask without having to want to raise an issue you can kind of come in there and we'll basically uh answer a lot of stuff in there nice now one quick question from the audience coming up from youtube and um this was around the ca tutorial that's in your repo he says i opened the repo but it's hard to know how to get started yeah it's a little bit broken because i'm trying to fix it up um what give me a week or so i will fix it up i will put stuff in the reaper that explains exactly how to use it and what it's showing you step by step um and i will tweet out about it and i'll make sure tesla knows so he can basically talk about it maybe uh next time and remind you that there's a link up it it i i i not like all software developers i estimated it might take me three or four hours to fix it i discovered that it was a little bit more work than that so probably generally my estimates are usually out by a factor of three so um probably give me like in about a week or so i'll probably have it done but let's call it too awesome awesome and then so if folks want to learn a little bit more about brighter they can come here to the repo um i sent the links to the different tutorials so it was the 12 factor tutorial and then the ca tutorial yeah um is there anything else necessarily any other things you want to share with fox before you take on um this page is is a useful starting point we've got some elevator pitch stuff uh and some some basic stuff here uh we need to uh when nine is just coming out it's in beta um uh nine we won't update the docs for yet so if you want the kind of safer earlier version of write to untag the pre-release thing and just pick up eight point whatever it is which is which is the last release we did if you want all the newest features um you may have to come and ask a few questions or look at the examples we will respond to the questions but we always uh like any open source project find the docs are a little bit behind the uh my final like to everybody is if you want to make friends and open source projects right and become welcomed amongst the community the best thing you can do is get the project and help help write the docs for everything you learn trying to get it working um uh and you will be you will be loved and hugged i i i can completely definitely relate to that for sure well ian it's it's been amazing man we've learned tons of stuff um we have lots of folks in the chat saying how interesting this was and how much they've learned from it um and we got a lot of thumbs up for you coming back on and doing something about like um you know back pressure and talking about some other topics so we're gonna have to talk about this again at some point yeah it's about me from stuff like that maybe um that's something uh you could pitch you in in from from your your experience my friend um yeah i think we could definitely we could definitely talk about like some of the things that i've seen from from that perspective as well yeah yeah so for folks again thank you all for watching definitely appreciate y'all spending your time with us um ian thank you as well and we'll be back again next week i think we have an f sharp topic on the stream next week but you know again stay tuned and we'll we'll post that um to our channel at some point but until then i hope everyone has a good weekend have a good evening and then we'll see you all again next time bye everyone [Music] do [Music] you

Channel: dotnet

Views: 14,303

Keywords:

Id: IRvDGPbxdTs

Length: 98min 26sec (5906 seconds)

Published: Thu Jul 01 2021