Distributed .NET Core (DShop) - Episode 7 [Handling asynchronous requests, SignalR, Polly]

welcome to the seventh episode of distributed dotnet called curse it's Peter and Derek and today we have a good news and a bad news so maybe we'll start with that so the good news is that we've created a very simple solution based on on donut color and the Blazer which allows you to manage your services via a simple web UI basically start and kill your services so you don't have to do it for the console yeah you can find it here at deaf mentors governor so yeah just another name for the manager 2 zeros for the winner and you can simply start it by going to the by cloning the repository and you can either go to the SRC and govern order dot server and type doesn't run or you can just stop execute the start a shell sweep which does the same so let's take a look it will start this simple UI UI and localhost 50:20 yeah so it's going to be available here in a moment and the overall solution is also quite simple so in this on the server part you can find up settings here and under up settings you can define any apps so for example here you have the name API customer service and service and so on then there is a path so if our governor service is located in the same same directory as our services are we can just simply type this path and you have to be aware that it has to go these free folders Adolphe starting from server to SRC to root and then yes ok so basically you need to clone this repository on the same level as you keep you yeah services yeah change is required yes yeah and here you just type the file name is what the file that will be start as process arguments so doesn't run so it will basically go to this directory execute doesn't run ICS prod and you will be able to navigate to your service so let's see it should be up and running right now ok as you can see it's it's created a UI for this services and I can open this up and here comes the Blazer so forgive me for this UI yeah so I can for example click start on the on this API and you can see here in the logs that it will create a new child process for the service and the API will will be available quite soon of course it's it might take a little while but yeah for now it with that it just the UI shows like you know immediate result that is the service yeah running but it takes some times yeah so you can see it started here yeah it's just that you know MVP so maybe will then add some yeah option for the events that comes through the along with the process so yeah so yeah here here is my here's our API so we start with API and basically it's all fine it's called the internal API endpoint to check whether the services are up up or down so you can see whether they are running or stop and you can either kill them or start them but the only issue that we have for now is that when we actually set this when we click on is kill and even though we can see that it killed the process it's not down and it's simply because there is this buck right or or sort of an issue I don't know whether this is a bark or sorry yeah basically underneath as you as you saw what happens is basically we call the dotnet run so we call this CLI to to run the micro service for us and what it does it actually creates two two processes so as you see in in this case we have the one separate process from the dotnet run and underneath we have additional child process that you could say you know on the linux when you type ps3 or something so this creates this additional child process that actually does dotnet eggsy and we don't have an access for this for this child process we only have a process ID of the parent one and this is not enough for actually killing the particular service and I found a workaround for this or maybe not work around but this working solution but it is Windows specific and we we kinda need to figure it came up with we have some more Universal idea than just we know Windows specific so if you know that how to kill a process by giving ID and all the children processes you can contribute to this project and maybe you came up with some something more clever than we did because for now we kind of we need to take a look at this yeah but this is once it's done this will be much simpler to actually test your services because you can just start and kill and you don't have to type it type it into your terminal yeah so that was a good news good news and bad news so now let's get into our today topic and one of our goals is to make this video no longer that the previews up we'll see okay so we close this one and today today we're gonna talk about the thing that actually that's been actually there since first or actually second episode and I think is called correlation context and you can find collision context in much one of the handler right I think I think that yeah but I think that we should start with more general briefing I mean with I I would start personally with the thing that actually we followed for all these episodes which is the secure s okay and as you probably remember CQRS when you want to be you know be exact with this principle then writing to your application means that you can mutate the state you can persist some data to you databases or play with your domain objects and that is totally fine but as you see in this example that actually we were closed can you get back to okay yeah yeah this as you see because this is the command Handler and we do this interaction with our persist with our domain then we return the generic tasks with no result we cannot say whether this action succeeded or not because as you see we have this very simple validation that we added and as you see for now if the customer is not is no we kinda don't think we do is logging and then we just return we are quitting from this method and as you probably guess we would like to inform end-user somehow that his operation failed and it would be also cool to to give him a reason to display this on the UI or something so that's the that's the I would say that the CRS implies this this issue because in a typical scenario or when you you know when you have the well you don't when you don't follow this principle it's kind of it's not that easy because we could return some boolean or result saying that okay this just simply doesn't work and of course there is one more thing that we have this a synchronous communication between API and the particular microservice and we just use the publish/subscribe messaging know the request response so we even though we we could have some result returned from the handler you were impossible to return this from for the for the QE so we have these two issues and now we need to kind of solve it yeah and like like we said quite some time ago with a synchronicity all the fun starts and also all the issues arise yeah so yeah we have two issues there the first one is how how can we let our end-users know that something either succeeded or not for now we are only able to say that something's and see that simply because we publish this discount created or some other successful event type right to the app to the bus and then the user could you know just get the get the discounts by calling this endpoint and maybe they were there were there were actually in a database but maybe they weren't maybe there was something wrong right so even if the even if the user was maybe to create a product they already exist he couldn't really know whether the operation succeeded or not simply because he would have to fetch the data from database and see oh there is no data what happened right so how can we let the end-user know what's going on and the other thing that's actually part of this issue is that even if I will let my or how can I let my end-user know how can I know which user originated the request right because once you create a request for your API for example let's say products controller and you send a post create a product then how can you know that exactly this user and this request started here is the origin of the event for example product created or product updated that will be published later on right so these are two things yeah you remember that that having this this domain distributed into several different micro services implies this that for some actions that that starts from the UI you might go through several different micro services to actually to actually complete some in this case I would say that some kind of a use case right so one one business transaction might go across several micro services and you need to keep tracking this this operation we need to kind of track this and know that okay so we have this we have this event that was rised by another event which was rised by a command and the command was posted by this user we need to kind of keep track of this to let the user know that something happens with his particular message that is very yeah it's very important for this yeah and even for such simple use cases like comment event you get this issue immediately because you can't match who created who like where this event that came from who originated this event which command or which user so this is it this is what I always said at the beginning this is where this thing called Croatian context comes in handy and the idea behind Croatian context is quite simple so collation context is your context that you can pass as part of your of your message and if you're wondering how is it being passed if it's for example for rabbitmq you can pass it via headers but maybe we will show the color later on yeah and if you take a look at this interface is very simple and you can put here anything you want so this is this is some of our properties that we decided then will might make sense here but really the only thing that you need most of the time is the ID this collation context ID okay so if you take a look at our API let's say our disk controller or any other post put and delete method and if you come down to the set async you can see here that we have this get context and what it does it just creates a new context so we have this simple extension method here and a constructor so it will have this data set and it passes the new ID right so create first parameter ID so when you send a request to the API and it's you know this sort of common request which is not a query it will get a new ID and this will be our correlation correlation ID it's called solution ID and we just decided to put it into this class this interface because we wanted to have more properties here but we could be just fine only with having a single ID property nothing so we have this collision ID and then let's say we have this discounts trade this confounder oh wait maybe yeah can you just get back to the because you show the get context and we need to kind of actually show how it's published oh I see we have this send icing and that's right gets two arguments we have this command and the context that is actually then yeah as you see yeah so yeah there is it context and you can pass this for example for O'Reilly library and we have these salt contexts use message context and you can put there any class any interface use message context is provided by rabbit one of the pockets actually yeah and it will sell eyes using the headers in the rabbitmq yes so just to make it clear correlation context is nothing more about the metadata that comes with the message but what is important that we will kind I think that there's a word like for this like flowing like this message will flows will flow through the together with all messages that will be actually part of the let's call this business transaction even though it's just Kommandant and event that corresponds to this so we will not mutate this correlation context as you see this is immutable message we just want to create this at the beginning very beginning inside API gateway and then just let it go with with the messages that will actually be involved in the particular business transactions yeah so use you publish it here for the first time you created here down below with this unique relation ID and then later on for all of these message handlers you get this at the second parameter whether is an even handler or common handler you get this here and then in order to you know pass it further you just pass it again for this publish a single setting methods as a second parameter and this is how you pass the same context from the API which has the same ID and the same metadata and was assigned in the first place so this is how we can resolve this issue of not knowing who originated the first request and this is where this context comes in this basically works the same way like you know when you order some parcel from the from the internet and you get this delivery number that you can actually between some website to tracker delivery so this works the same way but in the micro services we just want keep this in there in this correlation context object that's okay so maybe let's get back to our yeah and our my previous problem once we know how to deal with this with passing this at the same context over and over again so here we have this happy path discount created and here we have this path which is not very happy because our customer does this and we cannot create a discount so we need to let the end-user know that we couldn't create it and maybe you know send him a reason why you couldn't do it some some error code or whatever makes sense to you so what I can do here I could probably throw an exception but it's not really helpful because the exception will be from and it will be handled internally by the service so did so that end user would have to somehow get this exception you know send to him being you know send we will have to send this exception to him right back or the end user would have to you know call some and roars or exception service and see all there is an exception what happened right so don't wanna do it we just want to send him and response as you would send from our API like here and you could say alright so my post failed and here is the 400 a bad request that was called with the errors collection and with error calls we can do it here but we could try to do it in a little bit different way and we could do it again with the events so here we have our happy event this concrete it and now let's create an event event sorry yes that's a very happy event not so happy event exactly so we can call it something like create discount failed or or anything similar which means there was a failure and especially for this scenario we created an interface called I adjusted event so as you can see this is an AI event and we just defined this I rejected event and it has two additional properties and the reason so you can put your an exception message if you have some you know save exceptions same except some messages or some other description of your of your of your issue error code you can also put here some other properties yeah so we'll create a new rejected event for our trade discount so let's get back into our discount service and here under the messages directory let's create a new class and let's see if my handler if I writer doesn't doesn't stack hopefully get stuck so I will just call it create discount rejected like this well okay so right now yeah I was afraid I was jealous close one close call alright so it will implement this I rejected event and it requires to just add these two properties here and yeah that could be pretty much it but of course we can pass hit pass clear even more more data so besides this reason and code maybe I could also pass here the customer ID like to eat customer ID or the discount code or whatever that is right so feel free to put here anything you want and just a better and let's do the constructor now that's awesome awesome yeah that's my favorite yeah at least for now and mark it as a JSON constructor and this is this is our event now we can get back to our handler and let's publish this event quite similar stuff is down here below so we just call our bus publisher published a sync and then new create discount rejected we'll pass our customer ID then we can pass a reason so let's say we can just top it the same message or maybe there will be some exceptions you could get the exception message and let's pass you an error code so maybe I would say customer not found like the idea behind the error code is that and on your let's say web app you can take this code look it look for it in the in your dictionary and then render you know meaningful description for this error code so this is why we added this error codes here it could be animals in homes number because sometimes when your website supports more than one like this is very helpful to use this yeah so we have this additional parameter as you see this is the context and we don't create a new context for every publishing we just use the same one so as as we said we keep we just keep it with together with our messages yep so now when we start the app okay well we can publish this message once our customer doesn't exist but again we have no idea what happened because well it will be just a message there will be sent to our rabbitmq message bus but the end user still does know what happened and why it happened and unless he knows rabbitmq yeah and I unless we gave in give him an access to to our RabbitMQ which we probably didn't want to do it at least in the production mode so what looking and what you can do about it well if we take a look at our and post a missing or source a nascent in our API ID a gateway let's take a look this comes controller send a sink for a pulse and here let's see it returns the it returns dto to accept it status code but it also returns this magic magic header so in to up for up our response headers we are adding this operation we just called operation it's not like a standard we just figure it figured in our own and we have this operation header and I also had once a good discussion that something that it turns out that X is not needed for the custom headers as the specification says so you could just lay this so we have this X operation header okay and we are adding it here down below before we return our acceptor response and in this header we are returning this operation slash context ID so as you can see we are returning here some sort of endpoint it could be relative it's a relative endpoint but it could be of course absolute end point like in location header which which points to operations slash and our collation contacts ID and of course it could be some other ID but we just decided let's let's take our Croatian context ID because it will be the same for all of the you request all of this message chain and there could be a lot of messages so we are interested what happened with our operation ok and we could probably get some details about what's going on with our current request but before we can actually ask the disservice and the service that does the process if is called operation service and we can ask him to get returned this data for us about this operation so whatever is happening here we define this operation or it could be a request right you can treat it as a request that is being processed and a request status we just omit operation and before we can you know start using this operation for our and discount we need to define when to add it into these operations operation service because for now it's just message and this message will need will not be stored anywhere especially the operation service because because it has no idea at least for now that there is a new event right so we have to define an event for him of this type yeah basically what it does this OP whole operation service what it does is basically this it subscribes to all messages that we have at least that defined messages I will pull it put it this way so so we have this I think that we have this yeah as you see we subscribe to all commands on it to allow dance okay so basically we have this subscription for for everything and once we publish a command or event we create this operation domain object which has the internal status so we can say that this is the operation and the status is like pending or rejected or succeeded so basically as you see maybe we can say yeah so this is the DTO for this operation so as you see we have this ID of the operation which is nothing but the code yeah but the idea of the correlation context we have this user ID that actually post the first command and the name and the other stuff and the state as you see so in this case might be like successful or rejected or something so the flow is is very simple so once we publish the command first the discounts will actually get this command and that will start to process this but also the operation will subscribe to this message and will create a corresponding operation object that will represent represents this the operation and once we publish an event might be either successful so I event or reject it so I reject the event we will simply get the operation object from the MongoDB from some data access layer will update its state so we will say okay so that was the pending operation and now it is like rejected or successful so that's basically what it does yeah so yeah if you are if you don't have the latest report the latest suppose stories make sure that you will pull the latest changes for operations and super hour service and also the comment packet recently we made these things a little bit simpler so here within this generic event handler we are listening down below to either rejected events so we know that something failed and also for you know just generic events meaning that there was something successful and this is only scenario for this typical simple use case command event which is all we'll talk about this scenario probably the next episode but for now we focus only on this scenario so you send a command the API sends a command for example create product and then either the products gets created which is happy which is the successful event and it gets here or the product gets you know doesn't get created and the product service it just publishes create product rejected event and it gets here okay and you know it just says this so know this is that this is the version without the first step so yeah this is without the without the common just adjust this successful use case because with these commands there was sometimes this race condition and because sometimes you could have the event before the common and so on so I just made it a little bit simpler for for now so we only seem to this is events but of course you can you can extend it with comments and also subscribe to these commands as your first event so now we have our rejected event defined we need to add this to this operation service and just I want you to keep in mind one thing because here we decided that okay let's just put here all of our messages but sometimes you don't need to have this sort of you know very responsive you know subscription where you know where your end user knows right away what's happening with his request because sometimes let's say when you create an order the only thing that you need to need to get is maybe an email message in a minute saying all your order was completed or there was an error your order was rejected so maybe this is how you can handle this you know this whole flow and how you can let your end-user know that something failed or not maybe by sending him an email message or an SMS or whatever it is but if you want to have something very responsive and you want to know what's going on request right now sort of real-time then yeah you need to I would say that this also depends on the domain of your application because you know if if if your interaction with the UI and the messages that comes from the backend are very important and you should really take this topic seriously and make sure that all your responses will will come as soon as possible using the techniques that will present you in a couple of minutes but if you have like you know that your domain is it's not that focus on actually writing more than reading I would say that the example might be some social media or like reddit or something that you know you can upload something but if if something will fail and you will not upload the post and this is not the big issue for the reddit as a website right the for the parade it's the worst the I would say the worst thing is when you are you can't actually read something right when you can access some articles so in this case it's not like all messages needs to be keep tracking in the real time only some subset that really matter to you and you want it to be like being tracked real time because you have to remind it again it's another service so it can be another you know some point of failure and you have to maintain it service so again remember the less code the better so that's but ok let's let's define here our message so under messages I will create a new subdirectory and I can just call it discounts and here I can you know just take this credit or reject it and just add this event so just like we have our messages for the remaining services defined advanced and let's put here the gray disconnected and in this operation service the only thing I really care about is the actual the context collation context which is being passed for the handlers and also the either event or rejected event and these two fields these two properties so I don't care about customer ID I don't really care about you know product names price anything like that so I could basically copy just and this part without this customer ID because I don't care about it at all okay like this remove this field and now we are good to go off and a constructor yeah of course okay and now we are good to go because down below here as direct set produce reflection will subscribe to all of these messages at the beginning and there are type of either event or I command and of course of type I rejected again because I related event extends the I event interface so right now we should be good to go I think and we should we can give a try it right yep all right so let's let's see and what we need to start so we'll start free will start the API as a gateway so let's get it up I think dude we should use the down shoot of this yeah because okay that's not think the service is actually words let's see yeah now searching works but let's let's hope it it will not break anything this is the moment of truth for this yeah okay so let's get here and if it is just work we can at least showed what we which you want to start so I want to start the API for sure gateway and then we we don't need to start the customers or okay we can start the customers it will be for the happy path so let's start the customers let's see if it should start yeah quite soon okay yeah yeah so let's start the customers now API is up and running right now we are starting customers yeah the next version will have some progress bar here yeah we just need to subscribe to the event that comes the process object I think that the definitely exited for this I'm not sure whether this is something like started hopefully it seems that oh my thread is is down somehow okay it's not down yeah okay so API Oh customers yeah two services are running I'm an identity correct yep we don't end ratifications I know we need operations yeah and we need this cuz you don't have this comes here so we'll just start them for the console because I don't want to restart this and update my app settings so I think that we yeah we can start the discounts from the rider because we could debug them yep and yeah so the last one was operations okay let's see okay alright so let's start our discount service and in my debug let's choose the discounts yeah okay and let's start it and let's put the breakpoint into our handler which is here here and also here for both paths okay let's wait for our operation do you have any customer in the manga no no Mady B is cleaner because I always restart everything dr. Campos so let's start me with the happy buff okay okay let's go to our API and to the rest and yeah let's just create we don't care about products for now so we can skip this part I only want to create a new user so I want to sign up let me just check whether we have our discounts Handler without any and a specific authorization I think it should be plain and simple see allow anonymous when they call yeah let's see alright so we have a new user and we can just sign in I'll get the token okay and now let's let's try to create a new customer now let's see I will click send request and even here you can find there is this URL operations and here is our collation ID so what I can do now I should be able either to call it through my operation service directly like this okay let's see 404 is this yeah let's I'm not sure if the if there was a customer created event let's see okay let's see if the customer is here just to be sure yeah okay he's here maybe the endpoint was simply bad could be or let's say prescience should be fine precious controller yeah the only issue is that it's a little bit more there are some troubles if I want to do if we want to scroll the walks oh yeah there was no signup identity customers post operations okay and there all right let's see this will be much easier to trace once we actually introduce tracing yeah that's that rather than scrolling through the for the walk maybe just go to the eyes yeah or I know which which could be the issue it could be that our Redis doesn't work or it work let's see if there's data here okay maybe maybe our maybe he didn't store the data it could be that when operations are started where this was down or something maybe it had some issue to to talk to restart this yeah but yeah we have to stop this right yeah okay let's do this far away I will just kill my operations service that's what they are taught that it's going to work no but you don't know the process idea of this yeah you can you can choose from six or something no okay here it comes oh here it comes I will get I'll get I just need to make a little bit smaller at least for now okay one more time yeah operation we're looking for operation is this one four or five three six that was smart and I can just kill a force kill for five three six like this okay it should be down let's see yeah it's down look like a pro in a terminal yeah all right so it starts our operations here like this okay and let's let's try so our operations as I see it has this operations controller and it should be a big customers for this identity oh yeah sure I will but I'm pretty sure that it could be something related to readies because only it talks readies together data so it could be could be summation all right so let's let's let's drop the base and also remove this one from here okay all right it's up a lot of subscriptions to the messages you can see yeah or maybe maybe something failed during this installation I'm not sure all right so one more try create your customer again okay okay so we'll handle the message that were replacing service and now let's let's only take a look at our readies some already see Ally should be able to all right it's not here somehow but let's take a look we have the operation ID this one and it should work that's wait mmm-hmm somehow it didn't it where's the controller of this operation so operation storage set and it should store the message okay maybe they just run a debug mode maybe there is some buck keep yeah it's it will be the best way I guess [Music] okay it could be something with Freddie's butt yeah let's let's just be certain so I stopped our this comes from now and let's debug the operations ah I know what's the issue and this issue is quite obvious I guess so I'm afraid I forgot to put the message namespace oh yeah our message yeah this is I I think that's this is it but this is discounts ah all right so it's about okay let's let's hear but also this we would be surprised by the next next one doesn't work so alright let's let's put a breakpoint over here and here and we just put it here you will see maybe there's some issue of the connections ready yeah or maybe there is some issue and this part maybe it takes this message somehow see shouldn't but I think it should this we have one saga that actually corresponds to this yeah it could be this one it could be the issue with this one I'm pretty yeah you can put the breakpoint inside yet should we have you returned I'm not sure anymore I don't think so but there shouldn't be written right yeah okay maybe maybe those processing processing by the saga is the one thing but yeah okay alright that makes sense so but yeah let's let's let's do it anyways maybe I did something it wasn't meant to be here alright collections refresh okay yeah this is fun yep okay so let's try with this thriving debugging yeah so let's try right create customer and we should get this message here alright so handle alright we don't care about this one and we should get the event about customer being created let's see yeah as long as it was created a lot of love yeah it's missing our our locks and well we have some null object I see maybe did I see some type of customers but a typo in what no that's fine yeah yeah so we have this no reference exception and this comes from the Creator from the handler create customer that is fun okay oh hey I know which one I remove the customer and the customer is rated to the user identity yeah that was the issue okay yeah because user is a customer all right I was so that was the issue do ya okay okay and it could be also the issue of our operations of your turn but yeah finally this is the debugging everyone yeah alright so the identity service published this events steady sync and yeah that's only take a look if it works correctly and yeah right now right now we should be able to actually take a look at our already's let's see we have the data here or not yeah finally it works so we have some data here all right now let's create a new customer and maybe I will start our discount service just the empty grid yeah because for the identity we don't really pass anywhere this this context it has no context it's a separate service they're exposed right everything the synchronous here alright so maybe I'll just start our discounts here all right okay customers final try accept it and we can get our operations D and let's wait take a look okay think that something is wrong with our our customer service it does not create okay it's it's here okay maybe let me just ensure that I have the correct access token and let's make let's see if our customer okay it's normal all right sort in the products the customers okay finally we are here maybe I had a bad access token there were some mismatch right so we are getting here into our events but before I click a five let me take these operations ID and let's see I can call keys again it's my okay it's not sorry it's not it's not here yet because it's an event so f5 all right we are not getting here for now and the return wasn't it here that's correct so I have to publish the changes right now we will store our state which is completed because it was a simple use case dress way the customer and nothing else so yeah here is our key and finally I can grab my operation video which looks like this as you can see yeah finally our collection contains ad the user ID the name of the command the originating the request state for example the resource the end point where we can get our customers for all cold reason empty because there was no exception yes so maybe let's try to create a customer again and it should fail now now let's see so one more time create a customer and yeah as you can see we have our rejected event right now with these reason the customer account was already created for user and some code so f5 again we are getting into this rejected events ok saving the state yeah no service readies I will just remove this breakpoint here and again we have our data already yes former this know those first be the this one yeah okay I think it was age gets and data yeah I don't remember the exact as I just put into into the browser yeah I think I was such get the P and there was a field called data or something all right let's let us put it here so again our endpoint and we can of course code for our API which is this which is desired by the end user because our patient service is it's running internally so for the API gateway I will get my data about operation as you can see great customer state rejected and here is my cold customary completed and customer was already created for user blah blah blah right so here is our our rejected so this is how we can and let the end-user know what there's something wrong and maybe this last try and maybe we'll just do it here let's say try to create our disk aright maybe without the debugging so this will simply do this we'll see whether our validation actually works okay I think the thing that we actually very we didn't see it coming in their web browser so okay alright so as you can see even though our these concerns was down because some people ask what if my service is down and sometimes on Forbes or meter well if you have the queue and your service is upper running again it will get the messages and so our discount service has this customer it created this customer and everything is fine right so no worries if your service is down it's it's normal behavior just make sure that the RabbitMQ is up and your queue is alive I wouldn't say it's normal behavior but I mean yeah but it should be it should be afraid okay alright so right now this discount it's down below here and this is this post our customer ID is something fake we just call it and we have our operations ID and take a look event and it's a credit score ejected right cool so we should get into this part and we can grab our operation ID and let's put it here and yeah it's working so it's quite cool I will ready of course now because there is there was no authentication required so do it empty here or no if you want and this is how we can deal with this stuff by having some others type of service that subscribes either all of the messages or some subset of messages it's interested in and it will somehow internally track this state maybe simply by you know saying that it completed or was or failed or something else happen and this is for this simple use case create comment event yeah but it's more complex for the first if in this case we will get in the next episode because for this this is more fun I would say even more fun yeah for now it was just it was merely any fun believe I thought that was the that was the happy path would say in this case yeah but we still have of course so now we have this expert presented to you we have this endpoint the both or inside this in the API gateway and which actually calls the internal API of the operations service but we need to kinda deal with it on the UI and of course some say that you could do some long polling and just set the timer in the JavaScript and and there once a few second just ask about this operation status and once it's it actually returns something from the API we could create some toast or some pop-up or alert whatever you want to do and inform the end user that your operations succeeded or or was rejected but of course I mean we don't claim that long polling sucks but it sucks actually so for the most of okay but we just want to do this more Pro and do this in the I would say more real-time way so we are going to extend this functionality by adding one more micro service that yes you know micro service this simple solution yeah we'll use the signal art so if you don't know this the signal art is very cool library that actually allows you to create allows you to push messages from the server to the client using the WebSockets and of course if your web browser does not support WebSockets then it just fights it's using some whole box but but by default this tries to push your messages using the WebSockets so this is the communication from the server to the client so there's no need for the client to ask the server whether the some operations succeeded or not we just subscribe to the particular channel and wait for the notification that comes from this from the server yeah this is way more I would say yeah I will just create less less around trips - exactly if you can push the data push from the server using you know WebSockets on the server side push thingie is usually better because you push it when in action you push the data where actually data gets updated so when things happening you know about any push it at distances time and you know you reduce the traffic reduce the latency so just update your end-user about things that are going on with your request or with your data and you don't expect from your end user to you know just call you further the idle times you know by long polling and everything starts here in our rejected event switch or sorry suitcase or here in this event switch case we have this operation publisher either published rejected or published completed so if we take a look here very simple we're basically publishing to these two the message bus our you know our customers are sorry our custom message cast an event called either operation pending if this time long-running process operation completed or operation objected so everything starts in our patient service which publishes the message to the to the message bus and then we have this similar which is another service and it subscribes only at least in our scenario to these three types of messages right because it once know what's going on with our request so we subscribes to this free messages being sent by the operation service and if you take a look at Hitler's we have this single handler for all of these events and down below it all listens to these events and then it it uses invokes one of the free available methods which is all about sending the data to the end user using WebSockets so if you take a look at its hub service hub service basically takes this event and publishes to the user right so it take the user ID which is part of our of our operations planning event of course we could also pass you the mate a decoration context and grab the grab the correlation ID or the user ID from the context by all I have to do it we can just you know just use the event here which also contains user ID but the collation columns will be also fine and just publishes the new message to the user and with this the date like I never salute everybody if you're familiar with the signal art this is one thing that you actually do this clients group sent a sink and for each user would create a group so we create a unique group based on the user ID so if you have like multiple devices you get the data on all of your devices right that's that's how it how can do with this alright so let's start our similar service and yeah let's just start here to see the logs and just to see how it's how it's working we can browse a very very simple website so okay I can go to localhost 5007 and here we have our double double root and the directory called similar so I can go to slash cellular in this index.html and here is my awesome UI for the for a similar so you can see he was trying to connect to our D Sub Pop but it would be selected because there was an invalid token so I can open this signal sorry objects signal j s is the is the library for the JavaScript objects is our our application yeah yeah sorry the rider is not the word about the ACMA script latest version and some of these some of these keywords so this is why i can see there are some warnings so the only thing i need to do is to you know just update my access token because when my when i initialize the connection we have I basically pirate or a string and you know create a group for user so this is this very simple snarl here so let me let me take my token from here and just pass it to my up and okay here alright say if this is this is great that you actually created the UI for this messages but you forgot to create the input for the for the yeah I was thinking about it but it was yeah it was late hour and it was to come yeah just well okay maybe might can't go to expired already I'm not sure but let's see maybe ask for anyone yeah well I think I will okay Oh some typos so I'm connected right so here I started the connection invoke the initialize the sync on my hub and I got connected previously I got disconnected as you can see I'm subscribing everything to this free messages operation pending completed and rejected there will be sent to me for the WebSockets so let's try to invoke here the again the cast the create customer yeah like this so post and wait a moment whoa operation objected right I got an error my customer was already completed alright so maybe let's try to create a discount one request and again from the WebSockets I'm getting the data that something something else or I would look at it up for this one because we don't have the authentication here so this one will will not work but maybe create a product or I need to be an administrator having yeah so maybe let's try to do some something that will succeed I will create a new administrator account I will corrupt the admin token and I will also pass the admin token to the to the app yeah which is here like this refresh the page and let's try to create a new product I will just need to start the product service first set of products ok and yeah we are connected so I will try to create a new product which is here yeah ok service is up and running let's try I will send a request here is my operation I can I can always use this as a fallback right so the only reason we have this API is that if you can use WebSockets then you know you can just call it using get and this is a fallback all right completed right and my my internal object with this data let's try to create the same product again it will fail now I'll create a new product without name again so as you can see everything works as expected and this is how you can deal with this you know real-time updates if you want to do destruction of your messages in real time and update the end user what's going on you can do it like this using WebSockets using the service the subscribers messages and you know by implementing your own tracing of the messages as now I wouldn't say this is yeah it's it's a kind of more complicated on the front-end to actually keep tracking of this WebSocket um yeah and make sure that you are actually subscribed because this is the now you're taking the responsibility of message delivery from the server to the client because we're actually assuming that we actually publish this this operation updates using the WebSockets you need to be sure that your client actually subscribe to this and because without this we just cannot inform the end-user that we have their yeah and when you're losing signal there for example you have to remember about Fink called backplane right axis which is here and you can add it through d4 the builder of services because once you run your signal or service in as more of a single instance let's say two or ten instances and the request a comes is you have the user which is connected to the instance a you want to ensure this for example for your load balancer which you know moves it directs how the traffic goes that you are always connected to the same instance and that you're the dis communication doesn't get you know mixed up between different instances so we can use this backplane which by default uses readies and is probably the best solution I think yes and yeah and last thing as you can see we decided to to use our token to connect to the hub but in some scenarios maybe you don't even need token you could basically connect to your hop anyone you know and any user without without his credentials and just send the data to him without providing education it always depends okay so I think that was pretty much it and the last the last part is something some something about these helper methods right because I think in the last episode when we were talking about console and fab Europe and all HTTP clients you could see that for example within our eyes within our fabio client or if in our college to be client and we were using sorry Fabio and oh this one or this one we use the poly for the retries right for example here we have this policy for retry so if I if my gateway or my internal service for example a calls the service B tries to get the data and it doesn't respond because maybe it's temporary offline it will do a retry and maybe after three tries it will return to me the data is not available but if within this two or four or eight seconds the cell service will be up and running then it would get the data within the third read right right so we use the poly for this and we have the same same stuff for our event handlers and our message handlers general maybe maybe you can can you go to the police github maybe okay yeah because maybe not everyone is familiar with this library it's very cool for creating the adding the resiliency for your app I think it's now part of the.net foundation yes some of the some of the libraries use poly yeah so if you go down you'll see that yeah this is this project and you have plenty different policies that you can actually choose from so we have the timeout cache there is the circuit breaker yeah retries and so you can actually adjust your policy so I would say it's kind of action for dealing with failure depending on the situation so you can use let's say retries for synchronous communication or circuit breaker for the synchronous communication if something is if some services are down and as you see this is pretty pretty easy to use so we have this fluent API when you can define your policy and and just you know as easy to see you can see that we if we have this exception we want to kind of deal with this using this policy yeah so basically this is the the library added as bill said this is the poly was plugged into few places in that common package so the one thing that you so already we've already seen was was the Fabio when we wrapped the HTTP synchronous connection and this is also yeah in the past subscriber yeah we have also this we also use poly here as you can see if something so the idea behind our message handler is that if we get this dish of exception or something else that we treat as our domain or business logic exception then we know that okay you violated some invariants maybe you provided the void customer ID maybe you provided this control which is not unique or something that will never will never succeed no matter no matter how many tries you're going you'll you will retry you know all this operation so if it's some business or exception double exception we know that okay so your your comment your even will fail alright so we don't care about this one if this is this dish of exception we'll just publish this rejected event right away but if this is some other if it's some other exception type maybe your database is currently offline or another service is offline for these few seconds so if it's some maybe HTTP client exception alternatives which is enough which is not related to the domain or some stack overflow exception and will try to retry retry the operation within our message handler simply by using poly here as well and one thing to mention here is that with RabbitMQ and other message busses there is this concept of you know dead letter Q and poison Q and put some messages so in some scenarios you could I mean you can also do we were all a bit when there is something wrong with your message you can publish it to the same to the queue with this header which tells okay this is retry let's say first retry and I want to retry it in ten seconds and you can use your message but skew to handle retries and maybe if after you know five retries it can be handled successfully then you can publish your message which is poisoned or broken somehow to this dead letter Q and in this data you get all these messages that couldn't be processed right so we can grab the new one one one one by one and poses them because there was something wrong with them but once you have multiple instances of the same services and you are sharing the same kill names so the only one instance gets the message to process right which is what you want to do and this is also the question that a lot of people ask how is it happening it's happening by default right and right now if I was to if I was to if I'm using the same queue name for my forum instant for my and instances of the same service same service type and if I will publish this poison message to some retry message queue which turns out to be common for the same for the all of the instances then all the instances will try to you know somehow retry the message so we have to be careful with this with this with putting this poison message to the queues you have to be a work we have to be certain that you really have this unique queue names so that other other services will not get this message if you are using some routine keys and you are using this hash or star wildcards within your rabbitmq keys you really have to be take that into consideration because a it gets a little bit tricky here so just to be just to make it simple we decided to use an internal poll we try and find out things how you can use it and so let's get into our discoloration yeah yeah and here in the startup you can define as you can see we have a few more things there and I can type on error for example sorry for for our kid discount so I will say owner and here if I take a look at this one and it will give me the comment or the message it could move a comment in the scenario the comment and the exception that happened and it expects me to return the new rejected event so I could say in this scenario I want to return a new create discount rejected and I can pass here my comment customer ID and my exception maybe message and then some customer not found status code right and then I could get back to my handler not this one and I could omit this point here then the customer could be now right and there there will be a no reference exception and you know there will be retry number one to free and finally we could get this get this event the customer wasn't found because something happened and of course you can extend this you can extend this you know on error and this rich rice shop exception is that yeah right oh yeah also makes that would be makes more sense because in this case you would not retry oh yeah correct yeah it would be a business logic that makes sense so here I could just throw a new disc of exception with this with this code which is this one and maybe our message yeah and the message is first one and yeah so that's pretty much up to you which person do you prefer because for this is just some sort of enhancer some syntactic sugar yeah wrapper onto wrapper but if you prefer you know defining your messages Publishing explicitly in a command handler or the event handler itself this is fine if you if you like to keep everything in a some sort of fluent API then you can just use this another overload of the publish command or the sub sorry subscribe to command or event that's presented so this is pretty much up to you yeah this is just helper method so you don't have to you know just publish these events here on your own you could just throw an exception I Mabry return from this handler in some some wrong way some failed scenario and just it will just publish the this rejected event for you so doesn't have to do any try catch it here it will it will be just hammered by the layer above and yet that's pretty much it for this episode we'll discuss everything pretty much everything related to handling handling our requests tracking then in real time talk about the collection context what it is and how we can you know use the similar to publish these operation updates the real time to end user and yeah that's a yes I would say that for for with this episode you should know how actually to start your journey with the micro services and how this actually should look like more or less of course so how can you fetch data how can you write to your micro service how can you exchange messages between them and yeah the future episodes may be except the next one will be just or maybe not just but will be some additions that will make your development and and your micro services even more resilient available easier to track easier to do - I would say develop and the power yeah so this is I would say that this is the core of the of the architecture that actually allows you to start the whole fun yeah and starting with the next episode we'll get into the some of the more advanced topics and also some tooling related to services but a direct set for now you should have the basic understanding right now you you should you should know how to you know match these messages one with another for this correlation ID and by using this context you know how to get your services where they live using service discovery using console and Fabio so you could start building your first solutions with this architecture yeah so I think that's it for now and I will see you next week yeah see you next time yeah bye

Channel: DevMentors

Views: 7,967

Keywords: devmentors, dshop, microservices, correlation context, resiliency, polly, signalr, websockets, .net core

Id: rKmL2Onh4hg

Length: 79min 46sec (4786 seconds)

Published: Sun Jan 20 2019

Reddit Comments

[u/therealmodx]

nice! pls finish this tutorial as there are very less good and complete .net core vids out there.

[u/tihasz]

I wanna say thank you. Very good tutorials and topic to cover. I hope you have success in spreading this so more people can see it.

[u/headyyeti]

Are you/have you covered how you are handling HTTPS redirects through the gateway?

[u/p1-o2]

I appreciate the work you are doing so much.