Build Clean Messaging in .NET with MassTransit

hello everybody I'm Nick and in this video I'm going to show you how you can get started with mass transit in.net this has been one of my most requested videos and that's for very good reason must Transit is an excellent library and pretty much a household name nowadays for building applications that do any sort of distributed messaging it is very feature Rich very flexible well documented and portable and in this video I'm going to show you how you can get started with it and use it with a couple of providers now because this is about messaging I'm going to be using rabbitmq and Amazon sqs in this video to show you some of the features and because I'm going to be using Amazon sqs this video is sponsored by AWS so if you want to claim some free credit check the link in the description to see how you can do that and if you want to learn more about AWS for c-sharp developers I have a free course on domcrane.com so check that out it's free to sign up you have to keep forever that's it if you liked our content and you want to see more make sure you subscribe and for more training check out my courses on dormtrain.com all right let me see what I have here now one of the coolest things about mass transit is that you can actually install templates created by mass transit themselves you can do that by using the dotnet new command and then simply saying mass transit those templates and if you were to do that this will install those templates however I want to take it easy and explain you everything about mass transit and how it works so I'm not going to use any of the templates and I think you're going to get a better understanding by this approach so I'm going to start by just creating a new project and it's going to be a simply console application and I'm going to call that mass transit demo so very simple variable Bones the reason why I do that is because I want to introduce you to the concept of messaging and the idea is very simple you generally have two basic concepts when it comes to messaging this can be more but these are the basic ones and the first one is that someone a sender wants to asynchronously tell something to someone who's listening for something so if service a or maybe Class A wants to tell something to service b or even Class B this can be either in app or outside the application then that a part will send something to that receiver and that receiver will receive it and do something about it but you generally have a one-to-one relationship there you also have the pub sub scenario where one publisher is letting everyone interested know that something happened so for example if a user was created in your system you might say hey user created and just broadcast this publish this message and not necessarily know what happens down the pipeline who's going to listen to it and what they're going to do with this that's not the important thing the important thing is that you let the message out so these are the two general approaches you have with what I'm going to show you today you can do both but it's up to you to choose what works better for your application and if you want to know when to choose one or the other that AWS course I have on dome training does explain quite a bit there so what I'm going to do is turn this console application to where the web app and I'm going to do that by saying sdk.web that behind the scenes will go and add some nuget packages but the experience I have in the front end is the same and all I'm going to say is that I'm gonna turn this into a obligation Builder which I have to spell properly here and then I'm going to create the app by saying Builder dot build and then app.run and that's it that will build a basic app and what I want to have in this application is a publisher class and a consumer class something will happen and I'm going to publish broadcast that message and something in this app will listen for that message now that message will be represented as a record so I'm going to say public record ping here and I'm going to say something like button here because I'm going to broadcast this message based on the button press now to wire that up I'm going to do the following I'm going to say ping publisher over here and this publisher will be a background service over here that's a DOT net thing I haven't added any nuget packets at this point and all I'm going to do is I'm going to inject here and an i logger which I have because I'm using a web application so I'm going to inject that and I'm going to log a message every time I press a button now I'm not going to bore you with implementation of this because it doesn't really matter for what I'm showing so I'm just going to turn this into an async method and I'm going to put the implementation here all this really says is that while this application has been canceled keep looping around and checking if any button other than Escape has been pressed and if it has just log it in the console so to register that over here I'm going to say to builder.services.ad hosted service and then use the Ping publisher as a hosted service I'm going to just run this application just like that to show you what's happening and then what's going to happen is every time I press a button you're going to see that there is something triggered and you see it in the console so we have a way to publish something on a button press which is what we're going to use to publish that message now to implement that I need to add mass transit into the project so I'm going to search for mass transit and that is it that's the main package it has something like 50 million downloads it's a very popular packet is very widely used it's a lovely library now configuring it is actually surprisingly easy to get started with I think it could be easier that's a bit of an issue I have with the way it's being registered but I think it's good enough so all I'm going to say is builder.services.ad mass transit and then I'm going to use the configurator and then to start with all I need to do here is say x dot using in memory and I'm going to use the overload here and I'm going to say configuration configure endpoints and then pass down the context and that's the most basic configuration needed without a consumer but we want to be able to consume those pings and to do that I'm going to make a ping consumer class and that class will be of type I consumer and then I'm going to pass down the object I want to listen for which is that ping object I'm going to implement missing members and that basically looks like a mediator Handler because fundamentally it will work that same way with the in-memory version and these mass transit consumers are configured by default to support the eye so I can simply say private read only I logger ping console tumor over here and then just grab a logger and do whatever I want what I want to do here is just print what button was pressed so doing something like this is enough now remember this is on the consumer sites but because I want to have that messaging functionality I need to go to the publisher and actually wire it up to publish that message so what I'm going to do here is a private read-only ibus and that ibus is coming from Mass Transit and we can use the publish method to publish that information so I'm going to say await bus dot publish and I'm going to publish a new ping object which is my contract over here and I'm just going to say take this button to string and just comment this one out we don't want to log it here and that is it now this will work in the same fashion as mediator would within process messaging now there's two things I need to do before I can actually run this application the first one is to say add consumers over here in the add Mass transfer configuration and that's because I need to let mass transit know that there are somewhere somewhere ready to consume messages so by saying don't add consumers I can provide the type of an assembly marker in this case the program.cs and then say assemble it all this really says is that hey find anything that implements the iconsumer interface in the assembly that contains the type program.cs and that is it the other thing I need to do is actually move that contract into its dedicated file and then once that's moved it needs to have a namespace that is a requirement so I have to add it here but once I do everything I can actually just run this and now if I go to the console I start pressing buttons as you can see these now are coming from the consumer not from the publisher so messages are being published and then consumed in here and that is the basic logic of mass transit publishing you publish at some point you consume at some other point and that is it and then you have this middleman which is your contract which contains all the information you want to carry from one point or the other now we're going to take all that logic and move it to a real API doing real work and start publishing messages and consuming them using a real queue so what I have here is I just added this movies API which I'm going to show you very quickly what it contains it uses an in-memory database just to simplify things in case you want to grab the code from the description and do something with it but we run this API now and what I can do now is go to password and say hey create this new movie called Nick the Cricket came out in 2023 and it's a comedy and an action movie so it's going to go ahead and just create that and I can use things like get endpoints here to list all movies get a specific movie and so on I can update the movie I can say that you know what this is not actually an action movie and a comedy it is just a comedy so I can update the service and as you can see it has been updated and then I can go ahead and delete it now it can be very useful to actually also publish messages into your system that every time a movie is create deleted or updated someone can listen to that and act on it for example if a movie was deleted maybe you want to delete all the ratings from that movie there's many reasons why you might want to do that so in our case what I'm going to do is wire up mass transit in this application and publish a message every time an action is taken so let's go ahead and add mass transit here now to wire up mass transit here I could just install the mass transit package but because I want to use a real Q system in this case Amazon sqs what I'm going to do instead is use the Amazon sqs transport so must Transit has a common interface the same I consumer the same ibus or the same I publisher and based on those abstractions you can actually have different transports so you can have Amazon sqs you can have Azure service bus you can even have S3 Kafka grpc there's so many things you can have we're gonna start here with Amazon sqs so I'm going to just install this mass transit package with that transport and I'm going to configure it in the program.cs now as a publisher all I need to say is builder.services.ad mass transit over here same idea as before and now I could say using in memory but I also have this extra transport of the Amazon sqs here so now I can use this same overload as I had before in the in memory but now as you can see it has Amazon sqs specific types which will allow me to configure sqs specific things in my case I just need two things the first thing I want to say is host over here and save this is in the London location in AWS and then I can simply ignore all the other settings and that's because my machine is already authorized against AWS so I don't need any connection strings any access Keys any nothing you can however if you want to use the overload over here to have things like credentials secret keys if you need them and so on but I don't need them so I'm Gonna Roll them back and then the other thing we have is the same configure endpoints method that we had before and that is it now it's also very common to use Kebab case for endpoint name formatters so this also sort of comes in many applications and we're going to use it here as well ultimately you use the naming conventions you want but in here I'm just going to use Kebab case all right so with all that very simple configuration all I'm going to do now is go to the movie service and configure my publisher now previously I used the I bus interface and I'm going to do the same here but there's a bit of a caveat if this service was co-opped and it's coming all the way down from the controller which is also a sculpt construct scope service then you would actually use the publish or the I publish endpoint interface that's because the ipublish interface is also scoped and you can maintain that scope in your publishing in this case this movie service is a Singleton so it doesn't really matter so with that now injected over here all I need to do is go to the Bottom of My Method after the actual work is done after the movie was created in this in-memory database and say oh weight boss dot publish now I need to publish something and I don't want to publish my internal domain model here what I want to publish is actually a contract a contract that my consumer can use on their end as well and stay up to date with me and do all the processing so for example my movie Object here has some logic in it on how to generate that slug which I have here in the movie if you remember we generate a URL friendly slug if you want to just use it in your url to retrieve a movie now that's not something I want to link to my consumer so what I want to do instead is actually create a class Library package that will contain my contracts which is going to be just dtos themselves and the consumer can use those to build its consumers with so all I'm going to do is Click class library and say messaging contracts over here and that's it and I'm going to add a reference to that package so over here messaging contracts and that's it now what will I have in here delete this class one useless thing and then simply create one record per action that I want to publish a message for so in this case movie created movie deleted movie updated all of them contain all the movie details and as those objects change these parameters can change as well remember however that these contracts are contracts that are versioned so you can't just break the contract because you're gonna break the consumer so you have to communicate with the consumer that you're going to start making those changes and then coordinate them so with that in place I'm gonna go back here and say publish and what I want to publish is a message which is generated by that movie Object and then map it to a created object here so map it to that detail with a mapper that I have over here just a manual mapper that's all you need and then just publish it here now a bit of another caveat here in a real application it's very likely you will not do fast the insert the database and then publish the message because that is not an atomic operation and it's very hard to roll back what happens if for example the ad in the database succeeds but publishing to the message Q fails there are ways to solve this and they can be a bit complicated and I Do cover them in that course link is in the description for what we're showing now this is more than enough then I just have to do the same for update over here and in the end for delete and that is it so now we have the publisher side of things but we also need the consumer to actually consume those messages and for that I'm going to create a similar application after that first one so just a simple console application and call it movies consumer and it's going to have the same logic so I'm going to turn this into a web application so I'm going to go here and say dot web and I'm going to add mass transit Amazon sqs over here just simple as that now I really the reference to that contracts package because I need to grab access to them to build my consumers but other than that the logic will be very similar to what you saw before so I'm just going to create a new directory called consumers over here and I'm going to create one consumer per message type so the first one is that movie created consumer which only listens to that movie created message and you can do whatever you want in this consume method now I'm going to add the two missing ones and they're going to have the exact same consumption logic which is just hey logout you consume something here you go everything is in place now and now I'm gonna start to write that program.cs now a lot of what we saw before is actually the same we still have this add mass transit logic and we're still going to have this using Amazon sqs logic but I'm gonna have a more full implementation here so we still have this Kebab case and point formatter but we also have an in-memory Saga repository provider which if you want to know more about sagas leave a comment down below I'm very happy to talk about them they're a bit more of an advanced concept and then we add those consumers in the same way we add Saga State machines we add sagas we add activities and then we add that Amazon sqs transport over here and that is it but before I do that I want to show you something very cool in the AWS UI if I go to the home page and I search for sqs and I try to see all of my cues in sqs as you're gonna see I have nothing there no accusing my system no topics no nothing but now if I go here and I say hey go ahead and just run this movie's consumer Watch What Happens the first thing that happens when the application starts is that mass transit acknowledges that what it's looking for what it needs the cues it needs don't exist and it goes ahead and creates them this of course is configurable but it's very cool that it can actually detect what it needs and build the resources it needs to function now in some cases you'd actually want to have all that scripted out with infrastructure as code or IAC and you actually can't configure it to disable this functionality and point to the resources you want but it's very cool that if you just want to get started it's really cool that mass transit can just hit the ground running build everything it needs and you can start processing messages so my consumer is running I'm going to go back to the Amazon portal I'm going to just refresh here and as you can see my queues per message have been created and now what I'm going to do is actually stop the consumer first and I'm going to run the API so just run the same exact API and go ahead and create a movie so I'm going to create the same classic Nick the Greek let's go created so the message has been published but the consumer is not running so nothing can actually listen for that message to do something with it this means that if we go back and refresh we're gonna see eventually one message to be available here you go in the movie created and if I go ahead and I pull for that message I'm gonna say send and receive and then simply say sorry not send Paul for messages you're gonna see that message published here with a bunch of options here you go and as you can see tons of details about the message we can actually see it comes from SNS not directly sqs we're going to show that in a second then you can see the body of the message with all the details we have and then you can see some details that are attached to that message like the machine name the process name you have the process ID and so on all things you can actually access if you want during consumption so now with that in place I'm gonna go ahead and just run the consumer that message is held on the cloud and now they run it as you can see movie created message has been processed and I can just take it and you can see all the values that are properly serialized in that record so I'm gonna go back and just create another movie and another movie and then maybe just update a movie as well here you go and as you can see now all of these messages are coming in real time going to the topic from the topic to the queue and then from the queue to this consumer service very very easily but here's why this gets very interesting and very exciting and it makes Mastermind it's a very interesting valuable position for those who are not working with a simple messaging technology or a single cloud provider let's say that suddenly you have to stop using Amazon sqs or you want to move into rabbitmq maybe you run it in Prem maybe for some reason you want to move away from one transport to another one what can you do and how hard is this with mass transit well it's actually very very simple all you need to change is this piece of code that's it nothing else so what I'm going to do is just remove the that Amazon sqs package and I'm going to add the rabbit mq over here and that's it now I'm going to comment out this using Amazon sqs and what I'll do is you guessed it using rabbitmq and that is it all of my another code in the project and the consumer can stay exactly the same because the abstraction stays the same what changes is how all the wiring will happen behind the scenes but you don't have to know about this this is a responsibility of mass transit so you're going to have the same sort of context and configuration functionality here and you're still going to have the same configure endpoints call but we also have to configure the host here so to do that all I have to say is configure it to localhost because I will run it locally and then the username is guest and the password is guest those are the defaults for rather than Queue at least on Docker and I'm going to take the exact same code and then update the consumer to also do the same thing so remove AWS sqs go here comment this out and paste this code and that is it so what I'm going to do now now is actually use Docker to run rapidmq locally and I'm going to go ahead and just run it over here can take a second for it to spin up but it should be up any second now and sorry for how bright this is but Robert and Q also has sort of an interface you can see all your queues and your messages and exchanges and what I'm going to do is go to the queues and show you I have no cues in this system but what I'm going to do is go back and just run the consumer again the customer code is exactly the same nothing changed but this using Repton queue so I'm just going to say run this and that is it same logical kick and boss started and created those necessary cues so if I go back to the UI now you can see all the queues that are empty I don't have any messages in them but if I go back and I stop the consumer and I run the API with the exact same code but this piece of code changed here then I can go ahead and just say create a new movie and it will do it and if I go back to the UI you're gonna see that we had some activity here in this movie create we had one published message which is ready to be consumed assumed and if anything I can actually get the message here and show you what it has that's the message I just published using mass transit and now of course consumption is exactly the same as before I'm just gonna run my consumer the messages there and is now consumed how cool is that the exact same code able to process multiple types of Technologies in your application just like that if you want to move away you're not necessarily locked in to that specific provider I think mass transit is an excellent Library there's tons of places I can actually take this and I can even make a series if you're interested if you want to know more about sagas resilience and so many other things so if you want to know more about them leave a comment down below I would love to make these videos well that's all I have for you for this video thank you very much for watching and as always keep coding

Channel: Nick Chapsas

Views: 58,870

Keywords: Elfocrash, elfo, coding, .netcore, dot net, core, C#, how to code, tutorial, development, software engineering, microsoft, microsoft mvp, .net core, nick chapsas, chapsas, dotnet, .net, .net 7, aws, aws .net, aws on .net, .net on aws, masstransit, .net mass transit, mass transit, c# masstransit, .net masstransit, rabbitmq

Id: 4FFYefcx4Bg

Length: 21min 44sec (1304 seconds)

Published: Thu May 11 2023