Building Reactive Microservice with RabbitMQ and ASP.NET Core

hello everyone and welcome to dot net code central in today's video i am going to build a reactive micro service using revit mq now what is the need for a reactive micro service well reactive micro services are by default more resilient than a restful service because they are not directly connected to an http endpoint and calling to get messages they are listening to messages from a broker and if the message is available they are reacting to it so that's the fundamental difference between reactive micro services and using restful micro services now for today's agenda first i am going to walk to the problem statement or what is that we are solving secondly i'm going to use an existing restful micro service to publish an order message to an exchange and then i'm going to create a new reactive micro service using revit mq which will listen to the queue and react based on the message and then finally i will use another restful monolithic service which will publish message to the same exchange and the reactive micro service will react based on that message as well now if you have watched my previous videos on microservice i created a monolith as well as order microservice i'm just going to reuse those two as the publishers and create a new microservice for the listener now this was the architecture for the microservices that i discussed previously is that we had four micro services though i did not create all four i kept inventory and product part of a single monolith and order as its own micro service and then on the other side which can have a search micro service which is providing search feature and as i mentioned earlier in my previous video when i discussed about microservice is that the search or report this kind of micro services need data from all of the micro services meaning order product inventory because everything affects search or reports so today we are just going to build the search micro service and the search micro service is going to get data from orders as well as inventory and based on that it is going to change the data it owns so that it can return appropriate search results so let's start with that and for this project also i'm going to use the existing rabbitmq that i installed in a docker container now one thing i just wanted to mention is that if you remember in my previous video there was a lot of boilerplate code needed to create the exchange as well as cues and then talking to them so what i have done is i have encapsulated this logic and created a wrapper around it and i have published it in my github as well as i published it as a nuget package so if you want to use it feel free to use it from nuget or creating a fork from this repo if you want to contribute you can contribute in this repo as well i plan to maintain it and as the name suggests it's a very plain rabbit mq wrapper around the rabbitmq.client now there are other much superior framework out there like mass transit but mass transit is much more just beyond a rapid mq connector and it would need its own episode just to cover the features of mass transit and using it i intend to do that in one of my future videos so now let's get started so this is the e-commerce monolithic service that i had which had a inventory controller and a product controller and then i had this micro service for order which was just going to the order database and sending order back what we are going to do is we are going to update this micro service with few features first of all it doesn't have any swagger documentation so we'll just wet swagger to this service secondly we will update the post method of this micro service so that when the post happens it sends the data to the queue and once this is done what we are going to do is we are going to create the new search microservice which will be reactive to this data it will save the data in its own storage for search purposes so first let's start with adding the necessary nuget packages so first thing i'm going to add is the swashbuckle for swagger and i'm going to install the swashbuckle.asp.net core once it is installed i am going to go and add swagger here and then i am going to use the configuration and for the swagger documentation i'm going to keep the version as version one and for the open api info i'm going to provide the title that should be enough then i'm going to use the swagger ui and for the swagger ui also i'm going to use the config here now we're all set with swagger now next thing we want to do is we want to use the new nuget package i created for encapsulating the features of revit mq so i'm going to install this once this is installed i'm going to go to startup and i'm going to add the necessary services for the rapidmq nuget package i'm going to add the plain dot repeatmq namespace and for the url i'm going to pass the revit mq url here and then after the connection is created the next object i want to inject is the publisher for the exchange i'm going to provide name as report underscore exchange and for the exchange type i'm going to provide topic exchange so that we can listen to all type of reports the order report as well as the inventory report and for everything else i'm just going to keep default values so now my publisher is ready next thing what i'm going to do is i'm going to get into the constructor of the order controller and inject the ipublisher and then here for the post i'm going to change this to order detail and then i'm going to publish the message once i get an order detail and for routing key it's going to be report dot order and for attributes i'm just going to pass now so this is where as soon as the order is received it will be published to the exchange so that whoever is interested can pick up the order and react to it in our case it is going to be the report microservice now here apart from publishing the message to the broker it will also add the order to the order database now just like order detail provider we can have a order detail repo which is going to insert into the order database now i'm not going to do it just for the interest of time but here we are going to have order inserted into database now one thing here which is very important is we are doing two operations across two different data stores one is a queue and one is a database now if the first insert to database fails the service can clearly just return an error and we are good because there is no state mismatch but if the publishing to the queue fails then we have a mismatch between the data in the order system versus the data which is get into the report queue now to handle that we have to do some sort of retries and other mechanism now for this video i'm not going to cover it in my later video when i cover mass transit i am going to get into resiliency of message publishing because this is something which is provided out of box by mass transit so this service is ready now it's time to create the report microservice so for the report microservice also i am going to create asp.net core web application and i'm going to name it as report service and it's going to be an api because we will be also exposing api for the reports now the service is ready we have program.cs we're not going to do anything here the main will remain as is in the startup we're going to add the necessary references to the revit mq so for that first i'm going to go ahead and install the repeatmq package so i'm going to install plain dot reptimq once it is installed i am going to go back to startup and register the repdmq services for that i am just going to copy this part of the code and here connection provider would remain the same but instead of publisher i'm going to have a subscriber and for the subscriber we're going to provide the connection provider then we need the exchange exchange is going to be the same the next thing we have to provide is the queue the queue would be report queue next thing we need the routing key so for the routing key here we're going to have report dot star because we want any routing key associated to come here and next we are going to have exchange type of topic and then after that we are going to leave everything as is i'm going to change it so now the subscriber is ready next thing what we will do is we will create a background service now i have a video of background service i'm going to share the link of the video in the description and i'm going to name it as report data collector and report data collector is going to derive from i hosted service and in the constructor i'm going to expect the i subscriber and then here in this place we're going to get the call back so let me just do a subscribe and the subscribes needs a func which will give back the string and the i dictionary of error values and return a boolean so i'm going to create a private function process process message and then here i can create private for the time being i'm just going to return true so that the message is not going back to the queue and i'm doing that because if we return true in the subscriber it is going to just analyze the message so for simplicity right now i'm just returning true and here we'll say done task dot completed task and same thing we are going to do for the stop async method just return a completed task now here after we get the message is where we are going to do all the processing now for that what i'm going to do is instead of creating a database i'm just going to create a in-memory class to save the data for the report so that we can access it from the controller so i'm going to create a new class i'm going to name it as memory report storage and it's going to have add and and for the timing i'm just going to keep a simple report object i'll create a new type called report and for the report i'm just going to have few properties and for the properties i'm going to use from the e-commerce if we're dealing with product then the product had name description type and id so i'm just going to keep product name and then i'm going to keep count that's all so based on a product and order i'm going to derive the count from the two services now go back here what i'm going to do is i'm going to create a in-memory list so we're going to add the reports and here we're just going to return reports that's all so the memory storage provider is ready i'm just going to extract an interface and then we'll go back to the report data collector and here we'll also take the memory report storage and then every time we get a call back we're going to deserialize the message and from the message we're going to figure out what it is so ideally we should be getting the type from the header and based on that we should be deserializing but right now just for simplicity i'm just going to check in the message if message dot contains and i'm going to look for product and else i'm just going to think that this is coming from order so for the order we're going to have order details so i'm just going to copy paste this object and i'm going to create a new order detail here and i can keep the name as order doesn't have to be ordered detail okay so here if it is not product i am expecting it is order again this is i'm doing just for simplicity we should not do this we should get the type from header so that we can figure out what kind of dc realization we have to do from the object so apart from message header also should be part of the contract we should say what type of message it is now here we can do i'll just install newtonsoft.json and if i get the order what i'm going to do is so order is some sort of change in the inventory so i'm going to expect that the product already exists and i'm just going to add on negate and if the product doesn't exist i'm just going to consider that the product has a default value of 100 for the time being and negate that ideally product will always be there and if not we have to handle some sort of scenario so i'm going to check if it exists first so i'm going to say if the product name is equal to order dot name and if this is true then the order exists and then all i'm going to do is i'm going to get the fast with this name dot minus equal to the order dot quantity again this is i'm not checking doing any sort of validation but we should be doing validation to figure out what is the quantity whether it's already 0 and things like that but for simplicity i'm keeping it as is and if it does not exist then what i'm doing is i'm just adding a new order so i'm going to say default equal to 100 let me put it as a constant so default quantity i'm putting it as constant so we're going to say memory storage dot add and here we're going to create a new report and for the product name it's going to be the order dot name and for the count it is going to be default quantity minus order dot quantity that's all we are going to do so right now we are not handling the product scenario which will handle later so now this is now the hosted service is ready it is time to go and update our startup with the memory report storage as well as the hosted service and this is going to be a singleton and then services dot add hosted service report data collector so that's it now my report data collector is all ready so what i'm going to do is i'll also get rid of this weather forecast as well as the water forecast controller and the next thing i'm going to do is i'm going to add a new controller and to create an api controller with redride and i'm going to name it as report controller first i'm going to create the constructor in the constructor i'm going to take the imemory storage and then i'm just going to have the get method implemented right now it's going to have memory storage report dot get and it's going to return an innumerable off report that's all i'm going to do for the time being let's run this project now here let's go to the properties and let's change some of the properties because we want to go to api slash reports and also we want it to run as report service and here instead of 5001 we're going to have 5011 and 5010. let's go ahead and run this service now i have to change the subscriber also to singleton given my poster service is a singleton service now let's run the application this is up and running right now we are not going to get any data because there is nothing but if we go to the revit mq we can see the report exchange is created this is a topic based exchange it is connected to report queue and it's listening to report dot stock now let's go and start the order service order service is running at 5001. if we go to swagger i forgot to add the app.uswagger that is needed now let me run the order service and if i go to swagger should be able to see the swagger and now let me do a post of the order so you can try it out here user for product let's say milk and quantity is 2. so let's execute this once we execute we got a 200 success back and if we go to the rabbitmq we can see the message is delivered to the exchange and then queue now if i run this i can see that the message is picked up by the report service and the product milk and count is 98 because we submitted two so two is reduced from 100 so 98. so this is all working fine now it's time to go back and update the monolithic service to send a message also for a new product so let's do that this is the product service so first i'm going to add the plain dot reptimq nuget package i'm going to install it and then after that i'm going to follow the same code as the order service in terms of registration so i'm going to go to startup and i'm going to copy these two and here also in startup i'm going to paste this and then add the namespaces once this is done i'm going to go to the product controller and the product controller i'm going to inject the ipublisher i'm going to declare publisher as a private variable now in the post i'm going to take product and then i'm going to do publisher dot publish and for the message codeword dot serialize object and i'm going to pass the product and then next thing i'm going to do is i'm going to provide the routing key and the routing key is going to be report dot and this time it is product and then for message still going to pass null for simplicity for the header so this is now publishing the product and then in the startup i'm going to also add swagger because it'll be easier to publish the message using swagger and for swagger also i'm just going to copy paste this part and here instead of order service i'm going to just name it as ecom service and then here also i'm just copy paste both of this line com service so now let's run the e-commerce service i need to change the port so let me change the port to something else let's run now now i'm going to do a sweater and then we're going to make a post to the product but in the report service we have not handled the product scenario so let's go back to the report controller and now first let's copy the product class and let's create a new class now in some cases these models these object models might be exposed by a client library from the individual services now if that is done you can just use those libraries as is or you can just create a version of your own sometimes you might choose to use a dynamic instead so that you can just extract the property if you want and not worry about the entire object it's totally up to you so when we get product we're going to use and we can pretty much copy this and make some modifications so instead of order it's going to be product and this one is also going to be product and then we can say product dot so we cannot use the string comparison product unfortunately because product doesn't have a product in the name so i'm going to change this implementation and here instead of name i'm just going to say product name so that the product name is going to be available and the product match will work and here we are never doing uh exist if it exists then that means probably the product is deleted but i'm not going to consider that case right now so if it exists it's just going to return true and if not then i'm going to create a new product so product.name and quantity is going to be the default quantity of 100. so that's what we're going to do let me go back start the e-commerce service again so this is up and running let's go back to the report service and start the report service also and hopefully there will not be any error but let's go back and use the swagger and let's post a new product and let's give an idea of one the product name as cheese description lazy me and type is dirty product so let's now execute this we execute we get a 200 back here let's get back to the report service and run a report and we see product name is null count has 100 so something got so i have to make the product name as product name as well because we changed it product name and here also product name okay so now let's run the application again the report service and the report service is ready and listening if i go back to api and reports there is nothing because i had in-memory data structure which got emptied out when i restarted now from here what i'm going to do is i'm going to execute a new product so i'm going to try to add product id of one cheese american cheese and the type is daddy now if i execute this this product should be added to the reports so if i go to the reports and run i see product cheese and count is 100 because that's the default count now if i go to the order and now this time if i execute an order of cheese and if i execute five and i go back to the report after executing this i should see the product number now is 95 and that's what i see so as you can see that this is the report service is completely reactive in nature and all it is doing is it is trying to get the product as well as the updated orders and from that it is figuring out what is the latest inventory and based on that it is going to provide the search so this is how we can build reactive micro services and as you can see it is extremely decoupled because it is not making any http call it is just listening reactively based on the data and whenever there is an event happens on the other side it is reacting to it and updating its storage and keeping its own copy of the storage so that's all i wanted to cover today in the upcoming video i tried to cover the mass transit and show how we can use mass transit with revit mq for building reactive services and how we can build more resilient service the issue that i discussed when it comes to order controller where we are doing multiple transaction how do we manage this distributor transaction and make sure that it is handled properly we'll discuss that when we discussed mass transit if you like this video please give me a thumbs up and if you are new to my channel if you think you are getting value out of my channel please subscribe to my channel thanks so much for watching this video

Channel: DotNet Core Central

Views: 12,929

Rating: 4.8222222 out of 5

Keywords: reactive microservice, reactive microservice rabbitmq, reactive microservice rabbitmq asp.net core, reactive microservice asp.net core

Id: rUKqaO8IQCE

Length: 31min 58sec (1918 seconds)

Published: Sun Oct 04 2020