Working with RabbitMQ in Golang for an Event-Driven Architecture

so in this video Let's explore how a message broker like RIT mq can make your microservices more reliable let's explore this first scenario here where we have the common case of an order service communicating with a payment service now this is communication could be JBC HTTP but what this means is that this is a synchronous communication if the user comes and makes a request to the orders basically he is creating an order uh we are going to communicate with the payment service which in turn is going to be synchronous so we are waiting for the response of the payment service to complete now what happens if the payment service is down and we want to scale this out for example uh we want to make more instances of the payment Service uh maybe the Ser the service is even dead allog together so how do we handle this case and this is where message Brokers like rabit mq come into play because they can help us make system more reliable and I'm going to show you how in this very quick introduction to rabit mq and message brokers in goang now unlike the previous example in this case here we have um a simple example where we have a Gateway service which is communicating with another service or it could be here also a payment Service uh but basically the way that we communicate is not direct so we are not uh doing the communication synchronous we are doing asynchronous which in this case we push a message to Revit mq and that Revit mq will decide how to best deliver to the consumer so we produce a message and we consume now everything here in between this is what we're going to explain in this video because this is a more complex topic but we are going to cover all of this into more detail into the full micros service project that is about to drop uh so stay tuned for that as well so just by looking at this diagram here we know that our services are decoupled so they operate independently so they are different servers so as you can see this is server one server two and this reduces the risk of cascading firor basically if the server two is on it's not going to impact the server one here with the Gateway service so this is one of the advantages of course of the micros service architecture not related to this but for example if the consumer uh in this case the order service has a lot of traffic a lot of traffic so for example uh it is in high demand what you can do for example is have these cues so these cues are going to hold the messages from the producer and basically what's going to happen is that they are going to consume from this queue instead of being a direct communication so if it's taking 10 seconds to produce uh to consume one message basically we have this queue of message in order so we can make sure the order will be delivered in order as well as the durability of the these messages so if the rabit mq dies these messages can be stored into diss so when we boot up everything again these services are going to read the messages back again I'm going to show you this in uh the later stage of the video when we're going to code all of this and all of this is related to the reliability and stability of our system as a whole of our micro service now other mechanisms that we can use at our disposal that we can do with ravit mq and other message Brokers really which the first one is rre so imagine this case that we send a message now this message is being delivered to the consumer but for some reason here in the consumer something fails uh instead of discarding the message we send this message back to the cube so this is a retry mechanism and it's going to be an exponential retry for example so it's going to take more time the next one and then it fails again and then it tries again and then there is Max of three retri for example this is something that we can Implement we're can implement this into the the full micros service course and basically at when reaching the maximum retries we can send these to a death letter CU which is going to be storing our messages and then we can do whatever we want with those messages so they are not lost and of course uh another really important feature of uh message Brokers like rmq or even Kafka is the way to send the messages that the producers send to the consumers so in this case in RIT mq we have the idea of the exchanges and the cues you can imagine in Kafka I'm not that familiar with kfka but there You have topics which is something similar to this and basically this consumer is only receiving the messages that he subscribed to so the cute that he's listening to so he doesn't receive the whole systemwide messages so by enabling the filtering you can have fine grain control of how the messages end up in your services in your system and all of these features are at the center of building event driven applications like this one that we have here so before we get coding this example I just want to show you the multiple types of exchanges that we have in RIT mq this is very important and I'm going to leave the link to this article here to this documentation below so you can see how the multiple types influence the way that the message is delivered I'm going to cover the direct and the out because these are the ones that you're going to be using so the direct exchange is the most simple because you have an exchange a direct one and what happens is that when you publish a message there it's going to deliver that message based on a routing key to that que so for example here on the example we have a producer which sends a message directly to this uh exchange and here we have two qes which their binding key is orange black and green so whenever a message comes with for example the black or the orange so if it goes with the orange key it's going to be redirected to this queue if not it's going to be directed to one of these two so this is what exchanges do they make sure to send the message to the right Quee and the F out for example here on this example it's when he wants to broadcast the same message to multiple cues so we want to make sure that for example these two consumers receive the same message and they're going to do different things with it ideally and this is basically it now let's get our hands into code and I'm going to show you how you can set up this project really quick so here is the install documentation if you want to install it yourself basically you can just do a rock Locker run and basically run this into a container I'm going to do something similar I have here Locker compulse just running an instance of rabbit mq3 and if you have any problems installing it there's one common problem which is that you're going to have some cash problem if you have that just delete the container or the volume and try again I had that problem before as well so the project is very simple this is the the Practical example that we the theoretical example that we saw before so here you have the orders and the payment service and all of these is inside a go workpace uh folder basically we have also the common package which is a set of reusable functions that you can use across all of the services go workspaces are very nice for this and here I have the rabbit mq uh module basically it says it has a a reusable function which creates a connection and then it also uh creates the exchanges so we are declaring the direct exchange which we're not going to use for now but I just want to show how you can do this and then the F out exchange with the order created event here this is the name for it and this is just here a string up here then we turn the channel and and close then here on the types I just have the order and the item struct very simple what we're going to do is that we're going to put up the payment and the order service and the order is going to send an event to the payment and the payment is going to generate a payment link and send it out to whomever wants basically uh this is what we're going to do so the order service again is very simple as well how it does it consumes the that function it does the connection then here we are declaring our CU so this is going to be the queue for the orders as you can see we have the order created event here uh I'm also setting to Che the auto delete the the durable so the messages are going to be durable even if the rabbit mq is down and the rest of the fs I'm setting them to false as well you can check them better into the documentation they explain it better than I do then we are marshalling just an order basically I'm going to send this has bites to the que and here this is how we publish our message uh I'm going to send to the Q do name so this is the name of the Q can just consume it like that and then I send the Marshall order into the and the content type is going to be Json basically then on the payment service we listen for this so if you go back here again we're doing a connection but here we are listening here again we also declare the queue because we don't know which service is going to put up first so I always do this on both uh ends then here we consume it so we get the channel we consume from this Cube and we get a bunch of messages which is a channel and here I have an infinite Channel just so we can have an infinite L here then inside of a go routine I'm looping over the messages that we receive I'm logging it I'm unmarshaling that message and then basically I just create a dummy payment link and here is the the code for that of course this is just an example uh and then but for some reason if the payment link returns an error we can handle it here and I wrote here that we can handle the retry logic here because in the full microservices course that is about to drop we can Implement with the retry with exponential by so this is going to be the place where you could do it and if the retry fails all of the times we can send that message to uh that letter que so this is pretty much it let me show you how to run all of this so here in the terminal I'm going to create a Docker compose up I'm going to boot up our my SQL um our rabit mq instance it's going to boot up and then um I'm going to go here and I'm going to start the order service and the payment service so so I'm going to start with the payment service because this is a listener so it's listening but when I run the the order servic is going to send a message so I just want to make sure that the payment is up before and if I do this I can see that we have poish an order and here we have received this order here and we have generated our link correctly so I send it multiple times we can see that we are receiving the message every time now if we go to the rabbit mqi because the image set up running on my Docker compost comes with the rabbit AQ management UI uh let make sure this is uh increased the font and as you can see this is the fold that we have uh created we have sent messages and they have been uh act so they have been acknowledged if for some reason messages are here dangling we can see it here we can see all of the stles here but very quickly I want to show you the zy as well but if you go back here what happens if we send a message and the payment service is not listening so I'm going to do that so I'm going to just publish a message I'm going to publish twice one more time and basically if we go back to our diagram we're going to see in a bit that these messages have not been delivered they are here on the Queue waiting has you can see we have three messages which are ready and they are not being delivered now if I go back to the payment service and I putot it up now what's going to happen is that it's going to read those messages from the queue all of them all of the the three messages and he going to process them at once when he can so has been rabit mq and message brokers in general because if you know how to work with rabit mq or cka you can work with any other rabbits uh with any other message broker that you want the idea behind them is the same so if you like this video you know what to do and I see you on the next one

Channel: Tiago

Views: 3,898

Keywords: golang, golang rabbitmq, rabbitmq in go

Id: 9F2NBTtUr-U

Length: 12min 23sec (743 seconds)

Published: Wed Apr 24 2024