Software Architecture in Golang: Claim Check / Reference Based Messaging Pattern

hello my name is mario welcome to another software architectural video in today's episode i'm sharing with you the claim check messaging pattern what is the claim check pattern the claim check pattern also known as reference based messaging is a pattern that allows handling large messages it was originally described in the book enterprise integration patterns written by gregor hope and bobby wolf in 2003 the claim check pattern handles large messages by storing their entire payload into an external data store creating a reference or claim check to the store payload and then sending that claim check to the receiver this way the message to be sent is not the actual content of that large message but rather a unique identifier that points to the large message stored in the persistent data store those large messages may contain anything such as images sound files documents literally any kind of binary data of any arbitrary size the claim check pattern is typically used in asynchronous communications where a message bus exists and is in charge of orchestrating the communication between multiple services it's beneficial in this case because there is already a predefined limit for the messages that cannot be exceeded in practical terms the most common example you can think of is html where image tags include a claim check to the final image in the form of a url that in the end is rendered by the browser asynchronously as usual the link to the code is in the description of this video so feel free to check it out i have a readme that indicates how you can run kafka locally for this example which is the message bus that i'm going to be using i have two binaries implemented already one called producer and the other one called consumer producer will be producing a matches message and consumer will be consuming a message the code looks like this in the case of the consumer which we have the configuration for the consumer as well as a way to initially initialize the consumer and then what we have to use next will be the logic for receiving the message that represents the claim check in this case in the case of the producer is a little bit similar we are going to be connecting to the kafka producer the kafka message bus in this case and then we're going to be depending on how we're going to be sending the message to the consumers we are going to be either creating a claim check or we're going to be i will demonstrate why this fails using the default configuration in kafka that does not support messages higher than one megabyte which is the way to produce the message is right here and the way to send dimension message will be implemented in this function that returns a slice of bytes so first of all how can we send a message to kafka for doing that like i said i have the implementation of this function in this function called new message new message in this particular example is reading a file locally and sending that to the kafka server in this case so if we run this you will notice that i have my consumer right here so i do a go run.main.go and you will see that there is an error because the file is too large so like i was saying in the beginning when we're trying to send messages that are too large we need to use this pattern that consists of creating a reference id that we can then use for sending that to the consumers that want to use our messages so if we go back to the server the presentation i actually cheated a little bit and if you were paying attention to the lines up below this you will notice that is actually uh just returning the url of one of my videos that i uploaded previously that consists of using event streaming using kafka so the idea is that when you're using this pattern you're going to be loading that file to persistent datastore in this case i'm using youtube but you could be using s3 or maybe azure developer storage in that case and then get that link which in this case will be this url that represents the unique idf identifier the reference id and then send that to our consumer let's see how this runs when we're using this pattern in particular so if we go back again to the producer we run it it will set it produce the value which fine and if we run the client or the consumer in this case you will notice that now everything is working as expected which is uh right here now pretty straightforward really simple let's jump into the final conclusions and that's it this is a claim check pattern or reference based messaging it makes sense when we're dealing with messages that are large we're using asynchronous communication but not all the time you can also use for sending smaller messages that don't necessarily have to include all the data that the consumers need let the consumers maybe refer to the afterwards if they need to hopefully this use is useful for you i will talk to you next time take care and stay safe see

Channel: Mario Carrion

Views: 1,740

Keywords: golang, go lang, golang tutorial, go lang tutorial, golang beginners, golang for beginners, learn golang, golang software architecture, claim check pattern, reference based messaging pattern, enterprise integration patterns, software architecture patterns, microservices messaging

Id: hLEB5pRax9A

Length: 5min 22sec (322 seconds)

Published: Fri Jul 08 2022