Integrate Spring Boot application with AWS SNS/SQS using localstack

Hello friends in this video we'll see how to integrate spring boot application with AWS sqs and SNS we'll use something called as local stack and we'll use springcloud AWS 3.1.0 so let's get started so this is the official documentation page of local stack a lot of times what happens is in the organization the infrastructure teams are busy building the infrastructure for us but at the same time parall we can start coding our application and while the infrastructure is getting built we need something to test the integration of the app with the AWS Services that's why local stack comes in handy so as I said this is the documentation page here is a list of all AWS services that local stack supports it's a big big list here so it covers majority of the AWS services but in this video we'll be focusing on SNS and sqs which is simple notification service and simple Q service now before we dive into integration let's see how to install local stack so in the getting started section we'll go to installation here you'll see step-by-step instructions for different kind of machines like mac Linux windows and others but for this video we'll specifically Target Docker so we'll see how to install local stack using Docker desktop which I've already installed on my machine so if I click here so here's the step for running the local stack on Docker note that there's a pro version as well which has a different image and this is a paid version but we'll stick to the community version for this uh demo so let me copy this command and run it on my Docker desktop just to confirm nothing is running currently I'll do a Docker PS so there's no container running and now let me paste the command but as you note here there's no tag specified for the image so I'll pick the latest tag which is available as of the recording of this video which is 3.1.0 as you see the local stack is coming up now if you notice there's no shell given to us right so we can't type in any command here as such so let me do one thing let me kill this and let me start the docker in a detached mode so that we can get into the container and run the AWS local commands so all I'll do is add an extra flag here which is- D which will prompt me a command prompt as you see we are at a command prompt now so if I do Docker PS it's showing me the container ID here so let me get into the Container using Docker exact dasit and then bash and the command to [Music] run so now we are in inside the local stack environment right so let's go back to the documentation and check the user guide for the sqs and SNS so here they have given bunch of example commands like for example for sqs how do we create a queue right so for example let's run this so let me run this command AWS local sqs create Q and The Q name is local stack - Q you see it returned a q URL which means it created the que and returned the URL of the Q now we can try to list the cues and see whether we get the response of the newly created [Music] queue so as you see in the output it's giving us the newly created Q URL as well so and now let's say we want to delete the queue right so instead of create Q I'll say delete [Music] Q something went wrong okay for the delete Q it expects the Q URL and not the name so what we'll do is we'll provide the URL which it printed in the output above and there you go it deleted so if we do once again list cues it should not show any Q now there you go so that's how simple using a local stack is now let's see how to integrate it with a spring boot application using springcloud AWS so here is the official documentation page for springcloud AWS if we click on sqs it will take us to the sqs integration page now here is the maven dependency that we need to add which is spring Cloud AWS starter sqs let's go to our intellig and check out the maven p .xml so here I'm using 3.2.2 spring Boot and then I'm using Java version 17 this is the standard spring boot starter and spring boot starter web dependency now here's the sqs uh spring Cloud AWS starter sqs dependency where the group ID is IO AWS spring Cloud as shown in the documentation here now there's similar dependency for SNS as well uh which is spring Cloud aw starter SNS so I've added both these uh let's check uh the source code now go to SRC main Java so local stack demo application is the standard runnable application for spring boot now I have a controller and a model here now in order to publish messages U instead of using a string message I wanted to show you a pojo where we can you know publish a message which is uh poo of type employee right which has a ID for first name last name email ID Etc and there's a controller which basically exposes couple of services um so here's the demo controller uh one service is exposed at the URL send sqs message there's another one which publishes uh SNS notification so we'll cover both these scenarios and again if we go back to the documentation here how do we listen right how do we listen to the sqsq so it's really simple all you need to have a method defined and annotate it with at sqs listener with the name of the queue that you want to listen to and the type of the message so in this example here they have given a basic string message but in our example so here I have a method called listen with which is annotated at SQL sqs listener and the name of the queue that I'm listening to is my q and the type of message that will be published here is of type employee now as soon as this listener listens to the queue and sees that there's a new message published it will log an info and print out what that message was now this was about listening to the sqsq but how do we publish a message right how do we send a message to the sqsq so let's take a look at the send message here so first I'm creating an employee object here it's a random employee with a random ID random name random email ID then I'm using sqs template so springcloud AWS provides this s sqs template which can be used to send messages to the sqsq and there's a similar SNS template for integrating with SNS Service as well so we'll take a look at both of these so in the send message once we have the employee object all we need to do is sqs template. send and we specify which Q in this case it's my- q and then we specify what payload we want to send so in this case it's the employee object itself and then it's simply returning saying message sent to sqs which is the employee EMP object now similar concept for the SNS as well here also we are creating the employee object first and then using SNS template we send a notification to the topic named my topic and then we send send the notification of type employee and then we return return a simple message saying we published to the SNS topic with the employee object now the way we are going to set up the SNS topic is we'll have a subscriber which is nothing but an sqsq this is called as a fanout model where basically if a application publishes a notification to SNS SNS will internally publish or send that message to an sqsq which is a subscriber so imagine there are 10 different applications which want to listen to a notification which your application sends right so your app doesn't need to you know keep on sending 10 different messages to 10 different cues so you can set up one particular SNS topic and you can have all those 10 depend depent application subscribe to these SNS uh using sqs Qs now as soon as your application publishes a message to SNS SNS will internally send a message to all those sqs Q's which are subscribed and then all those 10 applications can individually listen to those sqsq and process it according to the logic they want so yeah uh before we start let's set up these cues and SNS Topics in our local stack so I'm back to the terminal Let Me Clear these things out now this source code is committed to GitHub I've included the link in the video description now under resources there's a text file called local stack. dxt so there are some commands which we'll be using to set up the cues and the topics so I'll pick the first one to create the que with the name my- Q let's copy that and then let's run it in the local stack EnV one so it has created a queue named my Q next we'll create a SNS topic with the name my topic and it has created that as well now since the SNS topic is created we want to have a subscriber to it which is nothing but the sqs Q which we created right so let's so let's copy this command to subscribe now I'm for now time being omitting this last attributes um parameter we'll come back to it in a minute when we get there but for now I'm just copying till this notification end point so what this ISS is create SNS subscription where the topic Arn is the topic that we just created and the subscriber protocol would be sqs with the endpoint of the MyQ that we created a few moments back so let me run run this so the subscription is created here now let's go back to our intellig and try to run our application so one thing with spring Cloud AWS is whenever we bring up the application it will try to create these cues and topics if it doesn't find those already on the environment right uh so this is fine for local development but on production systems or uat right you don't want your app to and create a topic or a queue right so your IM am permissions should be such that the application doesn't have right to create or create a topic or create a queue right that those should be created beforehand and then the application should just go consume and publish messages onto those right so yeah let's try to run this demo application so I'll hit the Run button here so as you see it says Tomcat has started on port 8080 um actually I forgot to show you the application. yaml so here there are some spring boot properties which we need to set like the AWS region uh the sqs endpoint SNS endpoint now since we are running the local stack on our Docker desktop the URL would be Local Host and this is the standard port on which the local stack runs so 4566 and if you had noted the command we had used for the installation right um let me go back to that so in the docker run command we had a-p 4566 colon 4566 so what that means is we we we would be able to access the local stack URL using the port 4 45 5 66 on our post which is our local machine and the other 4566 was the port within the local stack environment so that's how we are able to specify the endpoint as Local Host uh port and Port 4566 so now that the application has started uh let's see what those two rest end points were first is the send sqs message so let's try to hit that so in the browser I'll try to hit the send sqs message URL and if you see it printed message sent to sqs with the employee ID 1 13217 first name last name and let's check the output of the intellig here it says message received and it's the same employee which we published right so so when we hit this send sqs message Api endpoint all it did was send a message to the sqsq where did we get this message received log from so that's from our listener here so while we published a message to sqs there was a listener continuously listening to that queue and once a message was published U it realized okay there's a new message and it printed this logger doino it says message received and the actual message so the employee object which we see on the browser this was the published message on the sqsq and the log here which is the listener printing out uh the employee object so that was about sqs let's now try to publish a SNS notification and see what happens so just for clarity let me clear out the intellig console output here and let's try publish SNS notification now so if I hit the publish SNS notification now you see the message change to message published to SNS topic my topic and there's a new random employee details here with the ID 20113 and name and last name let's see what happened on the intell console so here again it printed a message received logger which is again from The sqs Listener now since our sqs Q named my Q was subscribed to the SNS notification what happened was when we published the SNS notification the SNS service published that message onto sqsq and this listener picked up that message and started printing on the console but you may notice here the employee ID first name last name everything seems to be null with the ID as zero so something seems to have gone wrong here let's take a look what that issue is so what happens is when SNS sends the message to its subscriber which is sqs in our case it wraps the message and adds it its own set of metadata to the message uh and then sends that payload to over to sqs it doesn't send it as a raw message now there's a flag in SNS which reads raw message delivery and if that is true only then it sends the raw message as it is otherwise it will send a Json string something like this right as you see here the real message as per this documentation that was sent on SNS was this one this string which is this is a test message but when SNS sends this payload to over to sqs it adds bunch of metadata like type message ID topic Arn subject Etc now the Java the spring Cloud AWS Library fails to parse this right because of the extra attributes which it doesn't understand so that's where when it tries to print the employee object it was not able to construct the employee object out of these uh attributes which it doesn't understand now somehow if we can only send the actual message from SNS onto sqs the library will be able to successfully construct a employee object out of it now let's see how to do that now remember when we created the subscription I had omitted the last d-h attributes raw message delivery as true all we need to do is when we create the subscription we need to pass this attribute to set the raw message delivery as true in that case s will send the actual message as it is and not add any extra attributes of its own now since the subscription is already created let's try to update the subscription and add this attribute there so I'll copy this command to update the subscription attribute and add the raw message delivery the subscription Arn would be the Arn which it printed out when we had created the subscription right so I've updated the subscription now so if we again go back uh let me just clear out this console for clarity now if I go back to the browser and hit this public publish SNS notification again you see there's an employee ID with 7711 0333 employee ID and now the message received gives us the same employee ID this is because now the SNS passed the employee object as it is without adding any metadata of its own so please remember this flag um otherwise the spring Cloud AWS Library it currently as of the recording of this video doesn't have a feature to pass the SNS notification attributes so we need to pass this raw message delivery as true so that was about integrating spring boot with sqs and SNS so let's do some cleanup on the environment that we have so if we do Docker PS okay we are inside the local stack so let's do do an exit let's do Docker PS and let's delete this container Docker RM then let's copy this and since the container is running we need to pass the force flag so that it will be deleted forcefully now again if we run Docker PS now there's no container running so that's how you can integrate your spring boot application with sqs SNS using spring Cloud AWS and just for your Loc testing right instead of actually Integra integrating with the AWS Services you can speed up your development effort with installing the local stack and try out the integration on your local environment thanks for watching the video

Channel: Visa2Learn

Views: 2,170

Keywords:

Id: eOdlBTikFnk

Length: 19min 49sec (1189 seconds)

Published: Wed Feb 14 2024