AWS SQS | AWS Simple Queue Service | How SQS Works | AWS Tutorial | Edureka

[Music] hey guys this is aman from edureka and welcome to today's session on aws simple queue service simple queue service provides a fully managed queue for storing messages as they travel between different applications or microservices this service is used by many top companies like bmw redbus oyster and many more it is widely popular because aws manages all ongoing operation and the underlying infrastructure so the clients do not have to worry about how the queue works but before we dive deep into aws sqs i would like to address the agenda for today's topic firstly i will be explaining you what exactly is sqs followed by its architecture then we'll understand what are the various benefits of using sqs and also see a case study on it finally for practical understanding we will work on aws simple queue service and guess if you like our video do not forget to subscribe and hit the notification bell to never miss an update from the eduraka channel also if you're looking for an online training certification in aws check out the link given in the description box below moving on to our first topic what is amazon sqs it is a fully managed message queuing service that allow users to separate and scale the microservices distributor system and serverless application which means each component of the microservice or the distributed system can be scaled individually without affecting the other components sqs is also used to send store and receive messages between software components at any volume it ensures that messages is not lost during the process and it reaches the consumer on time now that you know what is amazon sqs let us now understand the architecture of sqs in this diagram on the left hand side there are three distributed system components which performs different tasks and on the right hand side there are messages in the queue which are stored across various sqs servers now let us understand how sqs works let's say component 1 after completing the task assigned to it sends a message to the queue let's say the message is a now this message is not only stored in one server but stored across various sql servers now component 2 is notified that there is a message in the queue now component 2 can pull the message a from the queue and now can start working on the task assigned to it at the same time the visibility timeout period starts if you're wondering what is visibility timeout period i'll be explaining you in the next topic there is an important point you need to remember as component 1 is sending a message to the queue it becomes a producer of the message and component 2 becomes the consumer of the message after component who pulls the message from the queue it can process the message and delete it from the queue the same thing repeats again component 2 finishes its task and sends a message to the queue which is then pulled by component 3. now let us see what is visibility timeout period so when a consumer receives and processes a message from the queue the message is not deleted and it remains in the queue this is because there is no guarantee the consumer has received the message for example there could be any connectivity issue or an issue in the consumer's application so only the consumer must delete the message from the queue after receiving and processing it visibility timeout period is a time period during which amazon sqs prevents other consumers from receiving and processing the same message let's say if one consumer is accessing a particular message no other consumer can access the same message during the visibility timeout period now the default visibility timeout period for a message is 30 seconds but the minimum is 0 and the maximum is 12 hours so in this diagram i'll be explaining you two scenarios the first one in which the consumer sends the receive message request and the message is actually returned the second scenario in which the consumer sends the receive message request and the message is not returned at this time the consumer can send the receive message request again and if the message is not returned after multiple failed attempt then the visibility timeout period stops and other consumers can now access the message now that you know how sqs works let us move on to the next topic and see some of the benefits of using aws sqs simple queue service ensures the safety of our messages and also ensures that a message is delivered on the right time it can keep our sensitive data secure and also can be scaled elastically using sqs is very cost effective we will discuss about these benefits in details the first one is ensure the safety of our messages for doing this amazon sqs stores them on multiple servers so when one server is down the consumer can pull messages from other servers as well it also allows you to decouple which is separate application components so they can run and fail independently which increases the overall fault tolerance of the system the second advantage is reliable message delivery using amazon sqs you can transmit any volume of data at any rate of transmission without losing messages or requiring other services to be available simple queue service supports two types of queue first one is a standard queue second one is the first in first out queue standard queue supports at least once message delivery which means your messages will be delivered at least once and first in first out queue supports exactly one message processing which means your message is delivered exactly once so you can be assured that your messages will reach the consumer on time the third advantage of using aws sqs is it keeps your sensitive data secure to exchange sensitive data between application you can use server side encryption in amazon sqs this will encrypt all the message body and can only be accessed by the mentioned consumer server set encryption can also be used with aws key management service which will allow you to centrally manage all the keys that protect your sqs messages the next benefit of using aws sqs is it can be scaled elastically and is also very cost effective amazon sqs uses aws cloud to scale based on demand sqs queues are dynamically created and can scale automatically so you only have to focus on building and growing your applications there is also no limit to the number of messages per queue and it also provides unlimited throughput throughput is the rate of transmission of messages using simple q service there is no upfront cost and you do not have to install any other messaging software you only pay for what you use and there is no minimum fees these were just some of the benefits of using aws sqs now let us move on to the next topic and see a case study on it austria is a company that shares unvarish reviews of hotel in nearly 200 destinations worldwide oyster has an extensive collection of photographs and it had to recreate existing image into new formats and sizes for mobile and tablet devices this was very time consuming and was slow in the growth of the company so they adopted aws services now they use various aws services such as elastic compute cloud s3 sqs and amazon linux ami they use sqs and ec2 in an integrated workflow to generate the size of photos they need they use amazon sqs to store the photos in the queue which can be processed later this helped them complete the job faster and using sqs they could also communicate the status of the job so by using aws oyster could process 1 million photos in 20 hours which usually used to take them 400 hours to process this was a 95 improvement they also saved 20 000 per annum now let us move on to the final topic for today and work on amazon sqs in today's session i will be showing you how you can create a queue and how you can insert and delete a message from the queue we'll also see how we can delete the queue for today's demo i have logged in into my aws management console now let us search simple queue service now in this page we have certain information about amazon sqs they are producer who sends messages into the sqs queue and the consumer who pulls messages from the sqs queue and you can see sql skus are used to store messages and wait for consumer to pull them next we have certain benefits and features of using sqs then there are information about company that uses aws sqs now let us create our queue so we have an option here to create our queue we will click on that option and it will direct us to a page where we can create our queue so first we have to select the type of queue there are two types of queue first one is a standard queue second one is the first in first out queue the major difference between them is standard queue supports at least once message delivery which means your messages are delivered at least once to the consumer and in first and first out your messages is delivered exactly once to the consumer another major difference between the standard and the first in first out queue is in standard queue your message order won't be preserved which means the order of the messages might be changed in some circumstances but in first in first out queue the message is sent and received in order so we will create a standard queue today after we select the type of queue we need we have to name our queue now let us name our cue my q we can use alpha numerical characters hyphen and underscores as well so we just named our cube very simple my queue then next we have to configure our queue in configuration we have visibility timeout message retention period delivery delay maximum message size and receive message wait time in this video we have discussed about visibility timeout period in details just for a reminder i will explain you what is visibility timeout period again it is a time period in which only one consumer can access the message in the queue and during this visibility timeout period no other consumer can access the same message so it can be anything from 0 seconds up to 12 hours next we have message retention period it is a maximum amount of time that a message will be stored in the sqs queue before sqs deletes it now let's say no consumer pulls the messages for 4 days then after that sqs will automatically delete this message in the queue it can be anything from 1 minute to 14 days now let us set this for 5 days next we have delivery delay it is the amount of time to delay the first delivery of each messages which means now if we give this as 15 seconds so only after 15 seconds the messages will reach the queue and the maximum message size depends on the producer who's sending the message next we have receive message wait time this is the maximum time that the consumer has to wait in order to pull message from the queue can be anything from 0 to 20 seconds we will set all our configuration to default and continue with our access policy now access policy will define who can access your queue there are two types of method one is the basic and the other one is the advance advance uses a json object to define an advanced access policy but we will stick to basic in this demo now in this there are options to see who can send messages to the queue and who can receive messages from the queue there are two options only the queue owner is the person who is creating the queue only the queue owner means the person who is creating the queue or you can also specify certain aws account user or ami user the same applies for receive messages also but for this demo we are going to stick to only the queue owner because we do not have any other specified aws accounts or iam users next we have encryption now encryption is optional as i've mentioned before we can encrypt our sensitive data using server-side encryption now if we enable this we can use server-side encryption with aws key management services which will allow you to centrally manage all the keys that will protect your sqs messages for this demo we're just going to disable it next we have dead letter q now what is that letter queue if a message cannot be consumed successfully you can send it to a dead letter queue dead letter queues let you isolate problematical message to determine why they are failing so for example if one message in the queue cannot be pulled by any of the consumer then you can send that message to the dead letter queue this message later can be examined by any of the user or technician and can find what was the problem with that message so we are going to disable the dead letter queue here in order to enable dead letter queue we should already have a queue if we have a queue we can just select the queue name from here but in this case we do not have an existing queue so we have to disable it next we have tags which is optional a tag is a label assigned to the aws resources we can use tags to search and filter our resources or track our aws cost now after we have configured and selected the access policy let us create a queue i'm getting a error here because i've given a space between the hyphen and q so now let me just remove this space yeah now i can create my queue we get a message saying cue the queue name is created successfully now you can see the name of my queue the type of queue and the arn of the queue it also shows there is no encryption and no dead letter queue now if we click on more there more information about a queue the date it was created the date it was last updated and what is the maximum message size it also shows the retention period the default visibility timeout and the message available in the queue now if you scroll down there are various other option there is sns subscription and there is lambda trigger we can use sqs to trigger any lambda function as well under dead letter cube we can see that there is no dead letter q set for this queue under monitoring we have various graphs such as approximate age of the oldest message number of message delayed and many more all of these are empty because this is the first cube we have created next we have no tags so even this would be empty and this is our access policy then we have no encryption so even this would be empty now let us move on to the fun part and send and receive messages into the queue so here we have the message body so now let us type hello world so this will be a message which i'll be sending to the consumer then there is also option for delivery delay which is the maximum amount of time the message will take to reach the queue we set it to default which is 0 seconds we can also add message attributes the water message attributes message attributes can be anything it might be an identifier or a signature or anything else we will add a message attribute as well now let's say name is demo and value is aws sqs now when we scroll down you can see the receive messages message available is zero the polling duration is 30 minutes and the maximum message count is 10 this is the default set by aws sqs now let me send a message your show is working i'll get a confirmation that my message is sent now if we view details over here we'll have the id of the message and md5 of the message body and the message attribute md5 is to check if the message sent by the producer is the same message which is received by the consumer now let us check our message so here we can see the message available is one so now let us pull the message from our queue there are two options we can either pull it from here or here you see guys it is so simple to use aws sqs this is why many top companies are adopting amazon sqs after the pulling is completed you can see that there is a message available over here when you click on this it will show the body of our message which is hello world it will also show the attribute which was demo and value was aws sqs we can also see the details of a message this is the message id the size of the message the md5 of the message body and the message attribute we can also see the sender account id and what time did it send it now that you know how you can send and receive message from the queue let us delete the message which is available in the queue so after we have processed the message it's time to delete it now we'll just click on this and select the delete option after this it will ask for confirmation if you want to delete the message we'll just click on delete again now we'll get a confirmation that one message is deleted successfully now you can see that there is no message available now this is how messages are sent and received from the queue now we'll see how we can delete our queue so we clear the content here and then just click on the name of our queue my queue so we have an option here to delete our queue we will click on this and then we have to type delete over here and then click on the delete option here we get a confirmation that the name of a queue has been deleted successfully and with this we have come to the end of the session i hope it was helpful happy learning i hope you have enjoyed listening to this video please be kind enough to like it and you can comment any of your doubts and queries and we will reply them at the earliest do look out for more videos in our playlist and subscribe to edureka channel to learn more happy learning

Channel: edureka!

Views: 103,674

Keywords: yt:cc=on, aws sqs, aws sqs tutorial, amazon sqs, aws sqs fifo, aws, aws queue, aws sqs example, what is sqs in aws, aws sqs demo, aws sqs long polling, how aws sqs works, amazon sqs visibility timeout, aws sqs message lifecycle, aws sqs architecture, sqs overview, simple queue service, standard queue, what is aws sqs, features of aws sqs, aws sqs dead letter queue demo, amazon web services, amazon simple queue service, aws tutorial, aws Edureka, edureka

Id: vLNDaZuA3Dc

Length: 17min 45sec (1065 seconds)

Published: Fri Jan 01 2021