Advanced Serverless Messaging Patterns for your Applications - Julian Wood

[Applause] [Music] [Music] continue our angle so going to be easier for all of us in english otherwise it'll be a bit of a mess this presentation has got lots of information in it anyone who's worked with aws knows we've got lots of services and love them or hate them there's lots of choice and that's what's part of the power of building with aws is you've got a lot of different cool things to choose from so the slides are already available up there you don't need to take notes or photos if you don't want to of the slides because there's a lot of information i'm going to go through and it's all going to be available afterwards so when we talk about serverless and we talk about lambda and we talk about functions and events how does it all sort of fit together we like to think of as lambda is the sort of small little part it's a function uh and it forms part of the bigger picture of functions as a servless which forms part of the even bigger picture which is event driven compute and when lambda was announced it was never announced as a serverless thing in fact there wasn't any such thing as serverless when lambda was announced it was announced as something with event driven computing an event happens and some cool compute can do some stuff and that's where lambda was born serverless idea came after that and we're sort of tacking that on so that's just to give you around about how it all sort of fits but i'm talking about messaging today because a lambda function is cool but one lambda function in isolation it's not going to do much serverless applications and the applications that you are building are going to be joined up of multiple microservices and the benefit is is as it says here you know looser they coupled the bigger individual services can scale they'll be fault tolerant individually they'll have less dependencies and that means that the faster that you will be able to innovate and build your applications and in fact tim bray one of the cleverest people on the planet who used to work in aws says that if your application is cloud native or large scale or distributed and doesn't include a messaging component then you probably have a bug so think of that when you're building your serverless applications that you messaging is going to be a critical part of it if you look at the amazon.com web page for example i should have done amazon.fr webpage i didn't think of that but anyway sorry about that uh when you place an order on this webpage a lot of things are going to be synchronous you're going to be viewing the web page you're going to be able to click and look at kind of things but once you place an order there's a whole lot of background processing that's going to happen there's going to be some payment that's going to be happening there's going to be shipping there's going to be logistics there's going to be a van that's going to arrive at your front door and all of that is asynchronous all of that is behind the scenes that's got nothing to do with how you are going to be interacting with the web page and i'll explain how that's going to be important because a lot of people think that if you're going to have microservices and you're going to have different components connecting together should every microservice connect to every other microservice via an api well maybe not if you think of your application as having two different parts a public interface like the amazon.com or amazon.fr web page where your customers or your business partners or people who interact with your website are going to communicate with you yes that's probably a good place to have an api but behind the scenes all applications have a whole bunch of different things that they need to communicate and need to do like the shipping logistics or the payment logistics that happen behind the amazon site and i'm sure the applications that you do too so what you can think of is the internal services focusing below the water line this is a bit of an analogy of looking at an iceberg if you know an iceberg only has a little bit at the top and a whole bunch under the under under the water below the water line in your iceberg you can think of that's all your internal services and you don't necessarily have to have apis there's some other options that you can use and messaging can provide more resilience availability and scalability for your application if you're wondering why i mean something like scalability well the individual services can scale in different ways and that can give you cool superpowers that you'll be able to have a more resilient and more available application because each individual microservice is responsible for one or maybe two things and they can be scalable individually and your bigger application can be more resilient because if there's a problem with one part of your application it's not going to affect the other part so let's have a look at some microservice messaging patterns that you may want to use so two common patterns that you can use is something called a cue and something called a topic and a queue is point to point and a topic is publish subscribe so in this example over here you can see we've got a sender component maybe one component i've shown here on the slide but could be a whole bunch of components that are then sending messages onto a queue and you can see here the messages a b and c are then sent out onto the queue and they're receivers who pull these messages off the queue now with the queue each receiver only pulls one message off the queue one receive there's a single relationship between your receivers and the messages so one message is received by each receiver when you talk about topics it works slightly differently that here you've got messages a b and c and it uses a fan out and you've got multiple subscribers to the topic and so you can hear see here messages a b and c um are sent to all three of the subscribers so that's topics and queues two different ways you can think about how to distribute your messages between components now within aws mentioned earlier before we've got sqs amazon sqs which is a simple queue service and amazon sns which is simple notification service both of these services are cloud native they are serverless there's no infrastructure to manage there's nothing to patch all the scaling is built in to the to the service and it's just available and can really go up to amazing scale now if we're talking about lambda as we were in the beginning lambda can be a sender onto a queue or a publisher onto a topic lambda can also be on the receiver side so you can have a lambda function that pulls information off a queue or a lambda function that can be a subscriber to an sms topic and it can get information off of that so a lambda can be a consumer and in fact lambda could be both so you could have lambda functions uh that are putting messages onto a queue into a topic and you can also have lambda pulling them off another way of thinking about publish subscribe is a thing called an event bus an event bus here you can see we've got messages a b c and d and the event bus acts as a sort of bit of a traffic router and you can build up rules and filter rules to say where should messages go and in this example you can see messages a and b are going to go to one location and messages c is going to go to another location so it's not quite a queue where the messages are distributed it's not quite a topic where messages are broadcast to everyone it's a bit of a mix and most a bit of a mix and match where you can have multiple publishers and multiple targets and it's a really uh efficient way amazon aws has a product called amazon eventbridge which has been spoken about earlier today and that is also again cloud native it's serverless and the cool thing about eventbridge is that you can grab events from aws services you can grab events from your custom applications and we've got some great sas integrations with partners like datadog who are presenting out here today and a number of other sas partners that events can flow directly into your aws account without you having to set up any infrastructure to manage and then you can create the rules and send those events on to any other services those services can be things like step functions which is workflow automation uh edible slammed obviously uh kinesis data fire hose or in fact any api destination on the internet and eventbridge as our next speaker syria is going to be talking about is a really powerful construct to think of event driven applications the name is eventbridge but it's a really cool way to have a lot of flexibility for events to flow around and between your application so just to summarize we've got sqs sns eventbridge and kinesis i briefly spoke on and these are some of the messaging services kinesis i haven't brought up yet and kinesis is basically uh for streams so the difference between streams and events is streams is normally a lot of data and streams what is important is the correlation between the different data points is important so something like a click stream data or something like log data where the time order is important the way the different points interact with each other is important rather than a queue an event or a notification where the ordering and the correlation between the individual events don't matter as much so lambda fits into this in a number of different ways there are three different ways that a lambda function can be can run one is a synchronous push system so if you think behind a normal web page if you've written any web pages and you may have an api in front of it api gateway which theodo was talking about earlier that'll invoke a lambda function and that is synchronous gonna go to lambda get the response come back and you have your response back from your caller the asynchronous one is very important for a lot of the events we're talking about over here and this is when something like sns or something like s3 the object storage service can invoke a lambda function asynchronously lambda can go off and do its processing and in the background and as soon as the um the message has arrived at lambda lambda tells sns or lambda tells s3 or a bunch of other services i've got your message you can carry on responding and i'll go and do the work in the background it's a very powerful and very important way to think about asynchronous messaging and the third one just the way that lambda works is a pole based mechanism and it's basically going to in effect reach into dynamodb it's going to reach into kinesis it's also going to reach into sqs for example and it's going to grab records out of those services and then perform some computation on it and maybe send the data back to some other kind of service anytime that lamber is invoked by each of these services or even via the console it uses the lambda api and this supports you know the synchronous and asynchronous models and it's just a basically a blob of json that gets passed between the various services and the client is included in every sdk so however way you happen to use lambda it's always using the lambda api to communicate and get lambda functions to run so lots of different services how on earth would i choose between them and the short answer is it's not going to be that easy to choose between them but there's some good information out there and the reason it's not easy is not because they necessarily complex it's because applications are different and applications have different needs and each of these different services have their own superpowers that you should be considering when building your uh your application i wish there was one uh messaging service to rule them oil rule them all but unfortunately in a in the past 30 40 years um computer science hasn't come up with one single messaging solution so all of these have different way in different ways and to think of them i like to think of them to compare it along these sort of six different axes scale concurrency controls durability persistence the way it's consumed you know failures and retries and also pricing and there may be other things you also want to consider when you want to choose between your messaging services maybe something like complexity that could be something that you maybe want to think about as well so this is where the detail gets heavy on the slides don't worry i'm not going to go through at all i'm going to you know do a quick kind of summary but these are the different sort of scaling and currency uh concurrency controls between the different services and [Music] the click went and i was panicking that i'm going to have to read through the slide which is going to go against everything i said about not getting into too much detail but the cool thing to think about is lambda's got a really cool way that you can control how many lambda functions run concurrently now concurrently basically means at the same time so if you've got an application that needs to scale up and you need a thousand concurrent requests lambda is going to spill up split up a thousand uh what we call execution environments that's going to run that lambda function run that little piece of code now as soon as the function is finished it's available for reuse so if another invocation comes in for lambda and that function is not being has just finished it's available for reuse and what you can do is you can use this lambda concurrency to control how many functions are running at the same time which could be useful sqs has just got something extra with sqs you can actually batch information so in a queue you can put a whole bunch of messages and lambda can pull out those messages in a batch just slightly differently for handling the scaling and then kinesis has got this concept of shards which is literally how many parallel processing units you can have and you can scale up kinesis to a crazy amount of shards and that's how you're going to have even more throughput for your kinesis streams so the two controls for um for managing lambda are these two things one's called reserve concurrency and one's called provision concurrency so reserve concurrency is basically saying what's the maximum number of lambda functions that i want to run at any one time and why would you want to do that well two reasons mainly you may want to guarantee that you have enough lambda functions available to run because within aws we do have some limits to protect the service and also to protect your applications that things don't go crazy and you don't get you land up having a huge kind of bill there are some limits built into the system to say that you can have a certain number of lambda functions that are running at the same time and so if you've got lots of lambda functions in your account you don't want them grabbing out of the big total concurrency pool so what you can do is you can reserve concurrency and say i want a thousand invocations for that lambda function and you know that there will always be a thousand invocations concurrently available for that function um the cool thing uh one the thing to remember that people may not know is if you set the concurrency to zero it's basically a stop switch so that just stops all invocations for that lambda function and you may be able to use that if you've got a downstream service that is broken or you need to do some maintenance work or something and you don't quite have control of the events that are coming in you set your concurrency to zero and lambda just won't invoke the function anymore it's a useful little trick to have in your in your back pocket if you need it and provision concurrency which daniel mentioned that people are overpaying for in the first talk from datadog but provisional currency is all about being able to pre-create lambda environments uh because it does take a few minutes sometimes to scale up the number of lambda functions that can run at some time and so if you do need a huge amount of lambda functions exactly available at nine o'clock in the morning because you have a you know a competition on your website or you know a sports streaming company is going to be doing some sort of sports stream and you know it x amount of time something's going to happen you can basically get lambda ready and waiting to um to to be able to process all the requests that you're going to need so the concurrency and models another way to sort of think about it is with um sms event bridging api there's no sort of storage of the events it's just passing through the system and lambda is going to do an invocation for each of the events and pass it on to the next kind of system for sqs when you've got a queue there is a concept of storing information in sqs and you can see there you've sort of got a batch of messages that the lambda is then going to invoke and in stream based kinesis when i talked about the parallelization that's how this works you have a stream and a lambda function gets attached to a stream and it can pull records off that stream and the more streams you have sorry the more shards within a stream you the more parallel processing you can have with lambda so it's just different ways that the services integrate to deal with concurrency so durability is thinking about how messages are stored while they are in flight so what happens to messages as they are going through as they're going through the system well basically lambda doesn't have any durability itself because it's a very sort of short run connectivity between these different kind of services that we don't need to persist the data in lambda but for the other services they all run across multiple availability zones so you can be really confident that if you're going to give any message to sns eventbrite sqs or kinesis we have an issue in one of our availability zones yes it does happen sometime your message is going to be you know pretty much guaranteed that it's not going to be lost so persistence is something slightly different from durability and persistence is how long are we going to keep the messages around and there may be some useful ways that you can use to keep those messages in the service for a little bit of time lambda s and eventbridge don't have any persistence so an event comes into one of those services it gets processed as quickly as it can and gets sent off to another service but sqs and kinesis have this concept of that you can actually store data in the service so sqs can act as a bit of a queue a bit of a buffer so what you can do is you can put messages in sqs and they're going to stay there until you are ready to take them out and that can be useful for a number of different scenarios one can be if you've got a downstream service that maybe it's a relational database or it's an api on on-premises or maybe an api on the internet that's got a throttle and you know you can't use x amount of you can't overwhelm that api this is a good way you could use sqs to store those messages keep them all ready for you and then have lambda invoke using the concurrency controls to protect that downstream resource super useful another way sqs can be useful is if you then need to do some work on a downstream service so you want to take a third party api is going to have some maintenance work you don't want to lose the messages or have timeouts or have things that are not able to connect what you can do is you can durably persist the information in sqs and then at your own time when the services come back up the messages are still going to be there and you can pull things off sqs really helpful just to carry on where you left off and kinesis has got also a similar concept where you can configure um how long you want to keep the messages in a kinesis stream form so the consumption model we spoke about the sync async and poll a lot of different information in the slide and some guidance on how to how to use these kind of services um for example things like sns you can have message filtering so what you can actually do is you can look in the part of the message attributes and say well i want one message to go there i want another message to go there eventbridge uses rules the eventbridge rules can also look at the message body so that's something different and for consuming lambda one tip to use is always look at step functions because step functions is a bigger sort of more orchestration level and workflows uh rather than building a whole lot of complicated logic within your lambda function there's some pace payload size limits you may need to consider so this is just one of the other things you need to think of between using these different services uh retry failure error handling uh you know all the different services have different ways that they can retry if there's a problem with the service you can see sms is up to 23 days event bridge is up to 24 hours sqs we talked about the queue the messages stay in the in the queue so these are you know retrying the failure handling is to think of some different ways that maybe your application needs um and how you're going to architect architect your application with these different services now things do go wrong within a lambda function an invocation may fail that may be a problem with your code not saying your code any code that i write i need to be extra super careful that it's not going to make any mistakes that's just because i'm a rusty coder i'm sure your coding is all perfect and brilliant and you don't have errors but sometimes there is errors in your code or errors in your lambda function and you need to be able to handle that so land automatically being a serverless service does retry attempts so you can configure the retries between zero and two maybe you want to set it to zero if you never want to retry a failed invocation there could be a reason for that and that's something that you can configure and there's also maximum age of uh lambda events so if something isn't available you can configure how long that invocation window can be and once that passes then that is going to that's going to be then a failure of your function invocation what you can then do is if you've heard of dead letter queues that were spoken about earlier dead letter queues is one way if something fails you basically send the message onto an sns sqs queue but a more advanced way with lambda is a thing called lambda destinations and lambda destinations allows you to send a message to eventbridge s sqs or lambda if something is successful and also if something is not successful and it's got a bit more information about the about what failed to just give you a bit more information so if you are using lambda and failure modes with um dead letter q's dlqs it's definitely worth taking a look at lambda destinations it just gives you a bit more functionality eventbridge so lambda has destinations but eventbridge and a number of other aws services do have the dead letter queues um it has been covered earlier so i'm not going to go through it now but this is just a way that if something fails you can keep that error message and root it somewhere else which you can investigate a really powerful pattern lots of different pricing things uh things to think of um but the really cool thing to know is that you know lambda s eventbridge and sqs your first million requests are free per month and that's every month so it's not just if you're within the free tier um you get a million of requests for each of these services in perpetuity for every single month and there are a lot of aws customers that literally are within the free tier for their whole business and um yeah so literally their lambda costs are basically absolutely zero sure they may be paying for other services but this is a really good way to be able to try something because the cost can be really can be really cheap when you when you're getting going and even up to moderate scale quick thing people often talk about the difference between uh eventbridge and sns basically eventbridge has got a whole lot of integrations with third parties and can route to more destinations but sms is really good for fan art so if you've got lots of subscribers you need to send to you know into the millions even sms is really good for you the filtering i mentioned briefly sms filters on just the message attributes eventbridge can filter on the whole message body sns is a bit faster so if latency is super important for you then sms is going to be latency is going to be a factor in your decision tree and there is a pricing thing eventbridge is going to be a bit more expensive than sns so just to summarize if you've used on the comparison if you've used cloudwatch events before eventbrite has replaced that so eventbridge is a really good service built on the history of cloudwatch events and has a lot more functionality sms for high throughputs kinesis you rather use that for real-time processing at huge scale and sqs if you need a resiliency and we haven't covered it today but if you need some sort of ordering which is in the fifo which is first in first out and you want to buffer messages um sqs is going to be preferable to eventbridge so part of these messaging platforms messaging products is that you can all use them individually but the real power comes is actually when you're able to combine them so when i check on my timing before i'm going on i need to be quick there's a concept of topic queue chaining and this is what you can do is when you can have an s topic which can fan out and then what it can do is it can actually put the messages onto a queue and then you've got some application that can pull messages off that queue and process them and the benefit of this is you've got the fan outs you've got messages going to multiple destinations maybe two different microservices but they get persisted in the queue and they store there for as long as you need them until an application can uh can pull them out scattergather is a similar kind of thing where you put something on sns fan it out maybe you want to get a request for quotes from a whole bunch of different places or you want to find out which restaurants are open or some kind of thing where you need to broadcast something out and you want all the responses to then come back in where you've then got an aggregator component that can then get all the responses really powerful to use sns and sqs together eventbridge could also integrate super well with sns remember eventbridge can look at the message body so maybe you can have some really powerful rules that can look at the message body and then send that information on to sns which can then be broadcast out to you know up to billions of subscribers including lambda the cool thing is you can also do these things across accounts so something like sns you could have something central in your event bridge and then uh send the sns message out to you know maybe a million subscribers that all over the world in their own aws accounts um a really powerful way to do that eventbridge works really well as well because you can then persist information into the queue and then you've got the you know the messages that can be rooted with eventbridge so they can go on to different queues there may be different microservices and then you can use lambda to pull the information off and these could be different micro services and uh the micro services b you can see here we could then use the lambda concurrency controls to limit to maybe only 50 concurrent connections to the amazon rds database so that we don't overwhelm our downstream system so really cool ways that you can combine eventbridge in this case and sqs pipes and filters is just a way that you've got a whole bunch of different services connecting to each other the problem is it creates these long chains and that's really complicated so what we prefer to suggest is a thing called saga orchestration where you have a source and a target and you have some orchestration component in the middle that handles all the orchestrations so you don't have to manage what's going on next in the chain and this uses event triggers and maybe you've got you know transactions and rollback and branching and parallel workflows and retries and all these different kind of things that you want to do in an orchestration something goes wrong with creating a user for example you need to undo a bunch of different systems or a flight gets cancelled and you need to refund a card and then you need to release the seat all this kind of orchestration of being able to roll forward and roll back workflows is what aws step functions is really really good for so step functions lambda works super well step functions is a workflow orchestrator and that can connect to apis can connect to lambda can connect to a whole bunch of services uh natively so quickly to sum up i did promise you there was going to be a lot of information if i was to cut it down um i'm sure i would have lost you even more but the thing about it is how real time do you need your info do you need synchronous or do you need asynchronous do you need to create an api or can you use messaging between your different applications what's going to break if lambda fails how are you going to protect your downstream resources all of these services are serverless no infrastructure to manage pay as you go they scale down to zero and scale up to a ridiculous amount of processing and all support i am so they've got you know lots of fine grain permissions to make this really secure so the takeaway is that we've got these great messaging services you can use them in isolation but they've also got superpowers that you can really use all together to be able to combine different things if you're going to be looking at workflows step function is going to be uh your friends as i said with serverless pricing it's really cheap to get started and working that all out so what we do have is we have an aggregation site called serverlessland which is literally everything to do with serverless on aws well a lot about serverless and aws are there videos and learning paths and blogs and a whole bunch of information but what is cool is we do have a thing on there called the serverless patents collection so if you're using aws sam or the cdk the cloud development kit and we have a collection of patterns that you can connect sns and sqs or lambda and dynamodb or api gateway and lambda or whatever all these different kind of services and this is just a really good powerful pattern collection and from datadogs thing over uh over there and i know they're people from the serverless framework over there if sam is in your thing and you want to use the serverless framework please go ahead we'd recommend whatever you're going to do is use a framework it's really going to make your serverless development uh really good um if if the serverless framework appeals to you fantastic we can absolutely love that it's going to make us make your life building serverless applications easier so there we go that's the link to the quick run through of the of the of the presentation you can go into more detail on the slides and look at those six different ways that i was going to compare and yeah thank you very much it was a pleasure and an honor to come to paris today to spend time with you i'm available now for questions uh afterwards as well so yeah thank you very much you

Channel: ServerlessDays

Views: 149

Keywords:

Id: nHbCojH2vEY

Length: 31min 11sec (1871 seconds)

Published: Sun Nov 07 2021