AWS Lambda Cold Starts Deep Dive - How they work

have you heard or experienced lambda cold starts and want to know more in this video we're going to be going deep into the depths of lambda cold starts what they are how they work and how that affects the way that you design and the way that you build landers as part of your aws [Music] services hi guys my name is sam with complete coding where our aim is to make you into the best developer that you can be this is the first video in a series where we start getting into the nitty-gritty details of aws services we're going to be starting with lambda and lambda cold starts lambda cold starts is where you'll make a request to a lambda for the first time and there's a little bit of time at the start for the setup of that lambda to understand this properly and how it affects the way you design and build systems we first need to understand the process of starting a new lambda once you've deployed your project the code for your project is stored somewhere in s3 behind the scenes as your lambda code the first time you invoke that lambda it isn't running anywhere so what amazon does is it has loads of servers that contain lambda containers these are ec2s which are specifically designed to run lambdas behind the scenes for us this lambda container doesn't have any information about the api that we need to run so what it first does is it sets up everything it needs to be able to run it that'll be your lambda permissions if you're having a vpc it needs to know about what size of lambda you need to run and it gets that all set up once that is set up you then download your code onto that container and once that download is completed you then are able to run the code and return the result the second time you make a request to that lambda what happens is that container is still alive so that means you don't need to download the code again or set it all up it is already configured for you so that means the second time it just runs the code this is called a keep alive period and amazon have implemented this so that you don't have to download the code every single time this cold start with most languages ignoring.net because that's a bit more complicated but with most of the runtimes it takes up to 0.8 of a second to do all of that setup at the start so that is an extra 0.8 seconds you could have on every time you invoke your lambda but with this keeper lives it only happens when it is a new container created so i talked then a little bit about keeping it alive and with amazon it tries to keep that container alive and there isn't a exact fixed period of time but most people seem to find that it's about 10 minutes so that means if you hit your lambda with an invocation once and it cold starts and you hit it seven minutes later it is almost certainly going to still use that existing container after 10 minutes aws decide that you're probably not using this lambda enough so it removes the container so that someone else can put their lambda into that container if you then invoke your api again it gets a new container and has that cold start again so that's how it works when you have single requests if you have requests coming in within 10 minutes it's going to hit the same box and restart that 10 minute timer if you have requests that come in way over 10 minutes so 15 20 or even like once an hour each time you're going to get a new container and it's going to cold start so you're going to get that extra 0.8 seconds so some examples of how this could affect the way that you design your lambdas if you have for example a set a couple of servers and every minute they the three servers send some information back to your lambda to i don't know put in a dynamodb table to do some data storage to see usage over time in this situation you've got something that is passing data back every five minutes and you've got three of them so the chances are that every five minutes you're going to have three requests which means that your lambda is never going to reach that 10 minute timeout which means you're always going to have one of your lambda containers alive this is really good as it means that the cold start never happens another scenario could be that you set up a smart home and inside that smart home you set up an api so that you can hit it and it returns whether any of the lights in the house are on the chances are you're going to check this maybe once an hour or when you leave the house and then before you come home but you won't be checking it that regularly in this case because the requests are varying frequent the first request is going to cold start it's then going to stay alive for 10 minutes but because the request is the second request won't come back in time it's going to close down and reclaim that container and when you make a request at lunchtime it's then going to make another cold start this means every single request is going to have that extra 0.8 seconds which sounds bad but 0.8 seconds on something that you've built yourself for your own personal use usually isn't that bad if you decide that it has to be as quick as possible there are ways of getting around this the first is inside aws lambda there is a way to reserve concurrency using this you're basically saying i own this one spot of this container and i always need to have that ready for this lambda this is really good because you're never going to get cold starts but it's also really bad because it means you're a paying for that lambda the whole time but also with reserve capacity it doesn't scale in the way that landers normally do if you for example invited all of your family onto it and they all tried to do the same thing at the same time only one request would go through because you've got one reserved capacity and the other two would have to wait until after your request is finished to process another option is to actually trick the system a little bit you can do this by manually invoking that lambda every five minutes you can do this with a cron job or a set up a task so that it hits that lambda with a empty request and the start of your lambda code goes check if it's an empty request if it is then just return the way that this works is it is enforcing that container to be hit every five minutes this means that it never reaches its 10 minute expiry time so it never closes down this has the advantage of when you do make a real request it is almost certainly not going to have the cold start but you're also not paying for that lambda the whole time and also you are getting the expandability so and the scalability so if more of your family members do join and they do all make requests at the same time it will scale they will receive cold starts as you're only starting one lambda at a time but it won't stop the requests from working another example is if you're taking this to a slightly larger scale say you have a retail website where you have lots of users and you're getting say five requests every single second for product searches and your product search api takes just under a second to do the search and return the data that means if you have lots of requests your containers are going to stay alive but because of the cons currency of these you're going to have five of your lambda containers open at all times then if there is a spike in traffic and all of a sudden you're now getting 10 requests a second those first people to make those new requests are going to be unable to use the five existing numbers because those containers are already being used this means that you're going to have five go through quickly but five new requests go through and have to do a cold start now you've got it so you have 10 of these containers set up and as your usage decreases back down to normal over time you'll see that the number of containers you have slowly one by one they will all time out once they have not been invoked for 10 minutes until you're back to having your five containers here you have again the two options you have leaving it and saying that actually a couple of people getting a cold start isn't that bad and if you work out how many requests would cold start it is fractions of a percent so with this it's probably not that bad having this if you are building a shopping app and you need to get everything back as quickly as possible then you can do something similar to the mock requests we did for our house you can do this so instead of invoking just one lambda every five minutes you can pick any capacity you want so you could say i'm going to request 50 lambdas which is going to spin up 50 containers they're going to be alive for the next 10 minutes if you set your cron job to every 5 minutes you're always going to have those 50 containers alive so you can have up to 50 customers making a request every single second and none of them will ever get a lambda cold start if you have a massive day and it goes to 80 customers requesting a second the first 50 will get really fast responses and the last 30 then will have a cold start but they won't be denied access to your api so far we've looked at what makes up a lambda cold start as well as having a look at some situations that you guys might have and the design decisions that you may need to make to suit your needs now we're going to have a look at what actually affects the amount of time it takes to do a lambda cold start most people think that one thing that affects this is the amount of resource you assign to your lambda lambdas can have anywhere from 128 megabytes all the way up to two gigs of ram and this doesn't actually affect the amount of time it takes to do a lambda cold start this is true in all languages except.net if you use net then make sure to search lambda cold starts versus resource size as it is very correlated for you guys one thing that definitely does affect the amount of time it takes to do your lambda cold start is the size of your lambda code if you've got really small code it's a lot quicker to download from the servers into your container if you've got loads and loads of packages and modules and extra code that you don't really need it can obviously increase the amount of time it takes to download that code therefore increasing your lambda cold start time a really good tip with this is always try and make your packages as small as possible using something like webpack can be really good to only requiring the code you need and then minify it to make downloading it as quick as possible one of the last things that does affect your lambda cold start time is whether you're in a vpc vpcs require a little bit more setup than just a normal lambda so there is a little bit of overhead in the setup of lambdas in those vpcs historically this used to be really bad taking seconds and seconds of time but now amazon have reduced that down to a level where it takes only fractions of a second to set up the vpc configuration it is still in a bit of additional time so something you might need to think about deciding whether you put everything into a vpc or keep some lambdas outside of your vpc in this video we have done a deep dive into aws lambda cold starts we've looked at the setup process and how that can add a little bit of an extra delay onto your aws lambdas we've then looked at a couple of different scenarios where people might deploy a lambda and the approach that you might want to take in those situations if you want to deal with this lambda cold start if you've learned something new in this video make sure to give it a like as it really helps more developers like you get to learn about this information and if you haven't already make sure to subscribe down here because i'm going to be doing more videos where i do deep dives into specific features of aws services that you probably already use these will help you become much better developers and design systems that are more resilient

Channel: Complete Coding

Views: 3,345

Keywords:

Id: Kja6EKFoZOs

Length: 17min 14sec (1034 seconds)

Published: Sun May 31 2020