Serverless Architecture Patterns and Best Practices by Chris Munns

[Music] hey everyone good afternoon for those coming in lots of seats kind of all over the place so come on in and get cozy and say hi to your neighbor well we kick off here just cuz I have a whole lot of stuff to talk about in only 50 minutes to talk about it so this session is service architecture patterns and best practices my name is Chris Muntz I am a senior developer advocate for service from Amazon Web Services based in New York City really excited to be out here at in in Belgium this week my first time out in this country so thank you all for having me and again thanks for coming to this session background wise so I've got come some of the other roles that I've had in my career up there but I'm primarily an Operations guy I spent a lot of time in datacenters spent a lot of time managing Linux servers wiring things doing infrastructure automation running databases and stuff like that and within the last kind of year and a half I realized that basically the entire first 15 or so years of my career we're coming to an end in the future and the way that this world is going is towards this thing that we're all calling surplice and so here this week there's been a little over half a dozen talks that have the name server list in the title probably many more that had server list we're talking about service technologies inside of them so this is a really exciting space and so in today's talk when we talking primarily about AWS services around a to be OS lambda so fully up front if you were thinking you have hearing about other service technologies I'll be I will be pretty heavily AWS focused it's totally cool if you want to leave because of that we've got fair warning that that's kind of be that the space that I'm in we're gonna talk about a whole bunch of different things and I'm going to constantly try to bring it back to best practices so as a developer advocate what I do is spend a lot of time connecting with folks like yourself and settings like this and then one-on-one and on Twitter and in slack and stuff like that talking about best practices how to build service applications something else to get out of the way yes there are still servers behind server lists we don't have some magical thing that Jeff Bezos popped out of his brain that we run compute code on that is you know completely unique to us but that's not the point and so the term servlet says it's been a little bit abused in the industry by various different vendors and providers and folks so we have at AWS is a pretty standard when I think definition and actually I got a chance to see Brett and I believe Gees talk a little earlier today when talking about service and so at least Austin Google and Microsoft and a couple others have kind of aligned around kind of the same kind of points for what service is so for us here at AWS it means no servers to provision or manage so absolutely no managing of an operating system no need to patch management anything no need to login over SSH RDP or anything like that this includes no management of containers so this is not similar to a kubernetes or a doctor or amazes for your ecs or anything like that should scale with usage so requests come in requests get answered whatever in this case from your application request is it's something that you shouldn't have to think about ahead of time how many requests you might get at any point in time that aligns very much with the next bullet point which has never pay for idle and so ideally with a service platform you're never playing with for idle right if you're not getting traffic you're not paying for something and typically when organizations think about capacity planning what they're doing is saying how much are we willing to spend in hopes that we get traffic to you know not be denied because we didn't spend enough but also we're not paying for too much idle and across the industry they say typically for IT organizations most server fleets it at less than 20% utilized so it means that you're spending your money on 80 plus percent of idle and so with service technology is no cost and paying for idle things will kind of be fired up and created as you need the lastly availability and fault tolerance built in so here at AWS we have this Mis model of regions there's about 16 or so of these today around the globe I believe regions have inside of them availability zones these are kind of think of them as independent data centers sometimes clusters of data centers we're in typical high availability architecture patterns for AWS you're thinking about having resources at least two or more of those availability zones so that you could potentially deal with any one of those going away in the case of server lists this isn't something that you should ever have to think about there's no real need to comprehend or think about availability zones so AWS kind of the core of the world of service is AWS lambda logo it's kind of right here in the middle of these slides and for a service application what you have is some sort of an event source something that maybe is reacting in to changes in data state requests to endpoints changes in resource state that's going to invoke a lambda function so for those who are not familiar basically what lambda is is behind the scenes here at AWS we run a very large large large amount of computing infrastructure when a invocation comes in from an event source to your configured lambda function we go and grab one of those computer resources for you basically say it's yours for your account take your code pull it down onto that host execute that code for you whatever it does maybe it talks to other databases or services or just has all its own business logic constrained and then I completes what it needs to do and so you only pay in the case of lambda in the hundreds of milliseconds of time many customers lambda functions take away less than 100 milliseconds worth of time and so it's kind of the smallest amount of compute that you could ever think existing or paying for again without you managing any of the underlying resources that are part of it there's number different use cases for this today so I'm very lucky to have three of my close peers here Randal Adrian and Tara who all give talks earlier today so you might see some of the same content that they had as well talking about various different use cases for server lists so things like web applications and backends kind of pretty standard stuff so we see a lot of companies moving from standard kind of three to your web architectures to service architectures or things like single page applications or native mobile applications then powered by service backends data processing so whether it be stream processing batch processing things like doing MapReduce there's a number of really kind of cool scientific libraries built on top of lambda actually kind of I would say a dollar-for-dollar if I was in the States let's say euro for Euro that kind of one of the biggest use cases of service today actually is data processing chatbots Amazon Alexa and then lastly IT automation which is a lot of people's kind of stepping stone into the world of serverless and lambda traditionally also foam-like the operations are all now in terms of how you work with lambda again I mentioned there's this invocation model so something has to trigger a lambda function and today there are three different models for how this works there is a synchronous model so a push based model in this case the example of API gateway I'm gonna make a request to be order API that's going to invoke a lambda function that's gonna hopefully very quickly return whatever it is that the order API is supposed to return and onward I go and the synchronous model you see examples like SNS and s3 where what's different here is I'm gonna put some sort of a message or an object into either SNS or s3 and at some point land is going to do something now typically this is very quick but from a user's perspective or from an infrastructure perspective you put your message or your object into those services you get back 200 everything's okay and then maybe you go later on and you check to see what happened or what the result was of that activity so you don't care about asynchronous reaction from lambda in this case it's just an asynchronous invocation then lastly with DynamoDB can you sis what you see here is the stream based model so in this case and actually with dynamodb it's using Kinesis under the hood requests are being streamed across the the fire hose of Kinesis or the stream of Kinesis I should say and what's going to happen is that's going to help invoke lambda functions behind the scenes they're gonna pull objects basically out of the Kinesis stream and do whatever they need to do with that so we see a lot of people using this for real-time analytics IOT sensor data processing do things like replication dynamodb tables doing things almost kind of like triggered functions off of dynamodb actions a lot of kind of interesting things there and today there's actually a little bit close to about 30 different services that can invoke lamda so lots of different ways to glue together inside of your architecture and make things go so that's kind of the quick overview of what lamda is if you haven't seen it before now I'm gonna start to get back into kind of some of the other aspects of this talk let's see so real quick where do you start and so I'll cover this here for folks who actually have never started in this space before and the answer is find a framework so there are close to about a dozen or so pretty mainstream frameworks that we see out there today these are some of the most popular of all of those so there are again quite a number of tools out there many dozens of them actually apex something called chalice which we actually released from a completely different part of EWS unrelated to lambda in API gateway Claudia Jas server lists Sparta and Zappa and so with all of these frameworks what they're going to do is they're going to simplify what's necessary for you to build and launch a service application so again you've got an event source you've got your lambda function it does something they're gonna help you glue together that entire process so it's that much easier so as an example of this of Claudia Jas what it basically takes is let me see if I can make this thing work here yes basically takes about five lines of code to get an API so you have here you basically are including the package we're going to then create a new version this class here we're going to export module and then we've got a very basic kind of handler for a return hello world really really basic example but given these five lines of code and then the command line it's all the way at the bottom here at Claudia create let's give me a lambda function an API gateway and all of the iam permissions glued together to make this API endpoint exist so really there's there's no easier way to get started creating api's than an example like this again so this is Claudia Jas no GS framework on the Python side of things we have AWS chalice and so it's kind of interesting story the AWS SDKs and api's team people who own things like boto and our various SDKs and frameworks were doing more and more with lambda and server lists and they were looking for easier ways to create api's for themselves and it happened to be a very heavy Python organization and so a couple of developers on the team when we can start hacking together put together this tool and they were like oh wow this is actually pretty awesome open sourced and now it has a really really robust open-source community so very similarly to all we saw with Claudia Jas again it's about five six lines of actual code here and we get an API endpoint so really really really easy to get started if you are completely new to this space so again the frameworks really awesome but some of them are maybe a little more opinionated they actually interact with how you write your code but maybe you're coming from the world or you've got existing code or you're really strong in a language that's not supported by one of these tools so where can you start in that case in this case this is where we have something called a WS Sam and so Sam here our wonderful little squirrel mascot you'll see it pretty much all over the place now in regards to what we're doing with server list this is the official mascot of serve list for AWS and you'll see in on things like on socks which I have my Sam socks on today Randal gave some out earlier and we've got lots of stickers as well and so with Sam is actually stands for service application models this is actually a framework that is built on top of cloud formation that makes it really easy for you to deploy service applications so again kind of gluing together things like lambda functions and api's and DynamoDB tables so if we were to think about what's required for deploying a lambda application with cloud formation we're looking at a whole bunch of code here and I'm not gonna attempt to make you read this gamal but it's about 80 or so lines of code and so that's no fun to write nobody really wants to write 80 lines of UML to deploy something and so Sam really really straightforward it's actually about 20 lines of code the way that this breaks down is the first couple lions are the headers of the template the second line there's kind of key it's called a transform it's a capability of cloud formation then what we have are resources so we have a get function resource and then a list table resource and then inside of the function resource we see properties for lambda and then events and so inside of events this is actually going to relate back to things like API gateway and Canisius and that's 3 buckets SNS alexa stuff like that and so in events there we have type api we've got some properties around the path and the method and so what's going to come out of this template is a lambda function an API gateway DynamoDB table and all the permissions basically to glue together the access for api gateway to invoke lambda lambda to talk to dynamo again just in about 20 or so lines of code another way to think about this some visual sense and this is using cloud formations design or to visualize it is we've got a sample over here and my apologies I lost the text for the link to the URL for this code but basically from lines 9 to 25 what it creates are roughly about six different resources and again otherwise you'd be doing this manually in the console or typing out much larger CloudFormation templates with Sam you've got two commands a package and a deploy pretty straightforward package is gonna actually zip up your files put them up in s3 for you deploy is then gonna take those packages of files and deploy them for you in the light environment and then real quickly also about this or just a couple months ago we launched something called Sam local if you haven't had a chance to see this definitely encourage you to take a look and take a spin on it so what Sam local does is it actually uses docker local on your laptop to wrap up your lambda function and make it available for you locally on your development environment so it gives the ability to basically create a mock API interface you can also do invocation test calls similar to how you do it with Kinesis or dynamo or s3 and it's got a bunch of really cool things to do that it's actually written in go it's open source we've got a lot of community activity here as well so take a look at this and check it out if you were looking at some of the other frameworks that exist though quite a number of them actually do also support local testing now so about six months ago we used to here in the community all the time a local testing so hard impossible how do you do it in that six months about half of the frameworks that are really popular now have local testing including Sam some of them are actually built on Sam and so this is an incredibly rapidly evolving space when it comes to all the various aspects of building and running applications in it cool so it's a lot about kind of the next section here of building and running a service application so building and deploying your application so we talked a little bit about what you could do with Sam itself but there's kind of a number of other parts of the family that fit into this so the first I'll talk about here is code build effectively this is our build and test tool at AWS from the development tools perspective completely managed paid by the minute of usage of it and useful for everything from compiling code to doing linting and syntax checking doing unit tests to eventually deploying your code and the way the code mode works is it has something called a build specs file you can actually put this text file into your code repository or you can configure the service to use it via its API is or the console and what this does is really pretty straightforward if you use kind of any of the other really popular hosted CI tools like Travis CI or circle CI or code build you've seen things pretty similar to this if you've done any of the kind of Jenkins 2.0 stuff you've seen stuff similar to this so just kind of an example of what you would do with code build but I've got some environment variables and then we've got four phases basically phases are sequential grouped areas where I can run commands you can think a little bit of code build is kind of a glorified shell script execution engine although that definitely doesn't give it enough enough credit for what it does and so I'm going to do an NPM install install my my Doge is modules later run EES lint to - lint and check the syntax of my scripts I'm going to run NPM test and run unit tests and then I'll use the cloud formation package command to package up my code around my Sam template and send it on its way so that's code build next up we have is code pipeline so code pipeline is kind of the sibling service to code builds it's a continuous delivery tool again similar to what you would see in tools such as teamcity or what you can do in Jenkins with pipelines a number of other tools out there these days and basically what it does is it allows you to first visualize and then very easily construct what's considered a pipeline effectively your your application lifecycle so we here see here a very basic example for a developer where I have my source repository I then pass that on to a build stage and then I have my deploy where I'm using cloud formation to actually do the deploy of what I'm doing in this example I'm gonna run some stubs unit tests or integration tests you could think of it using lambda and there's a lot of different capabilities in code pipeline that makes this all possible for you but if you're completely new to the space you're starting out completely fresh don't have any of these tools I would definitely say start with these again they're all pretty much pay for what you use products and so pretty painless to get going and always good practice to build CI NCD into your development as soon as you can so talking a little bit more here about lambda so lambda has a concept of environment variables if you are familiar with things like heroku x' or the 12 factor practices they talk a lot about storing configuration information in environment variables not putting it into your code not putting it even into a database potentially and so with lambda you have the ability to pass in key value pair configuration environment variables if you will which are then made available to you basically via the runtime environment from lambda so you pull them into your code the same way that you would pull in say an environment variable from the Linux OS so the same commands in those languages that you use such as the process Don m4 or OS and barn for Python can be used to grab these now you can also use KMS as our key management service to encrypt these so that's useful for api keys database credentials really credentials of any sort or type and so these are really useful for creating individual environments and we're gonna spend a little bit of time here talking about how you think about environments with server lists so api gateway supports something pretty similar in this case they're called stage variable stage variables also these key value pairs that you can configure and then they are made accessible via a number of different fields inside of api gateway requests depending on you know how you have your api gateway figured cool so how does this tie in to Sam so going back to Sam real quick here on the left hand side of the screen we have parameters that we'd pass in and this is pretty standard syntax for cloud formation and so we see that I have two parameters my environment and then special feature one and my environment has lad values of testing staging and prod special features has allowed values of true and false maybe it's a it's a feature flag for something I'm testing and then I actually go and pull those into my lambda function and API gateway definitions over here on the right and so pretty easily with my lambda my Sam template I could accept different values from whoever's using it pass those into my lambda function and then inside of my lambda function consume those and maybe my function behaves differently or they're slightly different logic that's inside of it and this is a great way to pass that in dynamically through that now I mentioned that it's considered not necessarily a great practice to pull in configuration environment variables from databases that said there's kind of an explosion in the number of services that do this today via different ways so there's people who use console which will use things like etcd we also another service called system parameter store which is part of systems manager which actually can do this as well for you so with parameter store you have the ability to store key value pairs and hierarchy you can encrypt them you can really tightly control who has access to them it's centralized and so if you're in an enterprise where you can't just give out keys to developers allow them to access it wherever parameter store is probably one of the best tools from an enterprise perspective for allowing your code to be highly dynamic and - at runtime get access to the variables that you need so we see example of this here and some lambda code basically I initiate the client using boto I then define the parameters that I want to get I pull it in and then later on I can use that value inside of it so really useful for figuration parameters feature Flags database credentials API keys stuff like that so if we put a bunch of this stuff together here so we have again this concept of stage variables we have separate or called stages inside of lambda there I'm sorry inside of API gateway we also have versions and aliases and lambda what you can actually do and combining the two of these it's kind of very easily create these different environments inside of your account so you've got dev beta prodded F staging prod whatever you want to call them and so this is one way for you to control multiple versions of your code kind of across the lifecycle of how you develop it that said there is some some back-and-forth and the thoughts of whether you want to have everything inside of a single stack or things inside of multiple stacks and so a lot of this comes down to personal preference comes down to team size it comes down to kind of governance inside your organization so again you have here the options for having a completely kind of single stack of resources where you've got your you're testing staging the production environments all kind of managed by a single template you could also then do something which is multi stack or instead of having all those things inside of a single stack you have basically reusable templates potentially that allow you to stamp those out repeatedly and then leverage kind of the dynamic capabilities of Sam for cloud formation environment variables and so forth to make it a little bit easier so typically we see enterprises people that are running things a little larger in scale use the latter but if you're smaller and you're getting started you have a smaller team the first will make sense for you now this actually aligns pretty well with how people think about doing their a TMS accounts so interestingly enough a DBS accounts are a unit of control and scale you may not think of it that way but actually what we see is many large enterprises will have hundreds if not thousands of accounts and even now we're seeing a lot of smaller teams try to do this where they'll separate out of staging and prod from different AWS accounts and so a number of kind of pros and cons that come with this around it's obviously a lot easier to manage single accounts first multiple accounts easier also from a management tool perspective so let's say you're using a product like a data dog or a log li well having multiple accounts listed is a lot more complicated in those products than having a single account so that's what you have to think about however when it comes down to permissions and access separation which many organizations do care about a single account model is just not going to work for you and so that's when you're coming in thinking again about multiple accounts and and kind of leveraging tools that help support that and so we have a product called a DBS organizations that came out I think earlier this year technically which helps make this a lot easier for you to manage multiple accounts kind of an umbrella account structure so my personal advice then the guidance that I like to encourage everybody a don't do the mega single stack thing there's a number of things that become really complex when you do that in terms of management of resources also there's maximum size limits to templates and so that can be a little bit of a pain you do want to leverage the dynamic nough so per a matera mappings in the side of Sam so that you don't hard-code values right so you really want to go away from any sort of hard-coded value inside of lambda inside of your Sam templates anywhere else you can make use of things like export and import value or what's called cross stack reference references and Sam so this means that you can have a stack where you launch your database resources and you can have a stack where your servlet stack resources are launched and you can basically point one at the other without having to launch them all as part of the same stack again really help some kind of a separation of roles and responsibilities and lastly you can build out multiple environments for development tests and so forth ideally using the same template even across environments don't do it different ways in different environments that's one kind of common mistake that people will make where production doesn't look like dev or staging and that's typically where a lot of issues come from with code pipeline in a cloud formation there's actually kind of two different ways you can make this really easy so include pipeline you can pass in a template creation file or you could actually hard code in the pipeline the variables that you want to pass in and so if we put this all together in terms of code pipeline but you can pretty easily get here is a pretty straightforward multistage pipeline where we've got a development a staging and a production or in this case sorry testing a staging in a production stack they could all use the same template you can make use of environment variables and dynamic Nissa pass this all in even across different accounts and kind of get all the nice kind of controls and stuff that you want and definitely this is the type of model you want to follow regardless of what your CI and CD tooling is now all of this aside we just a couple weeks ago as service coffin New York City announced that we will shortly here be launching something called traffic shifting in lambda aliases and so what this will allow you to do is assign a weight to a lambda alias such that what we will do behind the scenes from your invocation source is shift traffic towards your second version given the weight that it has so you just think of it as pretty much a weighted alias the way that it looks from the CLI is I have a function I've updated it I'm going to publish a version and then I'm going to sign an Elias and I'm gonna give it 5% of traffic and so we will again behind the scenes direct that 5% of traffic you don't have to handle that from say an API gateway or Kinesis or any other sort of invocation source you'll see separate graphs separate logs for those different versions and then you could very easily automate out okay I got 5% up it looks good for some period of time now I'm going to shift that all over to 100% turn down my old version and away you go and so that's going to change the story that I just told you little bit here but as of today you want to think of multiple stages for environments and having them all kind of broken out cool so given all of this I often play to say to you can with fast if you can't measure what's going on I previously used to represent all of the DevOps products that AWS has all the management tools and developer tools and what I used to find was that basically 9 out of 10 organizations really really under invested in monitoring logging Metris collection and application profiling and therefore areas where basically every dollar you invest is going to give you some multiple back into your business and so you really want to think about what you're doing from a visualization and measurement standpoint for a server list so we have is cloud watch built in across a number of different levels and so we like to say in that monitoring and logging are universal rights inside of service applications so for cloud watch metrics we have a number of metrics that we captured by default for lambda and API gateway 6 4 lambda SMF API gateway we're always looking at more that we can add and kind of what value that gives our customers and then for cloud watch logs for those use lambda today you know that it's very easy to do things like a console dialog and output information to cloud watch logs API gateway gives you a number of different ways to capture logs as well including you giving the ability to capture actually full method request and body content which can be really really useful for troubleshooting then the last bit down here what's really interesting that people don't I think know enough about is that there's this concept of log pivots and cloud watch now what this is is you can basically apply regex to a log line grab a data point generate a graph from that and now what you can do is things like percentiles inside of cloud watch metrics set alarms on those and so you can actually see what your averages your TP 75 80 85 90 95 99 and understand the kind of where your performance lies across that spectrum that could be really really key and performant is very important especially when people say things like things seem slow what what is slow what percentage of our users a little earlier today Randall hunt from our evangelist team talked a bit about x-ray what x-ray does it is not an APM tool so application profile of monitoring tools like a data dog a new relic dynaTrace those are much more sophisticated tools and actually almost all of them now or have or will soon have some ability to consume x-ray so x-ray is not a competitor those tools there's a feeder for those tools what x-ray does is gives you the ability to capture information about what's going on inside your code this could be lambda EC SEC - technically anywhere that you can run code you can tool lambda and x-ray into it it gathers information about your application and the form of what are called traces record those traces then makes that information available to you and so you could do this yourself with really verbose logging and metrics collection but use a tool like x-ray it's going to make your life a lot easier so we can see here try this highlight anything again awesome what I'm doing here is I am including the x-ray core SDK I'm then wrapping the AWS SDK inside of the x-ray capture call and then what I have is just a very basic lambda function that talks to s3 so it's going to put a key into s3 of the body of hello and there we go there's my s3 put object now inside of lambda or sorry instead of x-ray what this will look like is this and so what I see here at the top where it says AWS : : lambda that's actually the lambda service starting up or invoking my function then I could see my function information and then I can see my call to s3 now had there been issues with this or was I actually doing it at some sort of real-world volume I would see a lot more information about the requests whether they were failing when they were timing out with their errors in them heuristics basically in a diagram of the performance and stuff I could also do things like dive into kind of the waterfall for this so for a much more sophisticated application this is actually really really powerful what is my lambda function doing inside of its calls x-ray can help figure this out and so I'm very very regularly will talk to lambda customers and they come to me and they say how do I figure out what's going wrong with my lambda function well it's using all the tools that I just mentioned they're there they're built in for you they're very tightly ingrained and tied into the platform and you can very very quickly turn them tweak them a little bit and start gathering information so if you haven't looked at x-ray yet take a look at x-ray it's really really pretty awesome a lot of different capabilities inside of it you can actually wrap individual functions or individual calls inside of your code and x-ray and get lots of detailed information from it they'll underestimate the power of logging lambda so it's really basic you can console dot log whatever in the world you want and it's gonna get captured and shown in the cloud watch logs interface so for you know the kind of cheesy debug some message type outputs that you would typically do if you're developing an application where you would turn that off in production I guess what you can leave it on in production and lambda no end customer will hopefully ever see that because console dialogue is only captured by the underlying platform so you can keep the bug messages on you can capture all sorts of things and I typically find when customers are concerned about performance of functions it's a lot of looking at information that could be captured by logs and lastly when it comes to thinking about metrics one thing that people will do is they say oh they're the built-in clot watch metrics they don't give me enough information yeah that's absolutely true create custom ones especially ones that are more business centric an example of this I had a customer that was processing data off of konista stream and what they were struggling with is sometimes their lambda functions would timeout and we said okay well what is your average lambda function doing that's processing data from the stream I said we have no idea and so we said okay but turn on some logging let's turn on some x-ray let's see what's going on and they found under peak load for their application basically they were collecting beacons from a similar to like an advertising service if you will under peak load Kinesis bashers up messages or the client batches of messages then Kinesis itself batches of messages and when lambda was processing it it was unpacking all of that and some lambda function executions could try to process several hundred messages inside of that some would only process a couple dozen and so when you had that several hundred the time that it took to get through all of those would cause the function and timeout so we figured this out by just creating a simple cloud watch log entry that basically put out okay how many messages am i processing from Kinesis in this and nivel engine it right away highlighted what they had to do to fake which was adding more streams to Kinesis and on they went so we talked about a couple of best practices they're a bit more general for service applications not time necessarily to the things that I've talked about so far here the first is this concept of what I like to call lambda based monoliths this is the idea especially with things like API gateway where you have a single API gateway you've got many API calls and you're going to put all those things inside of a lambda function because it's nice and easy and simple this is a horrible idea um you do not want to put all of those methods and entry points into your API inside of a single function and so while Randal said that we shouldn't use this word I actually use it pretty regularly so I Randall but this concept of nan-oh services and basically what you have lambda functions is for many of our customers even internally in Amazon so typically in the tens to low hundreds of lines of code compared to what I see with typical micro service based architectures where you're maybe in the high hundreds of thousands of lines of code compared to mana lists which are in the tens or hundreds of thousands of lines of code you really want to have really small functions it really plays into the overall performance of your application as well as kind of the general management and and developer lifecycle of working with lambda so basically kind of rule of thumb for this is unless your functions share a handler split them into their own lambda functions files or binaries another option here is obviously to make use of language specific packages so NPM pip maven NuGet for c-sharp you know don't have duplication code inside of your code obviously that's that's a very bad practice and then unless you're if your lambda functions don't even share event sources don't put them in the same repository don't put them in the same template Sam template break them completely out so if you have code that gets in vocative I can use this and you have code that gets invoke ated by s3 maybe there's some business logic shared between them those are two separate applications they consume a shared library break them up treat them independent don't make them be the same thing now and then lastly if you are using tools like Sam or CloudFormation to deploy your code always validate that yamo typically locally and then in your CI before you go to the backend CloudFormation is kind of very specific tool it's not going to initially always catch everything that you're doing that's improper and so with Sam local now we have kind of a more advanced validate command even just a basic JSON lint Auriemma lint is actually really really powerful so talking with your Pugh power here so with lambda for those of you who have used it you know that you kind of have a single knob to turn for performance and that's the amount of memory that you'd give your function well actually what we do is we actually proportionally give you a certain amount of CPU or networking capability to that lambda function so crank that knob up you get more CPU one more networking pretty straightforward however people don't expect is that by turning up that memory number and getting that CPU even though it technically costs more as you're running the function it doesn't mean that you overall are going to pay more for it so in examples where your functions are much more CPU intense so if you're doing machine learning you're doing some sort of hardcore batch processing maybe doing image transforms or you're doing video rendering type things well we've actually seen is that more memory is actually cheaper than less memory because you're going to get it done much much faster and so this is kind of a canned example here where we calculated over a thousand times all the prime numbers less than a million and what you see here is just between 128 and 1024 that the timing obviously went way down I mean to the point where we almost at one tenth of the time by multiplying the memory by ten and the price is actually just a little little little a little bit more and so this could be one of those things that people miss especially when they're thinking about performance of their lambda functions often see well how much memory did you could figure for it and everyone likes to stick with the default 128 because it's the cheapest but again it may not be the most efficient burst that cost given the fact that CPU was in a line the same way so if you've got functions try cranking the CPU up see what happens looking at the duration of that function and seeing how that would play into the cost impact okay how many of you have heard of cold starts before how many of you have spent way too much time thinking about cold starts for and lambda functions call people cool so cold starts this is one of the most I think kind of over worried about thing when it comes to lambda and service next to the ghost of lock-in which I don't talk about in in my slides here today but is also a fake thing to worry about so I with cold starts basically the story is when you have a function that's invoked basically there's no compute infrastructure that exists for the first indication of it and so we have to do is take your few resources align them with your account pull down the code run that code inside of a container on the host it's kind of a low-level operating system container and then invoke your functions so the first time this has to happen on individual host it can take a little longer because we have to do basically all of the steps that I mentioned here up until we get to the green where we start your code now after that container is up and running it's considered warm and so subsequent invocations of your function will typically go to a warmed function if one exists across all of lambda globally cold starts are less than a quarter of a percent of all invocations so millions and millions and millions and millions of invocations less than a quarter of them are actually impacted by cold starts so what that means is that all the rest of them are landing on warm containers so you don't want to think too too hard and worry too too much about this but there are some best practices that you can follow to help even reduce your cold start penalties further so one is to separate business logic from function signature basically you have your Handler and lambda pull all your active work that you need to do your actual business logic outside of that handler now why is this important sometimes you could be doing things and actually talk about someone next slide such as creating clients to services for example things like connecting to database backends loading up variables like you know kind of global variables another is that the handler is basically always more or less gonna be pulled into memory and so the more stuff that you have inside of your handler the more that you're putting effectively work on the underlying host that's doing stuff so by pulling that functionality out and then maybe it's usage specific or case specific you can actually kind of streamline your functions a little bit so just mentioning things that load into global scope or say connections that you need to make the databases if you do those outside of their handler that will always happen during that cold start time and then ideally when you're warm you no longer have to do that so that's another way to speed up your lambda functions a bit so we see here an example where I'm going to connect to s3 I'm going to connect to my database and then potentially check inside of my handler if I'm not connected to reconnect but ideally that shouldn't happen too often so again cold starts ways to minimize them but again not the big boogeyman that everyone makes them out to be now one thing you do want to think about is if you run your code inside of the V PC so V PC or virtual private cloud is a networking construct that we have here at AWS and so when you run your lambda function inside of a V PC what we have to do is once we've gotten that host for you that runs your functions on it we have to create an attached what's called an elastic network interface and this allows your functions to talk to the network inside of the V PC now you should only only only ever launch your resources inside of a V PC if you need to talk to something else that exists inside of that V PC by default we run lambda functions in a more or less secure networking way there is no way to connect to them over the network there's no such thing as SSH into a lambda function or pinging a lambda function or port scanning a lambda function it just doesn't exist and so from a network security standpoint you don't ever have to think about that again unless you want to put them into your network even in this model there's still no way to really ports can lambda function you're not going to get anything interesting out of it but again if you don't need a V PC don't use the V PC another way to think of it it's kind of a very simple logic diagram here should my lambda function be inside of a V PC well again do I need to talk to anything inside of a V PC no great don't use a V PC yes ok does my function need to talk to something also potentially public on the Internet and something inside of my V PC which is not necessarily a good practice you should typically break that business logic into two different functions but if this is the case then what you need to do is have some sort of nat gateway whether it be our nat gateway or some sort of other proxy that that could be used as a gateway enabled so there is no way to put a public IP in a lambda function today and so that is kind of a requirement so again this is one of the other things that will impact a cold start but once your function is up and warm not something you really need to worry about so let's say that you're still really paranoid about cold starts you have a really infrequently used lambda function and so you could potentially be expiring out your compute infrastructure behind the scenes another option that people do and I really call this a hack is basically set up a scheduled cron job using cloud watch scheduled events to run some sort of ping logic that will basically invoke your lambda function for you now one common mistake that people make is they will actually invoke the actual service via its invocation model so you'll call API gate where that will call Canisius or they'll put a fake object in s3 don't do that should you just wasting money on something that you shouldn't so you can directly invoke lambda functions via API you can pass in a somewhat contrived payload that basically says hey this is a scheduled payload it's got some nonsense for ping inside of your lambda function have a function inside of your codebase that handles this again outside your handler and use this to keep your functions warm and basically the rough rule of thumb is you want to do this every five minutes for functions that are outside of EPC and roughly every 15 minutes for functions that are inside of EPC now we don't officially document this because at the end of the day we're doing a pun of really hardcore stuff to try to remove cold starts from being a problem inside of lambda and we also do a lot of stuff to tune how long we keep warm instances around so we don't have a real formal guidance on this it's shifting it shifts under a lot of different things it's fairly variable but the the 515 rule works out pretty well and again if you really are concerned about cold starts and you actually have seen evidence of them then the cloud watch hack can work so we've covered a whole lot of different stuff today between kind of CI and CD best practices environmental best practices variables things like parameter store when you use multiple stacks when to use multiple accounts when not to performance tuning terms of things like again CPU more memory equals more CPU more networking where you should put your code inside of your function invoking things outside of your function and how to do kind of monitoring metrics and logging the great news is is that if you forgot something that I said today you can read all about it on this page so AWS Amazon comm slash service kind of our key landing page we've actually got a lot of content that's going on this real soon in terms of other talks and our key blog posts and stuff like that we recently just in September launched a sub page off of this for developer tools so kind of all the key frameworks that we see our customers using our mentioned off of here all of our CI and CD tools documentation for things like Sam and so forth and so you can really kind of dig in pretty deep again my name's Chris Muntz senior developer advocate for server lists at AWS really thank you for coming to this talk hopefully you had a chance to learn something thank you to the folks from DevOps for having me this is a really cool event I'll be sticking around out in the hallway here for questions and I'll see you around feel free to reach out if you have ever any questions about lambda server lists anything any WS feel free to yell at me on Twitter I'm happy to respond and thank you very much [Applause] [Music]

Channel: Devoxx

Views: 14,247

Keywords: DV17, Devoxx

Id: _mB1JVlhScs

Length: 45min 47sec (2747 seconds)

Published: Thu Nov 09 2017