AWS Lambda in Go - Getting Started With Lambda

hey everybody what's going on welcome back to the channel um today we're gonna start up a new series uh working with aws lambda in golang so we are going to talk a lot about lambda in this series um we're gonna give a bunch of different examples on how to work with it so in this first part we're gonna kind of explain what it is what you can use it for why it's a good idea and some of the performance characteristics of lambda and then we'll set up a very basic function in the awms console in subsequent videos we're going to start using um a bunch of other aws resources to trigger our lambda so the great thing about lambda is that you can use you can trigger your lambda from over 200 aws services and follow-up videos for the triggers we'll be using we'll probably run through and we'll give examples how to trigger your lambda through like a queue for example so if we put if we make an sqs just a little simple queue we can send events to the queue we can trigger our lambda to process those events off the queue here's an example that they that amazon is showing here where they have a bucket and that's three bucket so file uploads like images videos code anything like that can also trigger your lambda um other cases may be kinesis so you can have large streams of data being adjusted into your system stream processing yeah so you can have like really large data sets being ingested into your system and lambda can scale up and trigger to handle all of that parse it out and send it to any kind of back end you want you know maybe um to your like you could send it to splunk for logging or if you have a database or something along those lines um your your data comes in through through a queue through s3 through kinesis many many services available and then you can just automatically trigger your lambdas they'll scale up they'll handle all of this to meet your needs process your data and then you can do whatever you want with it so that's kind of the goal for this series we'll do a very basic lambda setup for this video and then in subsequent videos you can kind of pick and choose what you like to see so if you want to just learn about sqs triggering lambda via sqs watch that video if you want to see kinesis watch that video if you want to see s3 um watch that value yeah so hopefully you can take some stuff away from this and let's hop right into it all right let's hop over here first thing i want to talk about with lambda it is serverless computing so that means there's no server running 24 7 behind the scenes lambda um when you trigger a lambda amazon basically starts it up it boots it up it runs your code and your lambda takes a little bit longer to run for the first time so there's there's something called a cold start and there's something called a warm start so a cold start amazon has a time limit on how long an idle lambda can be running so for example um you trigger lambda once and then no other no other requests come in that trigger it again so maybe five ten minutes later to save money to save resources amazon uh closes down your lambda shuts down deep provisions removes all the resources um and then the next time a request comes in it has to reboot restart and get your lambda spun up and running again that's what's called a cold start now for example if your lambda is already running and there's just request after request after request coming in that's what's called a warm start and your lambda just stays running as long as it's getting requests within a certain time period i think that's variable i believe you could set that we'll talk about that a little bit later but it's around three to five minutes as long as it's getting a request for three to five minutes or so you'll be hitting what this warm start is you'll have all your code running your lambda's already good to go so it's a lot quicker so i'm here on medium this is uh alexander flitkin's page he just puts a page together comparing the performances of those cold starts among different languages so let's see we're going to be working with go so that's kind of all we really care about here he kind of talked about how we set it up and here's here's uh here's this chart here we can take a look so over here these are all compiled languages like java grailvm.net go rest you get an executable right you compile you compile the file your code and you get an executable file that you then run on the right here these are all interpreted languages python node.js and ruby um there's no executable generated the interpreter the python interpreter or the javascript interpreter or the ruby interpreter interprets your code and runs it um so different that's why you see some different performance characteristics here so for cold start um we see that go is actually pretty quick among the compiled languages it's second place on all these rows it's second place losing out only to rust so for a lambda go is actually a pretty good choice it's a lot faster than uh net way faster than net significantly way faster than java as well grail vm it's slightly faster so it's a great choice to use for any kind of lambda development all right so let's talk a little bit about what exactly makes up a lambda you can have a lot of complicated code structures for lambda you can be you know many folders many packages you can go however far you like right if you have very very big team or a big company or something like that sure put in the work to structure your lambda properly but here we're looking at the lambda docs here on amazon and we see just their basic example their basic outline and you really get a picture of how simple the code can look here so basically inside of your main function um you just start the lambda you call lambda.start this comes from amazon's built in amazon's package that they've developed and released for us to use and you can call lambda.start and basically you just pass in a handler so lambdas get triggered by events and you build handlers to handle those events so here you see this handler is built out just has a context right so maybe this context is for the lambda's life cycle if the lambda crashes or gets terminated or the any kind of request gets terminated coming in this context would be closed and you would save some resources there the request would stop immediately we see the second parameter here is the event this event is defined up here um we define this event right we define what we're looking for in our lambda payloads and it's yeah it's up to us to validate that and make sure everything's okay and then you can return pretty much whatever you like any kind of types in and any kind of types out so here they talk about some a little bit more detail here here's a similar example this is returning a structured response instead of just like a basic string and here they list out valid handler signatures so what can this what can this handler look like um you see there's a huge huge option um just an empty function you can return pretty much whatever you like an error and then whatever type out you like and you can pass in whatever type you like so another piece of information here is global state so this is another important one to talk about here before we start developing so we see two functions here we see main that's important and then we also see init that's important as well so with lambda here this init function is executed when our handler is loaded so this is before this is executed before main is run so any kind of global state we want here is best to be set up so for example if our lambda was going to connect to a database we would want to do it in init you'd want to have your database object outside here somewhere defined outside of our defined outside i'm just in global state and then you would define a nit right here to set up the db connection all that stuff so here for example they're getting a bunch of s3 objects they're pulling objects from this bucket here and they're storing those in global state so this global state remember we talked about cold start hot start right cold start warm start so this global state is reusable between lambda requests so you make your lambda spins up you provision the resources we run this init function and this global state then is defined so now each time while this lambda is alive right while this lambda is alive and it hasn't it hasn't shut down and closed down right so while we're still getting a bunch of requests like a request every couple seconds each of those requests will be able to use this same global state so you can kind of reuse for example if you had a db connection going on you could reuse that connection here between all of those requests so this global state is pretty powerful um it's pretty powerful you can save a lot of resources you can save a lot of connections to your database basically you can use a single one um or a lot less right instead of having to make a connection for every single request we can use this global state so these are all things we're going to be walking through and talking about um that's kind of the outline of just how basic a handler is so let's hop into golan now and let's start uh writing some code here so i'm in golan's and i have made just a basic project we're using modules go modules and i've just called mine lambda base so let's go ahead and get those packages let's go get those packages here all right so the first module we'll get here go get is the lambda sdk all right so we pulled in the lambda sdk here and let's make a main file main.go so for us i want uh structured logging and our global state so let's define a nit here and inside of here we're gonna set up our logger and let's define main here inside of here what was it lambda.start recall and then our handler okay so let's define our handler for now just destruct and then we're going to take a context and then just an event okay and for us let's just return an error okay so this is our basic lambda we still need to define this logger and we'll set that up inside of a nit to kind of give a demo of global state but all we're doing um inside of main is just calling our handler this handler will return no we'll add in a line just log we'll add an align to log this event here um so let's get our logger set up here so for logger and go i'm a big fan of just uber goes zap this is put out by uber and it's a great structured logger relatively simple um it's pretty straightforward to call this here has really good performance number one but number two you get really clean logs they're kind of json formatted and it's a lot cleaner to look at than just doing goes built-in log.printline or goes built-in format.printline so let's go ahead and pull this in okay and now let's bring this line over here so inside of knit we can call this that'll import we're gonna so this defer logger sync i'm not gonna talk about what defer does it calls it at the end of this init function but let's not worry about that okay what does this return returns a zap logger and i believe it was a pointer okay so we make our new production here um and we assign our global state logger to be equal to that one okay for now that's all we're going to worry about so now we can actually access this logger logger dot info and we'll just okay so if you're not familiar with uh with this uber zap format here so we just defined a message in it it has error it has warren has a lot of other functions you can log at and then zap.any just we want to log something in addition to it so we'll see also in this structured log we'll see something we'll see a key with event and the value will be the kind of pretty formatted json formatted event object that we have here so let's just put some field in here uh yeah sure name um let's just make it a string and define what it looks like um in json okay there's our lambda um let's talk about how we deploy this now okay so i'm on the aws documentation page for aws lambda we're under working go and we're at deploy zip archive so when you write your code for a lambda you zip it up into a little bundle and you upload that bundle you can either put that on s3 or you can directly upload that to your lambda so here's their directions for mac i'm on mac so that's what i'll be using they also have windows if you'd like so i'm just going to write a script that does all of this here so kind of what they're doing downloading dependencies we already have those compiling executable building a couple different options to build and then you finally zip it up so if you were part of uh if you were doing this deployment if you're deploying your lambda as part of some kind of deployment pipeline like github actions or bitbucket pipelines etc you would obviously need to get your dependencies right you need to do this stuff too but we're just deploying from local so we don't need this top one all right copy this let's just make a script here new file just say deploy or generate zip sure dot sh let's paste this in and the second command it's just zip all right um let's see sh generate.zip okay so we need to do one thing uh go build needs this package to be called main so let's change that over all right so we run that and that generates our executable here and also generates our function.zip this is the the output that we're going to upload to our lambda let's provision our lambda on the aws dashboard inside of lambda we'll just hit create function [Music] um we'll do from scratch here and let's just say lambda base our runtime we're going to choose go one point x permissions that's fine um we don't need any uh networking yep we're all good and let's just default execution role that's fine um if we want so for example here if you're working with like a database or any other resources you can use existing roles or you can create roles as well right if you want your lambda to have access to write to your db or write to some other service or use something from another aws resource you can set your policies right here for us we're not doing that yet so we'll just do basic lambda permissions and let's create all right so here we see we've successfully created lambda base and here's all of our all of our details here so this is where you can kind of add triggers to your lambda so in the future videos we'll be putting in uh different resources here these are a bunch of the resources we can use you can trigger from kinesis like we talked about api gateway you can use a lambda as kind of a rest service as your back end if you put it behind an api gateway http requests come into the gateway those trigger your lambda and you get those requests inside the lambda and can process them yep so we don't need this for now all we want to do is upload our code source so let's upload from a zip file you can also set this to upload from s and s3 buckets so part of your build pipeline would probably um two ways you could do it one you could use the aws console sorry the aws um cli the command line tool as part of your build pipeline so you build your your go um your go project you zip it up and then you can use the aws cli inside your pipeline to deploy directly to this lambda um other options would be you could in part of your build pipeline deploy or save it to a s3 bucket and that would uh automatically update this lambda for us we're doing it real simple uh right through here so i'm going to go in and find that file all right so i selected function.zip from my desktop here and we can just click save that's going to upload our code okay successfully updated function lambda base uh let's trigger a test event we got a test here and um we had i think name it was let's remove these okay so new event and we can hit test here and we can see all right okay so no hello does not exist which handler to invoke by setting the handler parameter okay so let's set that parameter let's try setting it to main here okay your changes have been saved and let's try running this test again so we have name okay all right execution results succeeded so that time we um were successful here and here's our log output um yep so this you get these always with lambda start you get the start and the end of the request that passed through the lambda you can see the execution duration that it took our lambda function to run our code what we were billed for aws rounds up you can see the memory size how much memory we actually used and then how long the whole thing took how long a net took to set up so this is pretty much our logger setting up that took a lot of time and then between here we see our logs that we put out so this is the uh zap logs those kind of fancy structured logs um we see we get the level timestamp that it's logged at the caller the line number and we have that message in my handler and this is the event we received so it's uh object key name and there's our value okay so that's uh that's how easy and basic and straightforward it is to set up a working lambda so that was part one of our series um future parts we're gonna start um adding in triggers so that's where the exciting part comes in uh stay tuned uh please make sure to like comment and subscribe to the video really helps out the channel um lets me know i'm doing something right here and leave a comment below suggest any resources you'd like to see me work with as a trigger thanks

Channel: devtopics

Views: 3,016

Keywords: microservice, golang, microservices, heroku, api, backend, software, docker, docker-compose, docker compose, deployment, deploy, postgresql, postgres, go, aws, lambda

Id: tgrV2FmVyZY

Length: 21min 36sec (1296 seconds)

Published: Sat Oct 16 2021