Develop Lambdas And Debug Them Locally Using SAM

hi and welcome to programming percy today we will be looking at how to develop and debug lambdas locally using sam lamdas is an aws service which allows you to take small pieces of code and deploy them to the cloud and execute them based on a number of events such as an api call or an sqs event or a s3 event and many other aws services however debugging these lambdas can be tricky sometimes because they are meant to be run in the cloud and sam is a aws tool which stands for serverless application model and it's a toolkit that helps us develop and how use these lambdas on your development machine instead of the cloud which is great so during this tutorial there is one thing i will kind of expect you to already have and it's an aws account and also the aws command line tooling installed and configured and if you don't have that there's a link up here which you can use to find out how to set that up so sam about sam who is sam sam is a framework that allows us to develop cloud applications and like cloud application is many cloud resources and one lambda is a resource so you have one lambda as a resource which builds into a application when you have many and we will be beginning by learning how to develop these lambdas and also how to debug them so sam is a command line tool so it's ide agnostic i know many people think you need visual studio code for this but you don't you can use any kind of ide that you want i will be using this code in this tutorial however so let's begin by downloading and installing sam so you can download sam by visiting the website or also using vegas to download the zip file which was is what i will do so let's we get this fight and don't worry about copying the file the path to it you can find the url in the description so once we have that it's time to unzip the zip file and we will place it in sam installation folder so there's a bunch of files being unzipped you don't have to care about them there's only one file that we care about and we it's the installation file so we run that as sudo so the sam installation folder locate this install script and trigger that as sudo now i already have sam installed so mine will tell me that i can update is that but if you don't have time installed it will install and you can verify that sam is installed by doing sam dash dash version and it will print the current installed version of sam so now that we have it we can remove all the zip file we can also remove the sam installation folder we don't need it anymore if you're unsure you can verify by calling sam version again so it's running great let's begin developing our first lambda it will be a very simple lambda and in this tutorial we will be using golang but you can use any language that supports lambdas you can find all the supported languages in the aws documentation and i will put a link in the description if you want to use another language you can go to the chapters in the docs called working with node working with c working with whatever you want but let's go ahead we i'm i'm in my sam demo folder here which i have created which is a blank folder and we will go ahead and create uh the module go light module so we will go mod init programming percy which is my module and i will also create a main file which will be our lambda and i will also create a template dot jam we can leave yaml file for now it's the sam jamel which we will use to define the sam cloud architecture but our lambda will be placed in the main.go so let's go inside of it and see how our lambda works we will have the main package and the main package is just because this lambda will be a single binary which you can execute so let's just leave it at that and we will have a input to our lambda and that input will be a event and usually when talking about lambdas you talk about events because you talk about other aws services that can trigger these events so let us say that the trigger of our lambda will input his or hers name as an input so our struct will accept a json with the name and we also need the actual lambda function so let's call it lambda handler which is a bad name but leaving that that so lambda handler and our lambda handler will accept the event as input and we will output a string and an error so what we will do in our lambda is very simple we will simply prefix then inputted name with a hello so we will do sprintf hello from lambda and the name is contained inside the event event.name and we will return this to the user now we also need to make a main function because this is the main packaging go so we create the main function what you will do in the main function is that you will use the aws sdk for triggering lambdas so in this file it will be lambda dot start and we will pass in the lambda handler now you can see my ide wants to pass the actual function call so it triggers lumbar handlers and returns the output from it but that's not what we want to do we want to start the lambda and input the function without calling it as input so whenever all lambda is triggered it will execute the lambda handler function it won't trigger the main function it will trigger the lambda handler and go ahead and import that and it will complain a bit because we need to fetch the sdk so i will do go mod tidy and it will download and install great this is our first lambda it's as simple as this now before we begin executing our lambda because we can't execute this locally because the lambda.start prevents us from doing that so let's go ahead and see what we can do with sam so sam is a template and built cloud architecture tooling and what that means is that we write our architecture as code so infrastructure as code and actually sam is a extension of cloud formation if you're familiar with cloud formation this will be very familiar so inside inside the template.jamo we will create our sam application so let's create and the first thing you want to do is you want to specify the actual sam version to use for the templating engine and this will always be 2010 or 909 i know this is very old but that's the only accepted value for now according to documentation so leave it as that you will also specify a transform and it will be aws colon call serverless and this this states and again it will always be these dates so whenever i start a new sam i just copy paste these in because they have been the same for years now let's begin by adding a description so this is a description for your whole cloud architecture so in this case it's a demo of sam and we try out sam just passing whatever the next thing we want is something called globals and globals are interesting because this is how we define configurations for all of our resources now in our case we want to say that our lambda functions cannot execute for any longer than five seconds so what we do is we say globals we say apply this global to function and we specify what we want to configure so the timeout is for five seconds there's a bunch of globals that you can configure and you will find it in the documentation again i put all the documentation links in the description next up it's finally time to start defining our application and the way you do that is again by specifying the resources so let's go ahead and add resources and our resource will be named hello gopher because it says hello to the gophers who calls it and you specify the type all sam resources will have a type defined and the type is what kind of resource it is in our case it's unloaded but it could be an sqsq it could be an s3 and but let's leave it at the lambda for now and alumni is a aws serverless function that's the same thing so this is the way to create this is the way to create the lambda now a resource also accepts something called properties and properties are configurations specific to the type so depending on the type you set you can define certain properties in the lambda case we will need to define the code jury which is the path to the code that will be executed and we go ahead and put in code yuri and in our case our code is inside the root folder so you can just leave it as this however this could also be an s3 path or a path to a zip file we also specified a handler which is basically the name of the lambda now we also specify the runtime which is the golang version to use when executing the lambda and we can specify the architectures to use in our case it will be x86 464. now this is it for the lambda to start out we specify that we have a function type and we set the code unit to point to the code we will look it into how to change this later because having the lambda in the roots usually isn't what you want now we can actually try running this so you go into the command line tool and you do sam local invoke which is the command to execute the lambda locally so sam local invoke will actually invoke this lambda that we have specified in the template so it will look inside the template and see we have a function it's located in the root and executed now we will see an error and this is to be expected because we have only defined the resources we haven't built them yet so let's go ahead and build them in the next step so to execute it properly we need to build the sam application and the way to do that is very easy you go ahead and type sam build and it will compile your code into the lambda and you will see a bunch of output here saying that you can now deploy it which you could but since our application really isn't that great yet let's let's not deploy it but you can find this folder called aws sam which has a build folder and you can see our lambda is actually located here hello gopher and it contains the binary now this is built using the correct architecture the correct version and the path resets so now we can go ahead and invoke again sam local invoke to execute it and go ahead and pass that and you will see it runs hello from lambda one thing is missing it's the name so remember that we built a event as input we actually have to pass something into this lambda to work now we don't validate the input in this lambda but in a real number you would but let's look at how we can insert data into the lambda as well so same local invoke contains a flag which is dash e which stands for event this is the way to pass in information now to pass in data you can create a file and we will name it event.json and event.json will contain our json formatted input which was name and some name my name is percy so i'll leave that now we can pass this event file into our lambda when we execute it so we can go ahead and do sam local invoke dash e because it's the event flag and point to the event.json now when we run it you will see it executes and now hello from lambda percy this is great that's that's what we wanted now this is the way we can pass in data to lambdas which is great and sam locally invoked takes in a bunch of other flags and we will look into the dash d flag later which is the debug flag which you can use to attach a debugger also you want sometimes to invoke a certain lambda because now we just said sam locally invoked and it worked but if you have multiple numbers that won't work because it will execute the first one so to do another invocation and specifying the lambda you do some local invoke but then you specify the lambda name which in our case was not hello gopher actually it actually tells you so this is great about sam it has our template file so it actually knows the name of the resources so you can see here it tells me to then it corrects me basically saying this is the name and we can do that instead and we should see it execute so that's great because we can now have our lambda executed correctly now sometimes you have other services who executes lambdas and you might want to test that locally as well but you don't want to use the command line tool for that you don't want your programs to execute sam local invoke and to manage that you can set up a fake aws lambda api not on a http api we will look into that later but you can do sam local start lambda and it will start up a lambda api and what that means is you can now execute lambdas as you put a regular aws um lambda so if we open up a new terminal we can use the regular aws command line tool so not sam but aws and do sam lambda invoke and this is the way you would invoke a real lambda from from aws so the function name is the same as when we did the invoke so hello gopher and the end point is not on aws anymore but rather hosted on this url you find here so instead of going to aws it will go to your local machine so the end point is http localhost 3001 and we also want to specify the payload and i saved so the payload should be the json payload as same as before so we do name and my name is again percy so let's close up the json there all right aws lambda invoker also wants an output file so we need to specify response.json now it will actually invoke my lambda if we switch terminal here you can see that it evoked and we can go in here and see the output it's a status code 200. and if we open up response.json hello from london person so this is great and this is very useful if you have say a react website which uses your lambdas you can just point them to localhost instead so many times you don't want to have a lambda api exposed many times when you develop lambdas you want a http api exposed to other services and we can do this using an api gateway which is a aws service which exposes your lambdas as http and that's really easy to add using sam so what we will do is that we will open up template.jaml again and in here we will define that this lambda or hello gopher lambda is expecting to be triggered by a certain event type and you do that by stating events now you could have multiple events in this case we will only have a http event trigger it and we will call this resource hello endpoint this is our http input and we also have to state by again using type that this is a api now when you say that this is an api as we do here sam will automatically know that this is a api gateway that needs to be created so if i were to deploy this using sam it would actually create and set up the whole api gateway for you and you can also add some properties to this and because we want to configure some things such as the path where you will call the api in our case we will have slash api slash hello gopher and we will also want the http method declared and in our case we want to have posts so this simple few lines will allow sam to create an http api for us so before we move on we will need to modify the lambda somewhat because when we have a api gateway in front of it it will modify how the input event in the lambda looks like now the aws sdk comes with a events library which you can import and in there you can find all the different events that exists from different services so if we go ahead and do and that our input event will be an api event instead so we go ahead and go events dot api you can see here how many there is there's a ton and this is a sort of gateway um then there is a ton of different events depending on if it's a dynamodb event s3 event iot event kinesis kafka there's a bunch of predefined events now we'll look more into other events later so let's go ahead and add api gateway proxy request because this is how the request will look like when we get it you can see there's a bunch of things the user input will be inside the body so we will need to handle that but before we do that we also want to modify that the output from our api will be an api gateway proxy response and the reason why we did it you would you actually don't have to change the event types you can make it work anyways but it's pretty nice because aws will append invocation ids and stuff into the requests automatically so which can be very nice to have um so let's go ahead we will need to create a input which will be r custom event type and into that we will marshal the user payload because the input from the user will still look the same so let's go ahead if error and we will unmarshal the json and remember that the input from the user was a string so we need to convert that to a binary and the payload was located in the body and we want to marshal into our input and we need to check that the error isn't new now if it is we will return the error message as the body again it's a string so you need to invoke the error function and then distort this code should be http internal server error and we also return an error in our case let's return error yeah so now we can do the same thing as before but instead we want to grab the name from input which is our event struct and we also want to change the return type it should be an immense api response and the body should be our string that we formatted and start this code will be http starters okay so once we've changed this we can go ahead and retry the lambda as an api instead so we can go ahead and go step build which will rebuild the lambda because we changed the code and then you can go instead of running sam local start lambda which started our alum apa we can now run start api which will start an http api for us so let's go ahead and run that start api and we can see that it's on port 3000 this time instead to not collide with the other and we can go ahead and do a curve to verify that everything is working so we will curl to localhost 3000 and remember we added the path to the api before in the template here we said that the api should be hosted on slash api slash logo so we need to add that hello [Music] and we also need to pass into data and the data will be the same as before so the same name jason and again perseus name and now when we trigger that we should see a response so let's think hello from lambda percy so now we're hosting our lambda as an http api and that's amazing so many times in our apis we will need to use environment variables and sam comes with a way to handle environment variables because now we have one lambda which we can configure pretty easily but you might have many i've seen like a ton of lambdas so handling the environments can be quite messy but sam helps us again so what we can do is in the template we can actually add all available environment variables and remember that you need to add them to the template to be able to change them later so if we define my cool variable maybe not so cool the lambda will now have x access to the environment variable mycool variable but and that's trivial but when we add it in the template we actually tell sam that this environment variable exists so you can modify that one and what that enables us to do or let's first add the output to print the environment variable so we can actually see that we can change it so let's go ahead and add here the variable to the output then variable yes and let's print that as well and let's get it my cool variable so this will add the variable to the output and we set the value to maybe not so cool but we can actually change this now using a small trick let's add a new file called environments that's okay so and this is a file which we can use to configure multiple lambda resources so we start by saying inside the json which resource we want to configure so in our case we want to configure a logofer and we want to provide my cool variable with a new value of now it is much cooler now if i were to change test variable here this would not apply because test variable is not defined in the template remember if it's not defined in the template you can change it so we can only change my cool variable for now so let's leave it like that and what we can do is we can take this environment file and pass it into the sam local start we can also pass this environment flag into the deployment so that's one way of doing the deployment but let's go ahead and do we can do sam build because we updated the lambda so and then we can do sam local start api and we can use the dash n flag which accepts a path to a environments file so if we go ahead and run that everything will run as usual and we can curl it again and this time it will print the variable is now so much cooler so that's how you can easily manipulate many lambdas at once using environments so one other common approach to lambdas is that the lambda should be triggered depending on what another aws service does such as one very common use case is that you might have an sqs queue which publishes events and you want your lambda to trigger on now we can't make sam listen to an actual sqs queue which is sad but there's ways to manage and the way you handle it in sam is that you can generate events this will be fake payloads that you generate which you can push to the lambda so it will work exactly as a real sqs event were triggered but you don't need sqs to actually trigger it and to try that out let's go ahead and add a second lambda resource to our templates and don't worry it won't be too long it's very short so let's call it sqs lambda just because our naming is very bad and again it's a lambda so you go ahead and do the serverless function and we need the properties to start it and let's go ahead and add the kojiri now kojiri points to the lambda file that should be executed and the last one was in the root which is bad i usually create a folder named lambdas or something so you can store your lambdas in there in a more structured manner so let's go ahead and add the sqs lambda again this is you could do lambda and then a folder inside there named excuse lambda or whatever but just to show you how it works so we need to point the code here to that and again the handler will be sqs lambda we need to specify same as before the runtime and let's not complex it so let's leave it as before and we have our new lambda but we want this lambda to trigger on sqs events so the same way we did with the api we can do the same thing with sqs instead so the way to do that is again we use the events actually you could put multiple events here so you could have this hello lambda trigger on sqs as well you can have multiple inputs but let's create a new one to see it ruled so we have sqs events which is our name you can set anything there the type is important this type is an sqs event and you have to specify the properties and in the sqs type you only need one and it's the name of the queue or the rn of the queue so let's go ahead my awesome cue now this is everything we need to set up our new lambda but we actually need to create the lambda as well so let's go ahead and do that because i want to show you how we can generate these sqs events so we have here let's go ahead and create a main.co and it's the package paint and this will be a very simple lambda so we have our lambda handler and the input should be an sqs event which is predefined in the sdk and you can see that it's we can actually very quickly review it it's an array or a slice of sqs messages and it contains a bunch of information so marking this can be troublesome or annoying but happily we can generate this output for us so let's make our lambda return on error and we will simply log the event to the standard out so we can see that the event is generated and we will return new and then we need a main function and in the main function we should do lambda start or lambda handler and again my id always wants to execute the function instead of passing it as an argument so here here is our sqs lambda it's not much but it's it's working so what we can do now because you don't want to create this big json structure yourself let's make sam do that for us so what we can do is we can do sam local generate event if you trigger that you will see all the supported services so depending on what what aws service you have you can actually create events for most of them and let's go ahead and see if we type sqs which which is the service we want to use it will tell us that there's one available command which is receive message hopefully we will see send message but it's not a big problem for now so let's receive a message and we will specify the body the body is the user payload so again always body is the user payload so we want to modify only that because all the other data is sort of this generated aws data or metadata about the invocation so let's go ahead and say that this is my own event payload now i misspelled the command let's receive oh there you go so now you can see that we got a big json payload outputted which is what the real aws sqs will output and instead of copying paste this we can actually make sam generate this and we can pass this from standard out into the lambda so if we pipe this we can do sam local invoke and we want to do the dash e because this is the input event remember before we passed a path to the event.json but this time we will pass a single dash which says the payload will come from standard in and the standard in will get the output from the generate message command so we generate the message and we pass that into the standalone of the lambda and we also need to specify which lambda will receive the payload and it will be our sks lambda so if i go around ahead and run this oh we need to build do sam build and it will build the application and now you can run the same command so let's see it is finding the lambda and we can see something was logged to standard out and you can see here that it's the metadata and also our payload so this is how you can generate events and pass them into the sam lambda so we finally know the basics of sam and it's time to start at looking at how we can debug our functions which is a very important aspect and what we do is all the commands to invoke alumna in sam so sam start lambda sam start api sam invoke all except a d flag which is the debug flag and what what will happen if you pass the d flag is that you will specify a port and the lambda will know to host a debug on that port and it will halt all executions until a remote debugger is attached so let's take a look at how to do that it's very easy actually so what we do is we need to specify a few [Music] options you need a debugger installed because the debugger should be installed on your host machine in golang we often use delve so you should go ahead and install delve if you're trying this with goliath and i i'll put a link in the description on how to install it and you also need to set your gobind so you know where the delve binary is installed uh so let's go ahead and start invoking a lambda we will invoke our sqs lambda and we will use the d flag and say on port 8099 you should host the debugger and we need to tell the la sam application that the debugger to use is located in a path on our host machine so we use the debugger path and we set that to i have my goblin in home percy go bin so that's the path i will use and also i want to use dell version 2 so you should also do that because that's the new version and you do that by doing debug arcs and then passing the flags to apply which is delve api equals two now if i if i execute this command my lambda will start up and we can actually execute it we will see the lambda startup and it will tell us a few things and it will stop because it's now waiting for a debugger to attach before continuing so let's attach a debugger now you can use any i know golan has a remote debug option and you can do the same in this code so let's go ahead and create a new file or a new folder named dot vs code which is the place you should put your debug configurations in ds code and we should put it in launch.json the naming is important because this is how the these code editor knows how how to locate the files and this is a [Music] this is how you specify debugger and debugging options in this code so let's create our specific one and it's debug with sam this is the name so you can put whatever you want and we want to tell the debugger to use golang and this is important the request is not a launch request it's an attach request because we want to attach to a debugger and we will set the mode you can set either local or remote and we want to set remote because we are attaching to a remote debugger it's actually still on your computer but it's on the remote docker which is not on the same network so go ahead and say the port which is 8099 make sure you put the same port which you used to the d flag in the sam local invoke command and the host in our case it's localhost so once you have created your launch json with this configuration you can go inside here in the debug options and you should now see debug with sam up here and you can execute this debug which will connect to the sam lambda which is running but before we do that let's go ahead and place a breakpoint because if i execute now we will just loop through and log the event but we want to place a debug point you do that by setting a breakpoint here now whenever we execute our program will stop here so let's go ahead and run this and now oh yeah i have a visual bug it picks the wrong main file but now we can see we have halted the execution and we can see the input event here which is nil because we didn't pass anything but this way you can now go through and debug your code while executing which is great because you can see the values used and stuff so that's that's really useful so now you can debug and test your lambdas locally which is great there's just two more things i'd like to cover really quickly in this tutorial and one of them is the vs code extension for sam and the other one is how to use and change the docker network which the sam application uses and the reason why you want to change the docker network is because sometimes your services rely on other cloud resources maybe your lambda will push a file to s3 now you can't connect to aws or you could connect to aws in your lambda but you don't want to you want to test that locally as well and you would use local stack for that which is a great software but i won't cover local stack but the thing is if you run local stack that runs in docker as well so you need to use the same network for your lambda and sam to reach the local stack network so we will cover that very quickly let's begin with the extension first so i don't know if you have noticed but i have these super nice hints coming up like do you want to add the add a debug configuration for this lambda and this is the aws toolkit extension for [Music] this code and it looks like this and if you're going to use sam i really recommend you to download this one it helps a lot so if you download this it will ask you to connect to aws and you will get this little button here which you can use to actually see all your deployed items so i don't have anything deployed in this account but you could manage them from here which is great but the most important thing is it actually adds these it actually helps you in your template so if you take a look at my template you can see that like in the api it says do you want to add an api debug configuration for this because it knows that you will be wanting to debug this so that's amazing also it helps you with the jaml somewhat it tells you like what properties and stuff you can use which is really helpful so what we can do is we could also go into the sqs lambda and if you press add a debug configuration now and it will ask you do you want to use hello gopher direct invocation or do you want to use a logo for the api or the sqs so let's go ahead and do the sqs and you will see the configuration we created and this is the configuration that was generated now the type here is not go it's on aws sam which is a costume type which they have built and it's a direct invoke in if you use the api instead it will be on http and you can see a bunch of other this is which lambda to invoke for instance this is the name of the lambda and in here you can specify the payload to pass and what's great here is that you can either specify a raw json by using the json tag or you could specify the path which can point to a json file so you could use sam generate to generate your payload and pass that in using the fi file path or you can also put the json right here i usually like to keep the event.json file inside each lambda because that kind of serves as documentation as well you know what the inputted number should look like and so the that plugin is really great and i recommend you to use it you can also specify the environment variables to use while debugging so that's also very helpful and uh that's also great when you want to kind of break things you could just change the environment variable really easy this way so regarding the sam network if you're using the if you if you're using local stack and you want to change the network or if you want to change the network any other reasons there's actually a configuration option called sam and it accepts the darker network let's see if we can make it yeah the darker network so we can put in the name of the docker network that we want the sam application to use while it's running and let's fix the json so if i were to execute this i would run on the aws mock network instead so that's really useful and we didn't have to do anything because the plugin generated this for us also if you don't want to use this plugin or if you're using go land or any other things you can apply the sam docker network inside the invocation command so if you go ahead and do some local invoke you can do dash dash docker network and type in the name of the network to use and this doesn't only work for invoke it works for start lambda start api and the other invocation types there are so that's what i wanted to show you regarding those two things so today we have covered how to develop simple lambdas and how to debug them or at least how to attach the debugger to your lambda now it's up to you to actually debug because i can't help you with that but you should have all the tools you need to get going and i really hope you liked this video and that you enjoyed it and if you if you liked it make sure to like it and subscribe and hopefully i will keep making videos for you

Channel: ProgrammingPercy

Views: 3,379

Keywords: aws, lambda, golang, serverless, sam, go, tutorial, sam tutorial, lambda debugging, debug, learn sam, learn aws sam, guide

Id: QWfuA5SrmW8

Length: 52min 2sec (3122 seconds)

Published: Sun Apr 17 2022