How to build a low-latency serverless GraphQL API on AWS with Node.js, AppSync, Lambda, and DynamoDB

so aws appsync recently released aws lambda direct resolver support allowing us to map queries and mutations from our graphql api directly into a lambda function so in this video i'm going to walk through how this works we're also going to look at how to look at performance and improve that performance by fine-tuning the settings in our lambda function and look at how to kind of view these results in cloud watch and also x-ray so i hope you enjoy this video i'm going to hopefully walk through it fairly quickly to give you a brief overview of how all this stuff works okay so to get started i'll just go to the aws console and we'll first go ahead into lambda and create a new lambda function i'll click create function and for this function i'll call this app sync data source lambda so appsync ds lambda is good and i will then go to create function and run time is nodejs after this function is created we're going to go ahead and then make a small update to the code so in the handler instead of returning that i'm just going to return an object that's like has a data property set to hello from lambda okay so save that and i'll go to another window in the aws management console and here we'll go to appsync and from outsync i'll go ahead and click create api i'll click build from scratch and i'll call this appsync lambda example so since we're starting from scratch we need to create a schema to kind of get started so i'll go to schema and i'll have a type of message that has a data property that is a string so this kind of maps directly to the data that we're turning from lambda and i'll have a query of get message that just returns a message so i'll save the schema and i will go now to our data sources and this is where we're going to connect the lambda function as a data source so here i can click create data source i'll call this the same as this so appsync ds lambda for the source type i'll call the uh we'll choose lambda function and for the region we'll choose us east1 for the function arn we should be able to paste in the name of our function and kind of see it pop up right there and i will choose an existing i'm sorry a new role and click create okay so that should create the data source for us and we can now use it use it so i'm going to go to my schema again and for the resolvers i should now see a get message resolver and i'm going to attach a resolver by clicking attach and here we're just going to choose that lambda data source from the drop down menu and then that's all we have to do we can just click save resolver and that's kind of it we should now be able to go ahead and query from this api using that function as a data source so i'm going to go to queries and we'll have a get message query and this should return some data so go ahead and try this out and there we go our lambda function is is now being set up and being operated via the graphql api um so there's some other things that we're going to now do the first thing we might want to do is what is actually going on in this event object now so let's go ahead and log this out and i'm going to go ahead and save this and i'm going to go ahead and hit it again and then we should be able to go to monitoring and cloudwatch and see what's going on with that event so the most recent event is this one right here wait a couple seconds i think that's it there we go after after a while we start seeing some some information here what we now are logging out though is this event object so this has all the information about the event so if we pass any arguments into the query of the mutation that will show up in this arguments object if we have an id token that will show up in the id and the identity we have our request headers we also have this info object which is what we're going to be using to get the field name from the graphql operation so if it's if it's a get message query then the fill name will be a get message and so on and so forth so let's now use that information to kind of start having like an idea of which operation is is happening and then being able to kind of you know do different things based on that so so let's say we want to like do a switch statement based on the events dot info dots field name i believe so event dot info dot field name and we want to say if it's get message then we will return this data and this way we can kind of now check for other cases and do other things so the what we might want to do next is interact with a database so a dynamodb table for example so to do that let's go ahead and create a new table and then we're going to be updating both our function and our api to interact with that so i'll go ahead and open up a new aws management console window and look for dynamodb i'll create a table i'll call this a user api table and for the primary key i will be choosing id and we'll go ahead and save that table and next we're going to go ahead and populate this table with a couple of different properties or a couple of different items so i'm going to have an id of 1 and we'll have like a name in a location and then we'll just add one more item idf2 a name of jennifer and a location of netherlands okay so we have a couple of properties a couple of items in our table we now need to give our lambda function access to this table so i'm going to copy the name of our function right here and i'm going to go open once again another window from aws and go to iam and i need to basically say i want to go let this lambda function interact with this dynamodb table so i'm going to go find the role for this lambda function by clicking roles and searching for that name that we that we have here and i'm going to choose this app c ds lambda dash roll the the one with the um the same upper the same casing and here we're going to add a new policy so we're going to say attach policy and what we want to search for is dynamo db and i'm going to give a full access but you could also just choose to give it access to only this single table arn but for this example to make this fast let's go ahead and just give it full access so now our function can access this table so i'm knowing that let's now update our schema to kind of have a new property that that maps well to this id location name so in our api let's create a new user type the user is going to have an id a name and a location and then we're going to have a get user by id passing in the user id returning a user we'll go ahead and save that and with that we can now attach this new query to another resolver and it's going to be the same thing we just click attach choose our lambda function and click save resolver and we're good to go so now that we have all this set up um we're going to need to remember that we need this query name of get user by id and our function because we're going to be switching on that event info.field name so we can now say case get user by g and then we're going to return something here now what we're going to need to do is interact with dynamodb so i'm going to kind of create a new file that does all that and then just returns some data and we're going to invoke that asynchronous asynchronously so i can say awaits um and then we'll maybe call this function the same as the actual you know the name of the the case so of the field name is user by id we'll call the function getuser by id and to get that we'll go ahead and require it and of course we haven't created it yet but we'll go ahead and just save this now and then we'll go ahead and create it here so i'll say new file get user by d dot js and this is going to be where we're getting that where we're interacting with dynamodb okay so i'm going to go ahead and open this up um first off we need to require the aws sdk and then we're going to say we want a new instance of the document client from dynamodb and i believe that's the casing is all right there and then we'll have a function of git user by d and i think this needs to be an async function and then we'll just set the module that exports as that get user by id this getuser by id is going to be passed in a user id and the way we're going to get that is in index.js this is going to be taking in a user id now this is going to become off coming off of the event.argument so we could say events.arguments dot user id and um using that user id we're going to begin populating some params so let's say params and this is going to have a couple of things first of all we need to set the table name and this is going to be a string and then we're going to have a key and this is going to have the primary key so we're going to have an id of user id and then the table name is going to be this user api table and using these params we can do like maybe a try catch block and then in the try block we'll uh we'll create a d structure actually an item which is going to be kind of how the data comes back from dynamodb document client and this is going to be set to [Music] the doc client dot get params and then dot promise to make this asynchronous and then maybe we'll just return the item and this item should just have the id the name and the location of that user all right so let's go ahead and save that and make sure all this stuff looks good go into index.js and take another quick look here so get user by d is equal to get user by id the event has getuser by d event.arguments.userid yeah this all looks pretty good the next thing i guess we can do is test it out so um the two ids that we have are one and two so we'll go to queries and we'll say query get user by id and here we'll say get user by d with a user id of like one and here we'll return the id the name and the location we'll try this out there we go looks like it's working so id1 id2 everything looks pretty good um what we want to do now though is let's kind of take a closer look and see things like the latency things like how long individual requests and different parts of our api are working so to do that um under settings we can go and we can say we want to enable x-ray and this will give us kind of a really a nice in-depth look of what's going on so we'll go ahead and save that i'm also going to go to cloudwatch and i'm going to just delete whatever streams that we have coming from from lambda right now and um i also want to make this be a cold start uh because we've already invoked it a couple of times so i need to just change the function and save it to make make sure that happens i might just have like a default case here just to give me something to write okay so we'll go ahead and save this and we'll go ahead and hit this up one more time and then we're going to start taking a look at some logs so i'll go back to queries get user by id see this takes a second and we have our data come back so let's go now to cloudwatch and again you get the cloudwatch by going to lambda clicking on monitoring and then view logs and cloudwatch here we can see that we have at the bottom here a duration um we also have an init duration so this took quite a bit uh 1.2 seconds and then the cold start was 460 second seven microseconds um also you know i'm kind of i don't really want to see all this stuff here so let me actually delete that uh logging for now so to to to start reducing this um let's go ahead and maybe delete this log and um we're gonna go now to the settings for our lambda function and i'm gonna go ahead and bump this uh memory size up to maybe like one megabyte and save that now the memory is gonna increase the cost of the the execution itself in general like it's going to actually incre increase the cost of the cpu but it's going to be a lot faster and therefore it's going to reduce the cost there so at the end of the day what you're probably going to end up having is something very close to what you would be paying anyway it's just going to execute a lot faster now this could be a little more it could be a little less but in general it's going to be kind of close but i wouldn't just take this as word i would i would test out your own api when you when you mess around with things like memory but that's kind of like what i've seen so that's just kind of what i'm saying but when we now test this out it should be uh significantly faster so we'll go ahead and run this again now that we've increased the memory and let's go back to cloudwatch wait a couple of seconds for it to register and then we'll go ahead and refresh until we get it and and before we saw the duration was 1.2 seconds and you now see that it's now 95 milliseconds so it's more than 10 times faster now that we reduce that we also see that we though have this init duration which is kind of the cold start so if we run this one more time it comes through a lot faster because this second time we run it there will be no cold start so let's let's wait until this new event comes through and you'll see that the duration um for this next one was 47 milliseconds and there was no indent duration so we we we already you know decreased the latency pretty you know pretty uh big time right and just uh with one setting that isn't really going to probably cost a whole lot extra if anything and maybe even cheaper again but what if we want to get rid of that cold start a couple of different ways you could do that you could set up some type of event that hits this function every couple of minutes to keep it warm or you can use something called provision concurrency that basically tells you i'm sorry it basically keeps the function warm for you but it costs money but let's look at how much it costs and how to actually uh implement that so to do that we're going to need to have a ver a new version of this function and to and to kind of to do that we're going to be going here to to publishing a new version but before we do that let's have like a new field that um that we're going to be invoking that's going to be basically be doing the same thing as before it's going to be calling this git user by id but we'll call this git user by d provisioned this way we can have a different graphql operation coming in that does this okay so now that we have this new field that we're going to be using let's go ahead and create a new version of this function and i'll go here to actions publish new version i'll call this v1 and this is going to create this this v1 version and then i'm going to go down here to provision concurrency and i'm going to set um this to one and you're going to see now this this is going to cost 11 a month if you kept this running for the whole month so i'll go ahead and click save and what this is going to do it's going to give us like an arn for this versioned function and it's going to also keep it warm so we're not going to see a cold start time here so to use this now we need to actually use this data source in our apps absolute api it's not going to automatically do that so let's copy the arn and we're going to go back to appsync and we're going to go to data sources we'll click create a new data source we'll call this versioned lambda or maybe we'll call this provision lambda for the data source type of aws lambda region usc 1 or wherever your region is for the arn now um we're going to just click don't see your function because we need to enter the arn ourselves so i'm just going to go ahead and and do that and then we should now see that we have our new lambda data source set as provision lambda so we should be able to use that now and what we're going to do is we're going to need to remember that we need this new uh query to make this happen so i'm going to go to my schema i'm going to create a new query called get user by d provision with the user id that returns a user just like that other one i'm going to save the schema and then i'm going to attach a resolver to this new query and for the data source i'm going to use provision lambda and i'm going to go ahead and click save um so now now what we're going to be able to do is we're going to actually be able to go into both cloudwatch and x-ray and kind of see the differences that are going to be going on here so i'll delete this log stream i'm going to go to lambda and i'm going to go to monitoring i'm going to click view traces in x-ray as well for maybe the last one minute and then we're going to be testing this out and what i'd like to do is essentially um we're going to be having this query and then we're going to also be having the provision query and we'll we'll perform the exact operation and what we're hoping to see is that when we go into x-ray that there is no cold start for that provisioned okay so i'll go ahead and click uh i'll start by using the original get user by id and then i'm going to do the get user by id provisioned okay so both of those operations came through successfully if i go to x-ray go ahead and refresh that i see that i have two operations that happened and if i go to cloudwatch and i refresh i see that i now have a stream for the latest which is kind of the main and then i also have one for version one so let's go ahead and click on the trace for the provision well actually let's start with the basic one here we see that we have lambda took 792 milliseconds oh you know what um i think that we actually need to wait until the provisioning is done now that i think about it okay so the provisioning is actually now done i think that we ran this function the provisioning wasn't quite done let's let's try this one more time so what i basically need to do is uh let me go back and update this let's see here let me update this to 2 just to basically make this and then move it back to 1. i just want to make sure that the lambda function is not i just want to basically prove that the lambda function is is coming from a cold start essentially but we're not going to see it whereas if we just invoked it it would be warm so i just want to make sure that we're proving that this thing is not being a cold start so this is going to take a couple of seconds so we're going to now have to wait for this status to go by and once it's done it'll say ready so this shouldn't take too long but um but what i basically saw when i was doing those traces that they both were um you know having a high response time which means that the provision concurrency just wasn't set up yet um but it is cool though you can actually go down here oops and see you know information about the different um things that are happening in the in the operation so waiting for this to happen it shouldn't be that much longer a couple more seconds and then we'll be able to to wrap this up all right so provision concurrency is ready so let's try this out again i'm going to do the get user by do request from the old function and again to make sure that this one is indeed cold i might even go back and just make a small change and save it okay so get user by id get your user by id provisioned you can actually see a little bit of a noticeable response there actually but the interesting really interesting thing to me is that you're going to see that the um the operation log in in cloudwatch is going to include the init duration so we see here the init duration is 800 milliseconds but in the x-ray we're going to actually see that this init duration wasn't there it was actually part of the provisioning and it gets logged out on the first request so if i go to latest we're going to see that the init duration and and the duration are there but we're also going to see that in both versions you know the init duration is logged in cloudwatch when we go to x-ray we're going to see that the response time is actually only 235 seconds for the entire operation from from graphql whereas and this was the version that is using provisioned and then the regular version is 764 milliseconds from a cold start we're talking about a pretty big difference so if i want to interest if i want to look deeper into this i can go ahead and click there and kind of dig a little deeper here we see that the lambda response time is uh 193 milliseconds and we see that the absync response time the entire thing was 235 milliseconds meaning that uh like the combination of the lambda and the graphql operation took that long and the lambda function was 193 milliseconds so the rest of that was was appsync so approximately um 20 or 30 milliseconds and then if we look at the x-ray log for the the one that did come from a cold start we see that just the lambda function took 700 milliseconds so 193 versus 700. it's a pretty big difference um on the same operation so yeah i think that's about it that's that's mainly what i wanted to cover um you know getting up and running with appsync and dynamodb using a lambda resolver is actually a really really nice user experience and it doesn't take very long to get it get up and running so i hope this helped out a lot and hope you learned something if you're not following me already follow me on twitter at dabit3 and subscribe to my youtube channel please if you're not already thank you

Channel: Nader Dabit

Views: 16,532

Keywords: graphql, serverless, amazon dynamodb, dynamodb, aws lambda, node.js, javascript

Id: _9DFFg-pNss

Length: 28min 16sec (1696 seconds)

Published: Mon Aug 31 2020