How to Build a Serverless REST API with Node.js, TypeScript, and AWS DynamoDB

hello everyone and welcome to my web development cloud and iot education channel my name is thomas i'm full stack developer and today in this video i'm going to show you how to build fully functioning rest api with input data validation included and dynamodb saving as a data storage everything deployed to aws as a set of lambda functions thanks to the serverless framework however before we can get started there are two requirements to meet that i'm just going to quickly talk about the first one is aws cli installed and configured on your machine and second one is serverless npm package installed globally if you haven't done this yet check my other video where i explain how to do this step by step you can find the link in the description of this video now let's get straight to building the rest api okay so let's start from opening a command line where first of all i'm going to change directory to projects this is where i keep all of my projects basically and from here i'm going to run serverless create command where i specified template url to my template repository with a serverless typescript project and a path that's going to be a directory this command is going to create and where it's going to pull all of these files from the template i called it products api because product is gonna be our resource in our rest api that's gonna be something we're gonna be operating on i picked product because it's it's very common right like in internet stores e-commerce systems okay i'm gonna press enter now and that should download all the files and move them to products api now i can change directory to products api and i can run code and dot that should open visual studio code and there we go visual studio code opened so what we can see here are the files downloaded from my template repository serverless typescript template repository um what we've got here is a few files the most important files are package.jsonright this is where we have all the dependencies ts config with typescript configuration serverless.yaml with all the infrastructure configuration and src handlers.ts this is where the code is kept right and as you can see right now we've got an error here because this module cannot be fine found right the problem is with the dependencies they're not installed yet and that's going to be the first step so i'm going to go to terminal and in terminal i'm going to run yarn right can be yarn or npm install depending on what package manager you use for node.js i use yarn so just done that let's give it a few seconds okay done and now before i move to the infrastructure configuration i'm gonna do one extra step with my visual studio code i'm gonna go to code preferences and settings because what i would like to do is to change the default formatter right so i just typed format and then default formatter let me just close the terminal i'm going to change it to prettier and what is important is to have these options here this select boxes ticked right so i have format and paste format on save and format on type okay i'm gonna close it you can do it at the workspace level if you want what this is gonna do is basically work with this prettier rc configuration and out of format my code whenever i change something in here and i save that's gonna just automatically format it for me nice okay so i've got my workspace configured now let's move to the infrastructure so i'm going to close handlers for now and package.json because what we're going to focus on is the data storage first right because we need to have a place where we keep all of the data all of our products and that place is going to be dynamodb table for that we can use a resources section in serverless.aml file this is where we can add cloud formation aws cloud formation resource templates i've prepared this beforehand so let me just copy paste it okay there we go so as you can see this follows the this pattern commented out pattern here we've got resources many resources again then the name of the resource this is up to us what the name is going to be then we have a type of the resource in our case this is dynamodb table and properties of that resource and these properties are table name which is a product table again right provisioned throughput this is the number of reads and writes our table will be able to do in a second right so higher this value is more operations you can do simultaneously in one second on that table right i i picked one because you know this is an example project and also i don't want to generate a high cost right because you know basically more units you use you utilize more cost you may generate and then you have to pay to aws um then two extra additional properties we've got attribute definitions and key schema and this is quite important because dynamodb is a document database so the documents you store in a table can have any sort of properties right it's not restricted it's not like with relational database where you need to define all of the column names here you can just you know create anything you want however you need to specify a partition key and that partition key is something like a primary key in database right so that there's a unique value key that identifies a document in a a table in my case i it is a product id so i picked it as a product id in the attribute definitions you need to specify the type of this key and the type i specified is string you have a choice between string number and binary but i'm just going to use a uuid here so string is the best type then in the key schema this is where you actually tell that you use this particular attribute as a partition key which is a type hash this other optional key you can specify in dynamodb table it's called sortkey it allows you to make a specific queries to retrieve the data from the table but i'm not really gonna go into details at least not today i'm probably gonna cover this in the future however if you are interested in that have a look at the description of the video i left a link to aws dynamodb documentation cool okay so that's the resource and now a second step is going to be to define i am role statements and again i prepared something beforehand so let me just again copy paste okay so here are the row statements why this is important is because creating a resource on its own is not really allowing for our lambda functions to access these resources we need to specify the permissions explicitly with aws right and in serverless framework you you do it through i am role statements so basically by adding this section in here i'm telling aws that this particular lambda functions i have specified in here are going to have access to this to perform this operations right which is put item get item delete item and scan on these resource and that resource is our products table so what's happening actually here that uses cloud formation function get attribute to get product stable unique aws identifier right arn that's the name of the aws unique identifier so essentially there's going to written a string this identifier and that's gonna set permissions right to perform these operations nice okay so now the last step is to actually define the functions and here i'm actually gonna do it along with the implementation so let's open handlers.ts and let me show you how this actually points to the code right so here in the functions we can specify the function name this is internal name because what's important is where this function points to and it points to src slash handlers and this is src handlers.ts file and dot hello is basically pointing to the exported function of the same name right so we've got hello here and hello there and basically when this lambda function is executed it's going to execute the code from the hello function however we've got something missing in here because at this moment we're not really able to access this function using http client right http communication because there is no event specified so the only thing you could do is to use aws cli to run this lambda function when i add this section over here with events and http api i'm gonna turn this into an endpoint an actual endpoint behind api gateway um the example one is a get endpoint right with the path slash users create and when you call this endpoint using http client or even browser right because the method is get this code is going to be executed and we're going to get this body as a result right however i'm not really going to be deploying this because that's not that's not the goal of this video we're building rest api on products so i think the first step is going to be to rename this function because we don't want to do hello world but create product get product update product delete product and maybe one extra to list all of the created products right so yeah for the handler i'm going to rename it to create product so let's just quickly do it because this is pointing to create product now i'm gonna rename this to create product as well and uh in here i'm just going to use rest api restful specification right so what you do for creating a resource is you use the path to the resource and use post as a method right so that's the create product and now let's do let's implement this endpoint right so create product what it should do it should create a new document in our dynamodb table right like inset in order to do that first of all i need to import aws sdk it is already installed here that's gonna be aws sdk and i need to create the actual document document client that's the name of the of that dynamodb client let me just quickly show you so that's document client instance right that's the only thing you need to do you don't need to specify any connection string everything is going to happen automatically so doc client is basically the our dynamodb client that's going to be the instance i'm going to use and now let's create something with it let me show you how to create something with it so i'm going to use async await syntax so let's just doc client dot and put put where you need to specify the following properties table name that's gonna be products table right it's exactly the same name as i specified in the resources over here right products table and then the next one we need is an actual item right and this is this is where we have to specify product id right we need to specify that that's our partition key and as i said i want to use uuid for product ids so what i'm going to do is to install an external dependency uuid i need to add the types as well because this is typescript right so we've got uid and now when i just call v4 that's gonna generate a random hash v4 uuid but that's not everything right we just we don't just want to create an item with a product id property we want to have all of the properties that has been sent in the body so let's just define a new constant and use a json parse to parse our event body i need to cast as a string though because it might be null for now i'm just gonna go with the positive path so i'm not really gonna be handling errors yet but i'm gonna move to it later no worries right um now this awaits actually we've got an issue here it has it says has no effect of the title of this expression this is because if you want to use async await syntax you need to use a promise method on the doc client right okay so now this is actually going to create it but except for the product id i'm gonna use the spray operator on the request body so basically our item now is going to be whatever we send in the event body hopefully is a json and then product id with auto generated id right so that's the create product however one more thing and that would be actually nice is to instead of sending this default success message to send an actual product item right the actual product so i'm gonna move this let's extract it to a variable that i'm gonna call product right and then i'm gonna pass it in here and we also going to respond with that so we will see newly created a product nice so yeah create product is done let's move to the next one let's do get product now and to add get product i'm gonna go back to functions and we're going to define a second function let's call it get product get product in here and the method now is going to be get but this is getting a single product right so we need to have a way to specify the product id that we want to retrieve right so i'm going to do it in the path that's going to make basically an end point with the path parameters right i'm going to show you how to read it so having get product let's go to handlers and again let's create a new function get product this time and because i specified this path parameter id here in the path i can now access it like that so i can do const id equals event dot path parameters question mark an id a question i mark because id might be undefined which is not really going to be the case with serverless framework and this configuration but this is how the types are defined so yeah we can we actually gonna get something for the id right so having id now let's use doc client to retrieve a document from dynamodb and that document is going to be the product basically right so we're going to use the get method and forget method we need to specify a table name as well so because i've already used products table in here let me just maybe create a constant in here const variable right so let's call it table name products table and now i can actually table name sorry so that'd be table name and what we're looking for is a document with a specific key and that key is product id and that's our id that we retrieved from the path parameters right also obviously i need to use a promise and i need to assign the result of this promise to a variable let's call it output because that's not gonna give us a product straight away there is some additional data on the output so basically output is an object and we have a consumed capacity and item an item is going to be a product if the product exists right so if that document exists we're gonna get it if not i'm actually going to return 404 status code and body is going to be something like jsonstringify and let's do something like error not found so if there is no item on the output right returning 404 that means it doesn't exist otherwise i'm just gonna return it so that'll be status code 200 and body json stringify and that's going to be our output item okay so yeah so that's get product so we've got create product and get product i think at this point i could give it a go let me just confirm if everything is fine so get product create product maybe i change the status code to 201 that's that's a bit better because that means created instead of okay and serverless configuration yeah that seems to be good right okay yeah let's let's let's give it a quick test so i'm going to open a new terminal right i actually have lots of terminals in here i might actually close them okay and to deploy our lambda functions we just need to call serverless deploy and that's gonna take a few seconds or minutes okay done so as we can see in the output we have two end points now post and get let's start from post one let's create a new product so i'm gonna copy this and paste it into my postman right my postman http client let me just create a new request i'm going to chat a method to post and let's specify some json request body right so let's create a product maybe we have a name iphone x description and the price maybe that's enough okay and let's send this now nice okay so what we get back is almost the same json object except for the product id right so that's been generated with uuid and now we can use this id to let me just show you to retrieve a product using get right so we need to specify the id as a par as a part of the path so i'm gonna go back to postman and gonna paste it here so as you can see we've got now just slash product slash our uuid and when i send it i should get back there we go that's what we got right there the product that i've created so our end points are working well however there is no validation at this point if i break this somehow so let's say i change the this json to text and send something different i'm probably going to get something like 500 let me just see yeah okay so i get internal error this because we don't have a body validation but no worries again i'm going to do it right okay let's go back to the code and implement the rest of the end points so what we've got missing is delete there is update and finally a list so let's just quickly add them for the update product i changed the method to put right the the path stays the same right because we want to specify we want to give the id of a product we want to update but for rest apis for the update operation we use a put method then it's going to be very similar situation with delete product so we also need to specify id as a path parameter but the method this time is going to be delete and finally the least endpoint is going to be more like a create one that's why i copied it from there let's call it the least product let's give it a get method and maybe let's change the path to products so it's going to be a bit better right using the plural here that means we're getting a list of products okay right let's implement these functions now in the handler so we've got get product let's do update product now so maybe i just again copy paste from here and just rename it to update product update product and for updating the product this is going to be again important because with dynamodb document client and there are actually two ways of doing that but i think the simplest way to update something is to use a put a method on the doc client as well because put works as a like an upset basically it creates an item creates in our case a product in the products table if it doesn't exist but if does if it does exist then it's just gonna replace it right so if the product id of this product that you specify here does exist in a table then it's just gonna update it right it's gonna update it with whatever else you you've got on your product and i'm just gonna use the put upset operation what i'm gonna need to do in update product is to get it first to make sure it does exist because if it doesn't exist update product would create a new one and that's not the behavior we would like to have right we want to return 40404 right not found if the product we're trying to update doesn't exist so that's gonna be the first step so let's just copy this and paste over here and then let's copy this and paste here right the only difference in the product itself to what has been in create is that we want to specify id so the one that we retrieved from the path parameters and finally what i want to do is again written a status code of 200 and a body with json stringify product okay nice so that's our update product however you've probably noticed we have some duplication right we have a duplication with get product and we have a duplication with create product we can get rid of this duplication easily by introducing a simple error handling and that's actually going to be also useful for the body for the request body validation that i'm gonna introduce later so maybe let's just start doing it right now so that's gonna be the first that's gonna be the first logic that i can extract to a separate function and then reuse it in update product let me just show you how can we do that efficiently so let's create a new function let's call it fetch product by id maybe and let's specify the actual id as an argument let's make it asynchronous because we're gonna use async await syntax in there and then simply let's just copy this and paste in here right so that's what we're gonna do but instead of returning status code with the body i'm gonna use error handling so i'm gonna throw an error instead let me specify a new error i'm gonna call it http error because it's gonna end up with the lambda responding with an actual error status code and message that's gonna extend from the base error and then we will have a constructor where we can first of all let's specify let's allow specifying something like a body that's gonna be a record string to unknown so basically any sort of javascript object as long as the keys are strings and then let's do something like a status code as well but you know what maybe what i'm gonna do is to specify it first so we will have a status code as number and body as a record and let's make this actually a public property okay i need to call super with let's do it with json stringify of the body right so that's how the error is going to be created one more thing um let's make this body like that right okay so we will have this http error and when this happens we going to actually throw this error instead so we'll do http error 404 and then for the body i'm just gonna use the error not found okay and as a result i'm just gonna return output item so we have our our fetch product by id and now in get product i can use this instead so what i'm gonna do is to do maybe something like const product equals await fetch product by id and i can actually specify this i can pass it right to that function straight away but i need to cast it as a string right and then we can a written stringified product right that's not everything though because now we have to put it in a try catch block and we need to handle this error right so in here if we have error there is an instance of http error so if we have something like that http error then we want to return something like that e status code and then body is gonna be just e message nice okay and if this is not uh and http error i'm just going to return something like this so it's still going to throw an error that's going to result with 500 internal server error because that's going to be an error we actually don't expect right okay so with that logic now i can use that in an update right so in an update i can do something very similar but actually i don't need a product here i just need to run fetch product by id to make sure this error the http error is not thrown but that's still um one more thing okay so this actually needs to be an id so let's do it like that so id equals right so now this is better and try catch try catch and hit again to not repeat myself what i can do is to introduce another function let's call it handle error what it's going to do is simply what happens in this catch right so we're just going to do something like that and then i can just do return handle error and pass e same thing can happen in here right so we removed all of the duplication in the code right there is still some duplication on creating the the product like on parsing right but i'm gonna handle it later when i validate the request body okay the next step delete product so for delete product it's gonna be a bit simpler because there is no more refactoring needed i just need to create delete product function and then i still want to verify if that product exists so we will have something like that and we also have a try catch where we can handle error but then right after this what i'm going to do is to run await doc client dot delete where again i need to specify table name and then the key which is the product id and that product id is is this so i think actually this might be a good idea to create this const variable and use it instead so we'll do something like that right also with the promise right okay last step is to return proper status code let's go for 204 for delete so i'm not really going to return anybody just 200 and for that this uh to indicate that this operation has been successful right so we've got delete we've got update we've got get and create the last one left is the least product and this product is gonna be i think the simplest of all of those because for the least product i just need to run dog client scan methods scan basically scans all of the records in our dynamodb table we can specify some conditions in here but i'm not really gonna do it because i just want to retrieve all of the products from dynamodb table so we're gonna get output like that and then i'm just going to return status code 200 and then for the body we will have json stringify and output items right so we should get a list of items right so yeah this is pretty much it in terms of all of these end points that we have specified and now we can give it a go so i can deploy all of these changes and a serverless framework along with aws is going to detect all of the changes i've done all of the additional new endpoints we've created and deployed only those so let's open a terminal and then in the terminal i'm going to execute a serverless deploy okay done we've got list of all of our endpoints so let's test it let's test it out if i open postman now and i try to go to the get the get call right the get request i've done if i send it i should get back the product i've created beforehand right so that means it doesn't create a new table right if table already exists it doesn't wipe anything out it's all there our data is safe nice okay so i tested post and get and haven't really changed much in it except for the get that we've actually tested and it's working so let's try to do update now so for update it's going to be exactly the same end point as as this one like the the url actually is going to be the same but the method is going to be different so i'm going to create a new request just copy paste from get and change the method to put and then for the body i'm going to specify a different json maybe let's go to post and let's go back to our previous request body and then let's copy paste it and slightly modify it so maybe let's upgrade our iphone and let's change the price okay so i'm gonna send it and there we go so we got back exactly same uuid but we've updated data if i go back to get request and call it again i should get in a response updated data and that is correct okay let's see how the delete works so for the delete again i can copy this url create a new request paste it in and just change the delete and then let's send it and let's see what's going to happen that's it we didn't get any content but we got 204 no content basically but that means that the record has been deleted if i go back to get and i try to send it i'm getting 404 not found which is what we wanted to get right because i deleted that record so yeah let's go back to post and create a new one again and maybe let's create another one so we'll have two products and having two products let's test the last endpoint which is the least one right so that's just get products and when i do send i should get back a list of the products and this is what i've got right so we've got both of them as a collection however i've just realized there is something wrong in here because the way we've got this response body formatted is not really correct let me just confirm if the header content type header is application json okay this is not we've got text plain right i need to fix that because the content type of our rest api is json not text plain right so it should be application json here let me show you how to fix it it is quite a simple so i'm going to go back to the code and then in the code wherever i return right the api gateway proxy result this is where i am supposed to specify response headers so what i'm gonna do is actually create a constant at the top of this handlers.ts file let's call it headers actually and let's add a content type header application slash json like that and then wherever there we have a return in these handler functions i'm just going to add headers like that right so let's just quickly do it so we should have headers are not really here we should have headers in here as well and then in here but this is actually not needed in here because that's empty but it is important to add it here right so if i deploy the code now and then we go to postpone and send the same request again to get the product list i should get them nicely formatted right so now the now the header is right we get application json and that's the json format that we get here if i go to get end point and but let me just check let me just change the the the id to this one to the existing one right actually doesn't really matter because if i do send that should also be formatted but let's get entire product here and there we go right this is an actual json response right okay so now the last the last bit that's left uh on our rest api is the request body validation right because if i go back to post i i showed you this already and if i mess something up in here right then basically we're going to get the internal server error but i would like to get something like incorrect request body right and also this api allows for any sort of field right so you can specify any field in here and that field is going to be accepted which might not be the case for certain i think not certain it's actually for most of rest apis you want to have a fixed property names that you accept or you know you may have some option or some required right but essentially what i'm talking about is a request body validation at the object at the json object level so not just the format that should be valid json but also the properties of the json right the names of the properties and the types of the values um so yeah let me just quickly show you how can we do that in the code so we have our request body is validated so in order to do that maybe just let me just close this terminals and we're gonna go to we're gonna go to create product first so if this request body fails right if this fails it's going to throw a syntax arrow and this is how we know that json was incorrect what i can do is to put this again in a try catch block like like i've done with the update and we've get as far as i remember and i can use the handle error right i can use return handle error and then basically handle it there so i can add another if statement that checks if the instance of error is a syntax editor and if it's syntax error then we're going to actually not return yeah we're going to return something like status code maybe let me copy paste this so we will have a status code that is 400 right same head as but the message maybe the message actually can be like that but we actually put the original message but we will also add something like invalid a request body format and then the original message of syntax error let's do something like that right so our handle error is going to handle a syntax error and now let's do the actual validation of the properties for that let me use an external library because i think that's going to be the simplest and fastest way so i actually need a terminal now let me create a new one and let's add yup that's the name of the package right okay that's been added i don't need to install types for it because it's already included in the library so now let's just import it here come on okay so we've got our yap and now the way you use it is you define a schema so that's what i'm gonna do i'm gonna define a schema for an object of shape where i specify all of the properties i require so let's say we require a name which is supposed to be a string and it's required let's say we have a description right which is again string and required and then a price which is a number this this time right so we'll have a number and is required and finally maybe let's add a new one available right and this time this is gonna be a boolean and also required right so so that's how this is done we've got a schema and now with the schema what i'm just gonna do is to validate it use the validate method on the schema itself but that's asynchronous operation so i need to do a wait so we're going to do here we're going to do schema dot validate and then i'm going to pass a request body right so that's that's how this is validated and that's going to happen for create product but also for update product which is over here right so once we have verified that the product we're trying to update exists right in here let's make sure it's in a valid format okay right so that's the validation part and now what happens if the validate fails it throws an error a validation error and because we have one place where we handle the errors i can go to handle error and again add another if statement and if that error is validation error yup validation error i can return again status code 400 headers and body which is going to be maybe you know what actually i have realized it is a bit incorrect in the syntax error i'm gonna i'm gonna fix it in a second so yeah that should be json stringify so we have json stringify and here we will have something like errors because there is a errors property with all of the errors on yup and i need to fix that one as well because that should be json stringify and let's return it like that that's gonna be much better right okay and one last thing because this is actually related to this errors way of validating so basically i don't want yup to fail fast i think about early as is the name of the option that you need to specify and change it to false because otherwise it's true so that's going to give us all of the errors not just the failed one right so if we don't have a name and description we will get the information that these two fields are missing right otherwise if this is true we would just get a name is missing right and only when you fix the name then you get the description error that's not something i would like so yeah let's just fix that also in the update and there we go right so this is the validation that's handled in our handle error and i think i can now create a new terminal and run serverless deploy to deploy all of the changes and then we should see our rest api fully working okay deployed so let's go to postman and let's try maybe let's try from the from post so i'm gonna break it intentionally and let's see what's gonna be the error message okay so this is what we're getting we have invalid request body format unexpected token json at position 11 right so that's what we received and we have a for 400 baht request um let's try to create a new product but with missing field right now we require available fields as well let's see what we get okay we have every field is required field right if i remove another field i should get more errors and this is what i got right description required field available as well right let's fix it so i'm going to add available let's set it to true and this is working right if i change that to maybe to number then what we're getting is available must be boolean type but the final value was two three two three right so it also checks for the types nice okay so um that's the post so create works perfectly as we can see let's see how get works again so if i do that yeah i'm getting the the result correctly right let's move to put i'm just gonna copy this uid because that's the one i'm going to that's the one i'm going to update right and let's change it to maybe old iphone right and there's going to be a missing field so let's see what's going to happen right we've got errors available is required field now so that also working i mean the validation right is also working for the put method for update so let's say it's not right i'm gonna send it and it is updated now with the new values right okay let's do the same thing with delete so i'm gonna go to delete and test it again so let's remove this one right we go 204 and now i should get 404 because it's not there anymore right it's been removed from database and finally let's have a look at our products right so we can see there is a new field available and we have only two phones in here because i removed the third one and the one i have created with the post nice okay so yeah everything is working our rest api is done we could actually add some extra features but i encourage you to experiment with that you can find the link to my github with all of the code i've written today in the description of the video my suggestion would be to improve a list endpoint so we can add some sorting there maybe a search right you can experiment with a different method on the document client which is called query and you can look into secondary indexes of dynamodb it's very interesting subject i'm going to cover it in the future but if you are interested in dynamodb in general i really recommend looking into it and this is it for today thanks for watching if you find my content useful don't forget to subscribe to my channel if there is a subject you would like me to make a video about let me know in the comments cheers bye

Channel: Tomasz Tarnowski

Views: 249

Keywords: rest api, api, rest api tutorial, node rest api, nodejs json api, node js api, node js rest api, serverless, serverless framework, node.js, serverless tutorial, build a serverless app, serverless api, aws serverless, build nodejs apis using serverless, lambda serverless, create a serverless app, create a serverless app aws, serverless computing, nodejs with typescript, serverless dynamodb, dynamodb serverless, typescript rest api serverless, dynamodb api, aws dynamodb, aws

Id: yEJW4V7ddEQ

Length: 57min 29sec (3449 seconds)

Published: Thu Nov 25 2021