BUILD SERVERLESS APPLICATIONS USING AURORA SERVERLESS, THE DATA API AND CDK

hello world and welcome to another episode of fubar today we are going to learn about aurora serverless and lambda if you're watching my youtube channel i think this is the first time i will be talking about aurora so sorry for that aurora has been out since 2014 and i never mentioned it what is aurora aurora is a database that amazon created um some years ago as i said it was born in 2014 and the main difference with other databases out there is that it's built for the cloud i will not get into particular details on how it's built but if you are interested in knowing more let me know in the comments and i might do a video or dig more into this service but up to now is one of the fastest adopted services in aws because it's quite powerful it's a relational database built for the cloud that is compatible with existing relational database engines from the open source community like mysql and postgres so that means that you can migrate your applications there if they're using these engines and it should work pretty straightforward or build new applications using these uh engines and it should be okay to use aurora because of the way aurora is built it has amazing features like backup and restoring and replication and all kinds of things with the storage like the you can increase the storage as the database is used in other database services it's quite hard to increase the storage without stopping the database so it has a lot of lots of benefits aurera comes in two flavors the aurora basic like that is an rds service relational database service that it's built um with a cluster and you provision the amount of services you want and well like other rds services and the aurora serverless that scales as much as you need and it can shut down if there is no traffic already serverless had many difference from the uh normal aurora in many of the features were not out there the scalability was not really great so i was not that into it about talking about it until rinvent 2020. in revenge 2020 they announced aurora serverless version 2. and in this next iteration of aurora that is now in preview you can access it in the link i leave you in the description box in this version of aurora all the issues that aurora serverless had are fixed now the features are compatible the scalability is in milliseconds so now aurora became very interesting for me but i have to say i have been not using relational database in forever i'm a fan of dynamo and that's what i use for my applications but i think relational databases are fine and there's nothing that uh there is no problem of using them if you are familiar with them and with options like aurora serverless 2 this is something that now it becomes a real game changer for your applications so that's all i will be talking about aurora in this video i don't want to make a long video about aurora there is quite a lot of things that i can say but i don't want to get into the details if you want a deep dive in aurora let me know i will do it or i will find someone to do it for me depending on on the on the interest so i want this video to be a very practical video because i like to do tutorials i like to show you how to do things and i want to show you how simple is to set up and aurora serverless in this case will be version one but i imagine version two should work fine as soon as you get access to the preview uh using cdk and using a lambda function so we will have a simple lambda function that is triggered by an api gateway and that lambda function will be writing and reading from the database you can re-architect these in many ways you can have multiple of the functions one for each endpoint blah blah blah blah blah i will do it very simple one endpoint one lambda function one database but this is a demo there is many ways for lambda to connect to the relational database and in this case i will be using the data api what is the data api the data api is a way to access the database the database as a service so if you're familiar with dynamo this sounds obvious that you just use the api of the database to get retrieve update delete do operations in the database that's obvious but when you're coming from the relational world you are expecting to connect to the database uh then to do your operations then close the connection and so on so forth in the word of serverless that is um kind of problematic because when you have multiple lambda functions and a lot of them they will be all handling the connections independently and that can lead to a lot of problems in your database like you run out of open connections and you're killing the memory and you are making it perform really bad for that aws have announced the earliest proxy that is a proxy that will sit between your database and your lambda functions for example or fargate containers and will handle the connections but this if if it's you're using an old model that you have these uh open and closed connections and you cannot change it for example for legacy applications or things like that but if you're building an application from start and you can use the data api i think that's a better option is cleaner and it will take care of handling everything in the back for you so you just call the api for performing operations and that's it if you're liking this video up to now let me know by a like a likes really helped me to reach more people or if you like it more just leave me a comment i will really appreciate that so now let's go to the demo and see it um in action for this demo you will need an aws account you will need a configure it in your computer and you need to have cdk installed and configured in your computer i leave you the links on how to do everything in the description box so you can start from the same place that i'm going to start so if you have cdk installed and you didn't install it like today like right now i would recommend you to install it again so it updates it keeps on updating all the time so you have the latest version of cdk that's always handy and then from there we will start working on our project so now that we are done the next thing we want to do is to install all the dependencies that cdk needs in order to work so whenever you use cdk you are using different libraries and you need to install them in your application i will be using the ec2 the lambda the rds the api gateway version 2 and then the api gateway integrations and with all that i do npm install and the name of all these libraries and they will be available for me to use when i need to after this is installed we open visual studio code and we can start working on our application so if we go into the bin folder we can see the cdk auda serverless.ts and that's kind of where our application starts if we open there we will see that there is one stack that is the cdk or a serverless stack we could have multiple stacks in here and this of each of them will create an independent uh club formation stack so that's pretty neat in this case we only have one stack so the code for our particular stack will be inside the live folder and in the directory in the file called like a cdk or a serverless stack and this is empty because we need to create our infrastructure i really like cdk because it's so simple to create applications and you will see if you never saw cdk in action you will see how powerful this is so now i import all the libraries that we just uh added to our packages uh in the mpm install and then i will start writing my code the first thing i need is to have a vpc ppcs uh are needed because aurora will run inside a bpc so we need to have a bpc before defining aurora if you're using cdk that takes one line of code so basically you define the vpc and you put a name please put a name so then you can find it uh and that's it the next thing we're going to do is to create the aurora cluster and this is a little bit longer but still very short and very clear and the cluster will be a serverless aurora cluster and we need to put a name and then we will say what kind of engine we are using and we are going to say that is postgres then we will say which version of the postgres is in this case the 10 and then we need to put a name to our database i will call it testdb very original as always we associate it with the vpc that is going to run into and then we decide if the it will pause every x seconds on all the scalability and that's kind of it then we will create our lambda function the one that will be triggered by api gateway and invoking the and accessing this database and this lambda function is called my function it will be node and the code for this lambda function will be in the lambda function folder that we are going to create in a second the handler is the index file and the method is the handler and then we will give 1024 megabytes of memory to this function and we will pass some environmental variables function and these environmental variables are very important to be able for the function to use the data api and access the database we need to pass the cluster arn that we are getting from the cluster we need to pass the secret arn that is um the cluster secret and this is super interesting because when we create an aurora serverless this way it's uh using the secret manager uh to manage the credentials for the aurora serverless so if you see here there is no visibility on the credentials so basically the cluster that we create will have a secret manager element associated with it with the credentials for the database so then we can link that secret manager arn from the database to our data api and in that way the data api can access the aurora database without really having to know the credentials and i think that's super important because we don't want to put credentials anywhere in our code we want to be credential free as much as possible so in this way it's super super easy that means you have the name from the database we will need it as an environmental variable and that's kind of basically it and now now i will create the folder the lambda functions but we'll come a little bit later to this folder to build the lambda functions itself but there is the folder and there will be all the code for our application let's continue with our infrastructure and there we need to grant access to the lambda function to access the cluster so that's important all the lambda functions are born without privileges and that's one line of code that grants data api access so this is very fine grain access to the database so this is super cool we don't need to define any i am policies or roles or anything by using cdk with one line of code we just defined the exact fine grain access that we need from this lambda to this cluster the next thing we are going to create is an api gateway version 2 endpoint in this case we are having one endpoint with uh the method basically any so all the methods will be accepted and that will be integrated with a lambda proxy integration to our function so that's very straightforward i like cdk so much because everything makes so much sense at least maybe because i have the mind of a developer the last thing we are going to do is to write the output i want to have in our cloudformation output the api url so then we can use it in postman so i love cdk this is so simple there are 50 lines of code here that define a aurora serverless cluster well it defines a bbc a serverless agura cluster with all these chavans it defines a lambda function and it defines an api gateway and it connects all through with the right permissions this is so powerful with only 50 lines of code and it's very readable it's i love cdk so now i will go into my lambda functions and i will write npm init and i will basically initialize this package with a packet json because we will need to install some dependencies and then i will create the file index dot ts where our handler function will be i will do some imports that we are going to create later and then we will write some method there the handler method and basically this under method is a big switch that will tell which code to execute in each time so very simple and then after it's doing whatever it needs to do it sends a response and i have a method called send rest that formats the response in the right way and return it there is no magic here then i will create the file get to do they get to do again it's just getting things from the database uh i will get here a particular one based on an id so if you see there i'm using the data api by building a query and then i use db.query and i pass that query with the parameter and you can see there that i have an import that is the db so we are going to look at that in a second and then that call to the db database query will return something and i can just show it so basic sql we pass it to database query and it works so here is the database query and this is basically using the data api client so i need to pass some parameters to the database client and that's where i needed the environmental variables i need the secret arn i need the resource arm for the cluster and i need the name of the database so that's why i'm passing all these things and then i just export it and we can use it anywhere in our code so this is the definition of that and we will need to add to our project the data api client then i will create the create to do again it is using the data api to create something to insert um yeah and we here require the uuid so we are going to do npm install the data api client and also the ui uuid in our alarm the function so we can use them when we use the function when we deploy the function the next thing we want to add is the list to do's again uh just select all from this data from this table and then we will add the type because we're using typescript i like to have this studio type and this is basically the same type as the database when we create as a table so when we create the table we will have the same structure and the same everything so that's it if you're using cdk for the first time or you're using it for the first time in a region you will need to do cdk bootstrap and in this case i will do it in my uh virginia region in the profile virginia that is called because it's pointing to my region virginia and that will basically set up cdk to work that you need to the first time if you have any errors when deploying this might be missing so don't forget to do that so that takes a moment and then we can run the deployment so when this finished now we have the environment bootstrapped and we can deploy we will do npm run build to build the project and to build all the typescript and have everything ready and then cdk deploy and if you have a profile like i do just type the profile name then it will show you what it will create uh and that's pretty neat before deploying it and you say yes and this will take for a while so i will basically speed up until this is completed so when this is completed uh we will get in our output the http api from the api gateway that we will be using in a second but first we need to create our table we haven't created a table yet so our database is kind of tableless so if we try to execute anything in our application we will get an error so we will need to go to our aws account into uh we can type aurora or rds and get inside rds and there we will go into the um instances and you will see that you have an an instance so you will see that you don't have any instance but you have one cluster i just click the instances but feel free to to click whatever there is our serverless aurora and then we want to connect to it so for that you can open in a different tab the secret manager so we will do that and there you will see that you have one secret and that's the one that we will need to connect to the database so we get the secret arn and then we will paste it to connect to aurora so we do query actions query and there we will be prompt with some form that we need to complete the first thing is the username and we will use the connect with a secret manager arn so there we need to paste the aren and the next thing we need to do is to put the name of the database so you can take it from the code if you don't remember but it's this test database and then when is that we will connect to the database and we will be accessing it so now you can type uh the basic create statement from sql create table and put an id and a task and those are the same exact names that are in our to-do typescript file in our type so id and tasks so do that and then basically you can save that query if you want or not and basically that will create a table so that's pretty nice if you want you can describe and see if you have the tables and what is inside each of the tables let's go to postman now and basically in there what i did is i created a new request that has the type post and it has the url that we got from the output output from the deployment and then i put dashpro hello that that's the full endpoint that you can get from the api gateway and this is a post and we need to pass a body and in this case we need to pass the task so i put task avc so this is our to-do and basically when i send this request i will get back the new item the new to do and that's good that's writing into the database and then i can write a couple more to see how that works and all that is going through the lambda function through the data api into database and then i can do get and i can remove the body in this case i didn't need to remove it because nobody is reading it and i will get all the items that are in the database that's pretty neat that's selecting all from the database and that's cool and then i can grab one id for example and do a get with an id and that will just bring me the uh right element so that's pretty neat and it's so simple so that's it all i wanted to show you in this video the code is available in github i hope you enjoy it i think this is something i've been wanting to do for a while but now with the new release of aurora serverless 2 i think this is a great moment to get started with aurora and start playing with it and if you want to use cdk i think this is a great way to create infrastructures code applications and build everything very replicatable and like you can do in any environment with all the benefits of infrastructure as code so yeah this is something that makes me very happy to talk finally after like five years of running the youtube channel and never talking about aurora if you like this video please give it a thumbs up and i see you in the next episode of [Music] you

Channel: FooBar Serverless

Views: 3,011

Keywords: foobar, infrastructure as code, amazon web services, aws cdk, aws cloud development kit, aws cloudformation, aws cdk tutorial, aws cdk demo, aws cdk typescript, aws cdk typescript tutorial, aws tutorial for beginners, aws cdk tutorial typescript, aws cdk serverless, aws cdk lambda, aws cdk api gateway, aws cdk typescript example, amazon aurora, amazon aurora tutorial, amazon aurora architecture, amazon aurora data, amazon aurora database tutorial, rds data api

Id: kU8nwAbA8No

Length: 22min 46sec (1366 seconds)

Published: Thu Feb 11 2021