Getting Started with AWS CDK and Python | Step by Step Tutorial

how's it going everyone this video is going to be a getting started guide for using aws cdk with python so i'm going to show you how to get started with it and create some infrastructure however in order to do so there are a bunch of prerequisites that you need to have sorted out the first one is npm this is by far the easiest way to install cdk second one we need is pip because we're using python we need to install some python dependencies and it is done using pip and finally we need the aws cli installed because the cdk library relies on the aws cli in order to deploy its resources so i have some videos and resources on how to do all this i'll leave some links to the description but from this point forward i'm just going to assume you have all of this installed because it is a requirement for this so what we want to do first is we want to just open up our terminal and we are here in visual studio code so i'm just opening up my terminal now so that we can get started so the first thing is we want to install cdk and using npm the easiest way to do that is just to say npm install dash g aws dash cdk and this is going to install globally the cdk cli tool so that we can just type in cdk here and this should work correctly so you see here there's all this garbage that came up but i don't want to do that yet let's just confirm the version we have is most recent just cdk version okay we have the most recent that's great so let's just type in cdk-help and that should bring up that previous page um so there are a couple useful commands that i want to tell you about before we get into the meat here um so the first one is cdk list which lists all stacks in the application we're going to go over what a stack is in a little bit so don't worry about that cdk synthesize this generates uh your code into cloud formation which is how cdk deploys your resources to aws it's through a cloud formation template the third one is bootstrap now this isn't useful in all cases but it's good to know what it does so if there are aws resources that in order to deploy your stack you need to have those resources provisioned you can do that using cdk bootstrap so it's kind of like a preliminary step in addition if the code that you write generates a cloud formation file for deployment that is over 50 kilobytes you're going to have to use cdk bootstrap to update it so just keep that in mind we're not going to go anything near 50 kilobytes in this video but just keep that in mind if you do in the production case next we have cdk deploy which is the command that we use to actually deploy out to aws via cloudformation then we have cdk destroy which tears everything down and then finally we have cdk diff which is a useful tool to see what's the difference between the code that i've generated and the cloud formation that i've generated against what is existing on aws so it's a very handy tool to you know make some changes locally see how they may affect what exists on your aws account before you actually run the cdk deploy command so those are some of the commands that i think that you need to know about in order to work with cdk so the next thing that we want to do is actually set up or configure our initial application so there's a bunch of different ways to do this if you want to start out with a very bare bones application that has like literally nothing just raw files and you have to fill everything in essentially you can do so using the following command so it's cdk and nipped and app and you just provided the language that you want to use so i'm going to be using python here but if you're using anything else for example typescript you would put in whatever here and this will generate all the project files that you need for the corresponding language however i do highly suggest to use a different command which is going to set up a sample application for us which is just going to kind of show you the ropes in terms of how all the constructs work how everything fits together with a practical example and then i'm going to modify it a little bit and show you how these resources get deployed for our specific use case so in order to do that we're going to say cdk inits sample dash app and same thing we're going to say language python so and i'm just pressing press enter there so this is going to provision everything that we need if i open up our project file here you can see now we have all these new project files that were created and so we're just going to give this a minute here to finish what it's doing and if it takes too long maybe you should go grab a coffee and come back but actually this just finished right away so that's pretty good so now what i want to do is just walk you through a bunch of the different files here that got created so the first one is source.bat now since i am on a windows machine a bat file is going to be generated if you were just on a mac or linux just a source file would be generated and this is just a let me close this terminal so this file just runs a command essentially to set up your virtual environment for python virtual environment is just a way to group your i guess package dependencies so that you can install them to this local project and not pollute your global package namespace so that's kind of why this is useful now we have a setup.pi file now one thing that i wanted to point out here this just contains a lot of just random stuff that you probably don't ever need to modify however you may need to modify dependencies like for example if you want to create a dynamo table you need to import the construct that allows you to do that and so this is where you would do that here under the install requires section you would add your new dependency here so we can see we have a bunch we have iem sqs sns sns subscriptions and s3 so that's what exists as a dependency for this initial project but again if you want anything different you need to configure it here now traditionally this will be done through the requirements.txt file however you can see this is empty so where this actually happens is right here this is where kind of the dependencies are all uh put in place so pip will know how to install everything then you also have a readme um you can go and take a look at this to see kind of what's going on and understand a little bit more about the commands that are offered to you however i'm not going to really walk through that in this cdk json you probably never need to go here but just know that it defines some configuration for your cdk project now app.pi this is an important file this is kind of the entry point i would say for your cdk application and you can see we're just doing some basic imports here and we are creating a core.app and storing it in our app variable now you only have one app typically uh in the cdk project and that's kind of your top level grouping within your app you can have what are called stacks and that's what this line here sorry should i highlight it that's what this line here refers to so we are creating a cdk getting started stack we are passing in our app uh variable and then we're naming this cdk getting started so this is actually defined in another class or another file and i'll show you that in a second however like why do we have this why do we have this notion of stacks well essentially it's just a way for you to logically group different sections of your deployment so for example if i want to use this cdk setup to deploy a bunch of different infrastructure i may create a stack that just deploys the infrastructure for example a dynamodb table an sns topic and maybe an sqsq now alongside that maybe i also want to deploy a dashboard and i want to set up that dashboard to monitor those resources that i created you may choose to create a different stack that contains your dashboard definition and now having two separate ones you can deploy those independently because some of these if they're large enough they can take quite a bit of time to deploy so this is a nice way to organize your resources into logical collections that are fitting to your use case so that's just an example of how this works in real life so we do have the cdk getting started stack which i want to show you so just expanding this file here or this folder and opening up the cdk getting started dot python file we can see some things going on so first of all what i want to call out is that we can see our autocomplete isn't working we can see we're having an import failure here aws cdk cannot be resolved so if you are using a visual studio code just a pro tip you need to make sure the interpreter that you're using matches the interpreter that the cdk project is using so a quick way to do that is do control shift p and you can just find python select interpreter here i'm not sure what this command is on mac but you can probably just search with search for this and then click on that and i happen to know that what i'm using for this project is this bottom one here you can see these are two different interpreters and i'm going to click on that second one and now you can see everything resolves correctly so it's all configured and set up now so just a pro tip for you if you were running into this problem so taking a look at what's going on in this stack here this section right here is where we are creating our resources so we're creating our queue we are using the sqs well that's a big pop-up we are using the sqs library to do that we're calling the q method we're passing in ourselves a name and some properties we're creating an sns topic here calling it cdk getting started topic and then we are saying for this topic i'm going to add a subscription to this queue and then we're passing in the queue here so you can kind of see how powerful this stuff is if you had to do this through the console probably take you half an hour to set up everything make sure that you have all the permissions set up correctly make sure that all the configuration is good or kind of troubleshoot anything if it's not working however using these constructs that are provided by aws makes it very very easy for you to get started but for this project we are not going to be using any of these items because they're going to require some very special permissions which i am not going to set up so going back now to just our initial setup i want to just bring up the terminal here to install all of our dependencies so let me just clear what we had before out so the first thing that we want to do is run this source.bat file and if you're on a mac you just want to run it just doing you know dot slash source um and that should run that file to activate your virtual environment correctly however i am on a window so i'm going to do dot slash source dot bat and then it's just setting up that virtual environment for us let's clear that out as well now we want to install all of our dependencies so i'm going to do pip install dash r requirements dot text you may be wondering why my dependencies were already installed here like why autocomplete already worked that's because i ran this command prior and it installed all this stuff to a common directory on my machine so that's why everything kind of popped up right away but after this completes you should be good to go and it should be resolving all these dependencies and autocomplete correctly so we're just going to clear this out now and we are ready to basically proceed to the next step which is to create our s3 bucket so i want to add a new dependency here so i'm going to say aws underscore s3 as s3 and don't forget your commas here or else everything's going to blow up now i need to go and figure out like how to use this library so what i want to do is just bring up pi pi over here for the aws cdk s3 library and this is um great for seeing some python documentation on how all this stuff works so for this specific one let me make it a little bit bigger here gives us a demonstration of how to create our own bucket this is using a encryption setting which i don't want but this is good enough for kind of a scaffolding to get our own setup um going so i'm going to copy that to my clipboard going to go back over to my project file here and i'm just going to go and paste that in and let's just make sure this is all good actually i don't want this so i'm just going to remove that and bucket is not working correctly so what's wrong here ah of course so i need to say s3 dot bucket so that should work there we go don't forget to name space it with whatever your dependency is here let's just clean this up a little bit too and i'm just going to call this um my special buckets 2021 now remember these are need to be um globally unique with s3 so if anyone has this name this is going to work so i'm just going to add like some garbage here just in case to prevent any air from happening okay so that's all we need to do from like the resource creation perspective in terms of creating our bucket now what's going to happen when we synthesize our stack is that it's going to look at this stack find that we're creating this bucket here of course the app.pi is going to run so you know that's where the entry point is it's going to generate all this stuff into cloudformation and then deploy it to aws so there are a couple different steps that we need to do to make all of that happen so let's bring up our terminal again and in the terminal i'm going to say now cdk synth this is shorthand for cdk synthesize and what this is going to do is it's going to take our local code and it is going to compile it into cloudformation so we can take a look at it this is something i just like to do to make sure my code is doing what it what i expect it to do some of this stuff is just like boilerplate that cdk generates but if you scroll all the way down scroll all the way up rather to the top you see we're creating this resource my special bucket 2021 with a special name and it's an s3 bucket it's adding some default properties here so replace policy deletion policy and some metadata so this is the actual name of my bucket that we're going to be creating so this is kind of a cool kind of preview of everything that's going to get built when we deploy this out so i'm just going to clear this now by typing clear and in order to deploy this what we need to do is type cdk deploy and this is going to fail and the reason this is going to fail with this particular failure reason is because the cdk tool is trying to kind of create all these resources and it's relying on the user that we have configured for the aws cli now for me i have a user configured that no longer exists which is why i'm getting this error however if you have one that's like you know you don't have permission to do this particular thing you'll get a different error so now what we need to do is go configure our aws iem user and policy so that we have some basic permissions that allow us to update cloud formation because that's what aws cdk is doing behind the scenes so we need to have permissions to do that through the aws cdk and then also we need permissions to create whatever infrastructure we're trying to create here so i'm trying to create an s3 bucket that means i need the s3 create bucket policy so i'm going to head over to the aws console really quick to do that so i'm just going to log in really quick here um oh god i hate this so yeah 5q csr let's see how many times i fail that's first bqt nfm that's twice 483c zh please oh thank you um so this should be i believe that and we should be good to go here okay perfect um so just like going back here this is where you would have landed so we're going to the iam section and clicking on iam and we're going to go to the user section over here on the left and this is a default user that i don't want to use i want to create a new one so let's say top left add user and for this user let's call this a cdk deploy user and we're going to enable this with programmatic access i'm going to go to the bottom here and i just want to create this user just to start as is and then we'll go ahead with attaching the correct policies to it so go to next tags next review it's going to give us a warning that's saying you know you're giving you're creating a user that has no permissions this is basically useless are you sure you want to do this i am because i'm going to add that policy later so we're going to just type our click rather create user in the bottom right and now we have our access key id and our secret access key id so i'm going to take that and we're going to go back to our code real quick i'm going to type in aws configure in my terminal and now i'm going to paste in my access key alt tab and i'm going to paste in the secret access key as well so when i attach the correct policies to this user everything is going to work correctly here also by the way be sure to specify the region that you're operating in or else you're going to end up deploying to the default region which i believe is usc 1. so just press enter now once that's done you should be good let's go finish the rest of this for setting up our im user so let's close this we have this user now what we need to do is create a policy that allows us to deploy what we need so i'm going to go to policies on the left hand side here i'm going to click on that we're going to click on the top blue button here to create a new policy and i'm going to do this through json since i've already created this policy in advance and i know exactly what it needs to be so i'm clicking on json here and i have something copied on the side and i'm going to make this statement available to you in github later so that you can kind of copy this you don't have to copy it out of the video which is annoying so i'll put that in the description section too so these are the policies that we need in order to deploy to aws via cloudformation via cdk so we need to be able to describe stacks create change sets describe them execute blah blah blah also delete if you want to kind of tear down your stuff which i recommend and then finally we have s3 create bucket here which is what we set up our our cdk project to do or what we set up our cdk stack to do so that's the policy that we want to apply here i'm going to go ahead and click on next going to click on next to review let's call this cdk deployer policy and going to click on create policy should only take a moment perfect let's go back to users now go to cdk deploy user and then we want to add permissions and then we want to attach the existing policies directly because we just uh created them so now we can just put in the search bar cdk deploy there it is and we just click on that if you click on this little triangle here you can see the policy that you're attached attaching which is what i just created so go ahead and click on review now in the bottom right click on add permissions and you should be good to go at this point i don't think there's any other stuff that we need to do from the iem perspective so now we can go back to our code and if we try to do well before we do that actually i want to type in cdk dip just to see what this does since we don't have anything deployed yet um i wonder what this is going to give us back so it's giving us um actually it's just giving us this so yeah so this plus sign here with the resource this is indicating that if you deploy right now it's going to create your bucket for you so if you had a bunch of different changes that were all present they would all show here so if you're adding and removing adding will show the plus removing will show the negative so you can kind of get a preview of what's going on so now at this point let's clear that i want to do cdk deploy and this is going to deploy our stuff out to aws so this can take a few moments especially initially because it may be doing some bootstrapping right now but what's neat is that this is going to give us some progress updates as it does so so you can see here it's review in progress it's creating the initial stack it's creating it it's doing some cdk metadata stuff cdk creating progress for our bucket which is called this and then it's giving you some indicators here for the progress so for create complete it's done with our cdk metadata now it's done with our my special bucket and now everything is done here so we we've basically created our s3 bucket here so now if i go back to our aws console and i go to s3 i should be able to see what's going on with this new bucket so it should be present if all things work correctly would i call it my special bucket um my special there it is okay so it's appended with this extra thing okay so it's appended with our stack or our app name so not sure if that's default or what but that may be possible to override so that's that's basically it for setting this up so just to give you a preview now of like what does the diff process look like if i want to remove this so let's go back to our code really quick um getting started just lower this a little bit now i'm going to delete this line actually i'll comment this out and now if i bring this back up do clear and do cdk diff this should show us that we're tearing down that s3 bucket so okay so i'm not authorized to perform get template so i should probably authorize myself for this policy let's just go do that really quick to add that into iam i am [Music] and policy so just take two seconds here guys uh cdk and i hope it just doesn't require another one or else that's going to be annoying uh edits json and we want what did it say there um not authorized to perform get templates so grabbing that and put it in quotes get template make sure you add a comma and review policy save changes should be good to go let's try that again uh cdk diff hopefully this works now we need something else actually you know what the reason this doesn't work is probably because it's cache so let me just try again there we go so we can see now that we are um removing our s3 bucket so if we were to deploy this right now we would remove our s3 bucket so let's go ahead and do that cdk deploy and we should see after a minute or so that this bucket gets deployed uh it gets deleted rather and that'll be communicated to us through uh the cdk console as well when we go back into the s3 section of the aws console uh it should be gone so let's just let this do its thing for a brief moment here and you can see it is successful and you know what i actually noticed that we have delete skipped here and this is probably because we have a deletion policy attached to this s3 bucket remember when we were doing cdk synth initially we saw uh some notion cloud formation that says you're not allowed to delete so that's probably why this is happening so that didn't quite work as i wanted it to but just gives you an idea of what can go wrong here i also wanted to show you how to tear down your entire stack really quick so if we just get the name of our stack i believe it was an app.pi just scroll down here so ours is called cdk-getting started so just clearing that you can do cdk destroy cdk-getting started and this will destroy everything in our cdk stack and it's going to ask us for confirmation here so let's press y for yes and it's going to start destroying everything so that you're left essentially with a clean slate so i hope you enjoyed this video and if you did please check out my other ones on aws cdk on the right and as always please don't forget to like and subscribe

Channel: Be A Better Dev

Views: 81,526

Keywords: aws, aws simplified, awssimplified, programming, coding, cloud computing, software design, software engineer, software developer, software architecture, beabetterdev, be a better dev, be a better developer, system design, databases, aws cloud, aws tutorial, aws tutorial for beginners, aws cdk, cdk aws, amazon cdk, aws cdk python, cdk tutorial, aws cdk tutorial, aws cdk examples, aws cdk step by step, tutorial aws cdk, aws cdk tutorial in python, cloud development kit

Id: I2cXlYYoQqQ

Length: 24min 43sec (1483 seconds)

Published: Mon Jun 28 2021