Infrastructure is Code with the AWS CDK - AWS Online Tech Talks

everybody thank you for joining us for infrastructure is code with the ADA guys cloud development kit my name is Jason Fulgham and I'm the software development manager for the AWS cdk hey everyone I'm malade I'm the principal engineer for the cdk thanks for coming we just recently g8 the AWS cdk about a month ago and we're really excited to be here today showing you how you can use the cek to model your infrastructure and use familiar languages to create these infrastructure patterns that you can share yeah it has been a pretty exciting month for us we've been working on this for over a year and a half it's been in Developer Preview for a year a huge shout out to our team and to all the amazing contributors and community who have already been involved in the project it's it's been a delightful experience so far and we're just getting started so here's our plan for the session today we're gonna give you a little bit of information about the main concepts in the cdk and we're going to try to get through that really quickly so we can spend the bulk of our time building an app from scratch using the cdk and then after that we'll save a few minutes we'll show you some resources where you can go to learn more about the cdk ok before we dive into coding which is what this is all about let's do a little introduction and introduce a few concepts so we're all gonna be in the same page last year at reinvent jason and i gave a talk and described a little bit the background for building the cdk what we do what we've observed was that if in the past a typical service would consist of a fleet of hosts and maybe a database modern cloud applications will use dozens if not hundreds of resources and teams just didn't have the tools to be able to manage this complexity we realized that many of these problems have already been solved an application code and so the NBS cdk is a multi language framework or defining cloud applications using familiar programming languages by bringing infrastructure definitions to programming languages we can now use all the incredible tools best practices and techniques that we use to develop applications and solve many of the problems at once when we designed the cdk the fundamental capability wanted to provide was the ability to express cloud abstractions of any complexity we call these abstractions constructs and you'll be hearing us talking a lot about constructs in this session to make sure teams can really leverage their existing skills a cdk is available in multiple programming languages as jason said we have the GAF for support for python and chives grid and we also have Java and.net and developer people so let's see how this works you write your code in one of the supported programming languages applications are structured as a tree of constructs the root of the tree is the app and then within that app you basically compose constructs together that formulate your application it formulates stacks and resources and arbitrary abstractions then you use a cvk CLI which is a command-line tool that we ship with the CDK to synthesize a CloudFormation template from your application um the nice thing here is that you're basically using a period of programming language in order to define the desired state in the declarative way so you basically enjoy the power of cloud formation and as a provisioning engine and the power of programming languages or the accessibility and the ability to actually define your your architectures we kind of like this analogy of a compiler if you think about a compiler takes your source code and compiled it to an assembly language and so the cdk takes your application compiles it down to a confirmation it's the CD case shipped with three components it includes the core framework which is the construct programming model all the stacks the apps basic primitives on top of that we're building a rich class library of constructs for eight of a us those constructs represent all AWS resources and allow you to interact with these resources in a rich object-oriented API the third component is the CLI that I mentioned earlier and we're going to see how it it's used throughout this session to take us through the workflow of developing alright let's get into the meat of this session and start building the application together so in our demo today a lot and I are representing co-workers at a fictitious company called Walters Cove and it looks like the boss has sent me an email and he's made some promises to the marketing team and now he needs me to build this URL shortener as fast as I can for our website so this is going to be a fun demo with the cdk we're gonna get to see how the cdk works and explore a little bit of the construct library and this is the application architecture that we're going to be building today so we're gonna create a cdk application within that application we're just gonna have a single stack and within that single stack we're gonna have an Amazon DynamoDB table that stores the mapping data for us from the short URLs to the long full URLs we're gonna have an AWS lambda function this lambda function is going to implement really our back-end code it's gonna create short URLs and retrieve short URLs and work with the data in the dynamodb table and then last but not least we're gonna wrap our lambda function with an Amazon API gateway endpoint and that API gateway endpoint is just going to make it really easy for us to call our lambda function with regular HTTP request so now that we know what we're building let's jump over to the terminal and let's start building this so the first thing I need to do is carve out a little bit of space for us to build this project and then we're going to use our first CD k command-c DK and knit CD k annette is a great tool for creating CD k projects you don't have to use CD k net you can create your project any way you want and add the right CD k configuration but CD k and it gives you a couple of different templates and several different language choices as well so we're going to use CD k and knit with python we're going to play with a new Python support and the CD k and then now that we've got our project created I want to go ahead and get pycharm started and the first thing I'm going to do is go ahead and plug in all of the dependencies that we need for our demo so I'm gonna give these plugged in and then I'm going to first I need to start up our virtual environment so that pip will cash everything into our workspace here and then I'm going to go ahead and run pip install this is going to install all of our dependencies get them set up in our project so we'll give this a minute to cache everything will give PyCharm a minute to update all its dependencies and let's take a quick look at the rest of the code in this project so the starting point for our CD K application is this app dot Python file and we can see this is a really simple Python file here we are creating our CD K application just like we saw in our architecture diagram and then we're creating this URL shortener stack so this Maps up directly to what we saw in our architecture diagram we've got our CD K application we've already got this URL shortner stack defined for us so let's jump into this class here so the CD k init template went ahead and created this stack sub class for us and this is where we're going to plug in all of our infrastructure so let's go back to our map here and we're gonna kind of build this application from right to left so we're going to start with dynamo DB or model all of our infrastructure and then we're going to test this out and see if we can get each piece working along the way so the first thing I'm gonna do is model this table and to do that I am going to open the cdk API reference and I'm just going to take a quick look at the dynamodb overview so this is the overview information for the DynamoDB module in the ATS construct library so it's a lot of nice information on how to use the different constructs that are provided by the module and right at the top we've got some sample code that shows us how to pull in the DynamoDB module and instantiate a new table construct so this code right now is in typescript and we're working on making these samples available for all the languages that we support but right now it's pretty easy for us to map this typescript example into Python and there's even some documentation down here that gives you more tactical advice on how to do that mapping so let's go back to pycharm and let's go ahead and pull in dynamo DB so we've got this module imported and let's just create a dynamo DB table from the dynamo DB module now whenever you're in stanching a construct you're gonna have this same pattern where the first parameter we pass in is the enclosing scope so this is the construct that essentially contains the new construct we're creating and for the most part you're always gonna pass in self or this and in our example that's exactly what we're gonna do and that just means that this URL shortner stack instance that we're creating we're creating this table in the context of that URL shortener stack so you can think of this table as being contained by this dissolve URL shortner stack instance so that's that first parameter it's typically going to be cell for this and that's how you connect everything up together this next parameter is an ID so we're just gonna call this our mapping table and we'll see how that ID gets used a little bit later so the last parameter we need to remove fly we can see over here in the documentation the partition key is a required parameter we have to specify so let's go ahead and start defining that if you've ever used dynamodb before you know that the partition key we need to specify is an attribute so DynamoDB dot attribute we're gonna create a new attribute to model what we want this partition key to look like and it's going to be named ID and type is gonna be a string type we're going to use this attribute type class to grab that constant there for us so one line of code we've got our DynamoDB table modeled now let's jump back over to the command line and let's take a look at how we get this deployed so the cdk deploy command is going to synthesize going to run our program and it's going to synthesize a CloudFormation template and then send that to CloudFormation so if we look over here in the dynamodb console we see that there's this brand-new table and you can see this mapping table that ID that we specified is showing up there in the name and we can see this table is being created so let's jump back to our slides we've got our dynamodb table modeled the next thing we need to do is create our lambda function so back to PyCharm the first thing I want to do is carve out some space for our lambda code so let's create a new directory called lambda and let's add a new file in there we're gonna call it handler Python and then to save some time we're gonna go ahead and plug in all this handler code so let's walk through this code real quick we've got just shy of 100 lines of pythons this is a pretty simple little Python script here and the main method what we're doing is figuring out if we need to dispatch this to the handler for creating a new short URL or if we need to pass it off to the method that retrieves an existing short URL so if we get dispatched to this method you create a short URL we're gonna look in the environment and grab the table name environment variable and that's going to tell us what table we're working with so we know which the name of the dynamodb table that we want to store and load data from we're going to talk about why we do this in just a few minutes but essentially this is what is allowing us to let our application code be aware of the infrastructure that we provisioned and know which resources to talk to we're going to create a new unique ID and then we're going to use the AWS SDK for Python to store that into our mapping table the ID and the full long URL to target URL and then we're just going to return a 200 response in this case and pass that URL back to the user to use if this was a request to retrieve a short URL we're going to do something very similar again we're gonna pull the table name out of the environment then we're going to use the AWS SDK for Python to read that ID from the DynamoDB table and then if we find that we're just going to return a 301 response to the client and that 301 HTTP response is of course just going to force our web browser to redirect to that full long target URL so a pretty simple Python code that we're working with here now that we've got the actual code that's going to run in our lambda function let's figure out how we can model the lambda infrastructure that we need to run that code so just like before I am going to use the API reference for the CD Kay and if I look at the overview page for the lambda module I've got a lot of information about how we can configure lambda infrastructure and again right at the top is the sample code I need we need to pull in the lambda module then we need to instantiate this function construct and we're gonna pass these three variables are these three properties into the construct to configure our infrastructure so let's go see if we can figure out how to do that in PyCharm first thing we want is to pull in lambda and then next we want to create this function construct so from the lambda module we're just gonna instantiate function same thing as before we're gonna follow the same pattern for every construct we ever instantiate passing in the enclosing scope and then passing in an ID we're gonna call this back end because it's essentially implementing all the backend logic to create and retrieve those URLs and now we start to get to the lambda function specific properties so again looking at the documentation over here I see that we've got these three required parameters everything else is optional I've got to specify the code the handler and the runtime so let's start with runtime so runtime is just gonna tell lambda what language runtime to execute this code with so there is a runtime class in the land of module and that enumerates all of the different runtime options for us we're gonna use Python three seven for this example next up I'm going to specify the handler and the handler just tells lambda where to start executing code we put all of our code into a main method in this handler file so we're just gonna call it handler dot main and that's going to tell lambda to start executing from that main method and then last but not least of course we have to tell lambda where to get our code from and to do that we're gonna use another neat feature of the cek and that's the asset subsystem in the cdk so we're talking a lot about how we use the cdk to model our infrastructure but of course we also have application code that's using that infrastructure and we want that all to be tied together very seamlessly so assets are what allow us to tell our infrastructure what runtime code needs to be packaged up and configured for our infrastructure so again in the lambda module we've got a class called code it makes it very easy for us to package up assets and we're just gonna pass in this lambda directory that has our code in it and that's going to tell the cdk that this this is an asset that needs to be available at run time and it's going to configure our lambda function to use the code from that asset so that was it in that line of code we now have a new lambda function modeled let's jump back over to the terminal and let's run cuk deploy again so we can get this infrastructure created for us and notice that it's a little bit different this time we didn't see this prompt the first time we were in cdk deploy but in this case we're making some changes to permissions and our AWS account and we want to make sure that you understand those changes and that you explicitly acknowledge anything around security changes because that's so important so in this case we've added this new lambda function and that lambda function needs a role to execute with so that's exactly what we're seeing here that makes sense to me so I'm going to go ahead and let that deploy and let's take a closer look at our code we've got our table created we've got our function modeled but this code isn't gonna work together yet there's two problems that we need to fix the first one is that our function doesn't have permission to work with the data in our table yet so we've got to use the cdk to express our intent that we want this function that have permission to work with the date the data in the table so if you've ever used I am policies before you know that it can be difficult to configure a very high quality I am statement and lock everything down to resources and actions with the cdk the kind of overall theme that we're going to be seeing over and over is that the cdk allows you to abstract and work at a higher level and express your intent and let the cdk do some of the work underneath to tie that together into all the details so let's take a look at what we have on table if we look at this table object we see we have a lot of different methods available to us in particular we have all these grant methods so these grant methods are what we use throughout database construct library you're going to see this pattern and this grant method is how we grant permission to resources so this is just one of the patterns that you'll see throughout the construct library permissions is a big one there's also patterns around configuring event sources and working with metrics but in this case we're going to use this grant readwrite data method and we are gonna pass in our function and that's all we have to do so if you can compare this with the experience of writing a I am policy this is much easier to read we don't have to know about all the low-level details we can just trust the cdk it's gonna create that nice tight security policy for us and in a minute when we see what it creates we're gonna see that it's very very scoped down to only this one single table for only two actions that we specify those are read and write actions and only for this lambda function so we get the benefit of this high-level intent that we express is very easy to read but at the same time we have all the security lease privileged best practices going on ok so that one line of code solves one problem for us but we still have one more problem the lambda function code that we looked at remember it relies on the environment variable that table name in order to know which dynamodb table to talk to so let's take a look at what's available on function and this very first method that comes up at environment tells us adds an environment variable to the lambda function that's exactly what we need to do so we're gonna add this table name as an environment variable and we're just gonna pass in table dot table name so the result of this code is going to be exactly what we expect our lambda function is going to be executing and the environment is going to have this table named environment variable set to the right table name but if we look at this a little bit more closely we see that there's actually something pretty neat going on so remember that when we execute our cdk application what we're really doing is creating this desired state this model of what we want our infrastructure to look like we're not actually provisioning the infrastructure when we run this code we're only creating this desired state representation so at this point in time when this line of code runs we're not actually going to know what that table name is so what's actually happening underneath the covers here is we're using the token system in the cdk and tokens allow us to do this late binding at provisioning time so even though we don't know the table name when this code is run we're gonna output in our desired state this kind of Ford reference that allows it to be resolved at provisioning time and we end up with exactly what we expect happening and getting everything wired correctly when it provisions so let's jump over to the terminal and let's let's introduce one more new cdk command so that is cdk diff cdk death is probably my favorite command as you're iterating on your cdk application it's just incredibly useful to be able to see what's different between what I've changed locally in my workspace with what I've deployed so in this case we see that there are two sections in this diff view this first part is going to highlight any security changes and we see exactly what we talked about when we added that one line of code to grant that our function permission to write toward dynamodb table we see that this is locked down to this specific mapping table no other tables are affected by this the actions are locked down to all the read and write data actions available for dynamo dB and it's granted to our our back-end lambda functions service role so that all checks out that's exactly what we expected to see scrolling down to this next section we can see all of the changes for our resources so we've got this new policy that we just talked about that grants permission for our dynamodb data and then we can see our lambda function already exists but it's been modified this environment variable table name has been added just like we talked about and there's also this dependency to make sure the policy has been created so all of that checks out so let's go ahead and do the CKD deploy we're going to acknowledge these security changes that we've already reviewed and we're going to let this roll out and get this infrastructure provisioned so we're getting pretty close now we've got our DynamoDB table we've got our lambda function code hooked up the last thing we need to do is wrap our lambda function in this 8 API gateway endpoint so same thing as before I'm gonna go back to the API reference I'm gonna look over the API gateway module and I see that this lambda REST API this is really the easiest way to wrap an API gateway endpoint around a lambda function so that's exactly what we're gonna use in our code same thing as before we're gonna pull in API gateway we're gonna create an API and it's this lambda REST API that we just talked about creating we're gonna see the same pattern for every constructor we create and then the only required parameter we need to specify here is the handler so for handler we're just gonna pass in our function so one line of code was all it took for us to create this new API gateway endpoint and wrap it around our land of function let's go back to terminal and it looks like our last deployment has finished so let's go ahead and deploy this API gateway endpoint and then we will test it out so again we see that there are a lot of permission changes we've already reviewed these we know that we're adding Amazon API gateway and run our lambda function so this all checks out let's let our deployment go and let's talk about one more piece while this is rolling out so when this deployment finishes we're gonna have an Amazon API gateway endpoint that we can hit and we can access our URL shortening service so that's great but it's gonna be this long Amazon API gateway specific URL and we would really like this to be hosted as on our main company's domain at Walters Coco so what we really like to do is add this as a sub domain make the API available under our main domain name so we get a nice short URL and we just have a very professional-looking service running so in order to do that we're going to talk about how we share some code inside this company but before we get that let's just do a quick sanity check and make sure that our API gateway endpoint is working before we hook it up to the main site so I'm gonna plug in our URL here this is looking great just like we expected we get the 200 response with usage info and we're gonna plug in target URL let's see yeah let's go to one of my favorite sites cdk homepage on the AWS site so we get another 200 response and we get this URL and you can see this is what we were talking about before this is the API gateway default URL that we get we'd like that to look a little bit nicer and our next revision but let's plug this in and see if it works so sure enough you can see that it very quickly redirected us to the target URL just like we asked for so everything's looking good or our API gateway endpoint looks like it's working correctly so let's go back and talk about how we actually get this hosted on our company's main domain so one of the great things about the cdk is that it lets you shape your infrastructure and create these infrastructure patterns and share infrastructure very easily between teams just like we share libraries of code so our company does the same process we have this Walters Co common module that has all of our infrastructure patterns in it it has access to our domain to our certs to all of this kind of common infrastructure that all the application development teams need and we've got that hosted on an internal Python module repository so we're gonna go ahead and add that to our project that's Walters Co common so we've loaded in that new dependency from our internal repository let's take a look at how we use that so from Walters co common we are gonna import this Walters code stack and that's just one way one example of how you can share infrastructure patterns we have overwritten or subclass the core cdk stack with this Walters Co stack and all the application dev teams use that as a common base for the stacks they build so in this URL shortener stack we've swapped it out to extend from Walters Co stack and let's take a quick look at what we get from Walters Co so right here the very first method that comes up we've got a map Walters Co subdomain and the documentation says this is gonna map a subdomain of Walters cocoa to an API gateway endpoint so that sounds like exactly what we want the sub-domain we want to use we're gonna keep it nice and short we're just gonna call it go and then the API is of course the API we just create it we're gonna pass that into so let's jump back over here now that we've got this hooked up into our our main website we're gonna go ahead and deploy the changes and again we expect to see some permission changes we know what these are we've already reviewed them so we're gonna let them roll out and let's give this a minute while this infrastructure is provisioning and everything is getting set up let's jump back to our slides and quickly recap what we did in this first part of the demo so here's the architecture that we've ended up with we've got our cdk application we've got a stack that extends from this shared stack that we use inside our company that gives us access to our domain hosted by route 53 as well as the cert and then of course we we walked through building these resources using the dynamo DB constructs and lambda and API gateway endpoint and while we're building all this we got to see how we use the CDKs CLI we used an it to create our project we use deploy to give everything provisioned to AWS and we use diff to see the difference between what we've modeled locally and what we've already deployed out to AWS there's a few more CDKs CLI commands synth is one that you might want to use to see the synthesized desired state configuration that's going to show you the CloudFormation templates that cdk creates for you so you might want to share those with other teams or use other tools to review those or you might just want to dig in and see exactly what the cek is producing the other big thing we saw in this demo was just taking a tour of a TBS construct library so we saw how we can use the documentation and the API reference to explore the construct library we also started seeing some of these patterns that are used throughout the ADA based construct library particularly things like granting permissions that again let us work at a higher level and express our intent and let the tooling deal with all the details and making sure they're correct so let's jump back to terminal and it looks like our deployment has completed so we've got the moment of truth now let's go to go to Walters cocoa and see what we get so so far so good we've got the usage information let's plug in target URL let's go to the cdk home page again and sure enough we've got this URL created it's nice and short and let's plug this in and see if we get redirected awesome so we've got the URL shortener built it seems to be working correctly and we've got it uploaded and running underneath our main domain name so I think my job is done here the boss should be happy we've got the URL shortener built and it's ready to go okay so Jason's home and I'm left to always left with the stuff to do again an email from the boss asking how are we going to monitor and load test the URL shortner so that it'll work for the email campaign and that's that's a legitimate request you know if we're not going to test it we'll never know if it's working and to be honest I'm not exactly sure that this architecture is the right thing we're using DynamoDB here I want to make sure that if we hit a single entry point in the Dynamo to be in that peak it's gonna withstand that we provision enough capacity there's there's some stuff behind this service and so what we're gonna do in this part of the demo we're gonna first generate some traffic and so we'll create a little go generator traffic generator and hit our service to make sure that it's actually withstanding the load and then we're gonna add some monitoring to our system so we can see what's going on and make sure that we have visibility into the production system okay let's just get going so I synched from Jason's repository you can see that if I run cdk diff here it'll tell me that I don't have any changes in my application which is great how do we generate load for a URL shortener and so I've got this URL that I created earlier it redirects to amazon.com and so basically an email campaign would have this type of load profile right like all of our users are gonna hit the same URL and going to be redirected to the same response so it's basically a bunch of HTTP requests going into the same record the same URL so let's create a little script I'll create a directory called pinger and let's just start by just doing it with - right while true do just hit this euro every second that's a really simple script it basically sends an HTTP GET request into a single euro I can I can run this and make sure that it works let's see yep looks good and now what I'm gonna do I'm gonna wrap it in a docker container and I'm gonna use Amazon Fargate to orchestrate this and run it on the cloud because I'm gonna want to run it over the weekend and make sure that it runs I'm not gonna be able to run it from my own machine and so let's create a simple docker file here we'll use Alpine as the base image we're gonna install see URL we're gonna add our little ping script and we're gonna use it for our entry point one thing that I think I did here that's wrong how buddy doesn't have bash okay and let's just even try this doctor build pinger run well again neat okay so this little container generates a single transaction every second which is the basic building block of a traffic generator now let's go to our cdk application and see how we can define our Fargate service to run this docker image so I'm back at my code and what I wanted to show you now is this traffic generator thing is actually a pretty usable thing a pretty useful thing we have another team in our company and our fake company that wants to run some load load track alodia load generate some load against their service next month and they said they asked if we can give them some of the stuff that we're doing and so instead of writing this inside my application what I'm going to do is I'm going to create a custom construct that will represent my load generator I'm gonna create it over here as a separate file but as you know if you're if you're familiar with Python like it's very easy to take this file wrap it in a library publish it to pi PI publish it to our internal repository and so this is gonna give you a sense of what it make means to create custom constructs in the cdk which is really an important capability and wanted to show you we'll call it traffic oh all good reusable components have good fun names don't ask please okay so what is a construct how do you define a concert we wanted to make that as easy as possible and so a construct in the cdk is basically a class that extends the construct base class so what I'm going to do is I'm going to import the construct base class and I'm going to define my traffic co class as a subclass of construct and so this is basically what you need to to create a construct and I mostly like any other code that you've seen already the main business logic of the construct happens in its constructor and so we're going to create a constructor here an initializer in Python terminology we'll call the base super-super and like Jason told you earlier all constructs have a similar signature they expect scope and an ID so those here and basically this is it this construct is completely viable construct I can use it inside my cdk application and we'll do that in a second now I wanted to create a far gate service so let's let's just explore the library through through our ID I'll import enemy from AWS cdk import AWS e CS and a B SEC - okay so we need a far gate service so I'm going to go ahead you find Fargate service itself and this is basically our pinger service and you'll see that it basically except it requires if I look at the error here sorry besides the pedantic pepp aid it's it tells me that I need a cluster and I need a task definition so let's create one market services run within ECS clusters so I'll need to do pcs cluster and then I'm gonna do self and pink whatever we'll call it cluster and what I want to do here I want to run this cluster within the VP see that my company has like we have a central V PC all our applications run within the V PC so it makes it easier to connect to each other and in many cases I actually wanna generate traffic to an internal end point not to an end point that goes on the internet and so what I want to do here is I want to make the V PC a parameter to the traffic co-construct and so I'll just do this for now and what I'm gonna do here is I'm going to add a V PC keyword argument and it's gonna be of type ii c 2i v pc and the reason i'm using IV pc which is an interface is because we have a pattern in the cdk that allows you to either reference a constructor was created within your app in this case it would be V PC or a resource that was imported and so this V PC specifically I'm not actually defining that in my application all I want to do is import it from some existing ID and we'll we'll see how to do that in a second but for all intents and purposes for this tráfico cons front is just an input so I think it's pretty straightforward so yeah let's just put the cluster here we have cluster cluster and then the second thing we needed was a task definition so ECS and Fargate has this concept of a task definition it basically allows you to define which containers need to run together and then you can like how many times you want these these tests to run so I'm just gonna create a test to finish it here Fargate test efficient and finger task and you'll see that a test efficient accepts a bunch of parameters I actually don't need to specify any parameter at this asset finition level I need to add a container to my test definition and then that's going to be our actual you know pinger and the pinger accepts a required parameter called image and that basically defines the docker image that's running within that that task within that that the container is based on and so now we need to basically reference our little docker image that we've created you see that it's here in a separate directory and and now we're going to go back to using cdk assets which edition showed you earlier how to use assets to define lambda code you can do the same thing with docker images and the experience is very similar you go AWS CCS container image dot from asset and then you basically specify a directory and so in this case I'm going to just say Peter because it's a directory relative to the root of my application the second thing I need to specify here if you recall was the environment variable I need to tell it which URL to hit and so obviously if this is a general-purpose traffic generator the URL is going to be something that I expect as an input and so I'm real here if I want to be pedantic I need to put star star here or star something like that yeah means that I need to specify all these as keyword arguments and not position arguments okay so I'm going to specify the Tesla finish in here as test it and this is basically all I need to specify you see that I don't have any other errors here for required arguments but one thing that so this is gonna work but what's what's gonna happen is gonna generate a single transaction every second and that's hardly the load that I'm expecting I'm actually expecting much and so in order to actually scale this up all I need to do is duplicate this container and run it as many times as I want based on the transactions per second that I want we have this really nice one-to-one mapping between the concept of transactions per second and the number of containers running in our Fargate service and I wanted to talk to a little bit about this idea of abstraction right like the traffic generator here the mental model that a user has when they use a traffic generator is how much load to send over to a specific URL and so they can specify the URL and they can specify transactions per second let's see okay that's like that's like something that they understand right like they don't care that under the hood this abstraction is using Fargate using containers using some bash scripts it doesn't matter right like they just care that they want to send this amount of traffic to this URL and I could change this implementation behind the scenes as long as it does what they expect it to do that's great and so cdk gives you the ability to define these abstractions and I think what we're gonna see is we're gonna see people developing that skill of defining abstractions for infrastructure components which are there's very very similar to defining abstractions in your application code but we weren't really used to doing that and in past and so I think that's a new thing from any of the users that we see today that is really fun and interesting and creative and okay and so what we're gonna do with this TPS is basically gonna say is art count equals TPS this is the this is the translation of the mental models okay okay so we've got our little construct this set now we need to use it right like it's basically just a class right now and I'm gonna go to my application and instead of actually you adding that construct to our URL shortener stack what I'm going to do is I want to create another stack for it because it doesn't make sense if you recall from our little graph here we wanted the traffic generated be in a separate stack because we're just gonna we're just gonna you know destroy the stack at the end of the test and we don't need to touch our application so it makes sense that those things are deployed separately traffic stack and that's also going to be a Walters toast at because all our stacks in in our org are based on the on that base tag and we're gonna do the same thing that Jason did here so I'm just going to spend my less time doing that and now I'm gonna import from tráfico import traffic go and so this would be lets say i'm publishing this to pi pi to pi pi or to our internal package managers so this would be something that we need to pip install and then I get this little nice this little nice construct and you see that when I'm instantiate a fika it looks and feels exactly like all the other constructs that jason used earlier it has scope ID and then it accepts these three parameters v pc gyro and TPS okay so scope is always self and then traffic whatever test traffic and then v pc so for v pc what I'm gonna do is I'm gonna use the shared V PC that we have in our company and I'm gonna show you that basically if I go to self we have Walters Co V PC which is published by the walters co stack and so similarly to the map Walters codomain method that jason used earlier to work with the shared domain resource we have API that allows us to work with the shared V PC resource and so like this idea of using a base CD k stack as a way to share resources between teens is something that we're starting to see with customers and it's you know it's a simple object-oriented tool that we can now leverage okay so we've got sorry that was the V PC right and then URL I'm gonna be lazy sorry I'm gonna be lazy I'm just gonna use this URL for now we could have you know passed in the URL from the other stack and generated and but I want to keep something simple for now and then TPS let's say our it seems like our dynamodb table initially is provisioned I think with five let's do ten see how it how it behaves so 10 TPS okay last thing I need is to actually instantiate my stack because now I've only declared it I haven't really defined it and obviously this Mont one needs to be here because again we're very very organised so I'm gonna go to my app I'm gonna import my traffic stack and I'm gonna add so now we've defined the stack let's deploy it so I'm gonna do cdk deploy test traffic that it's starting it should show me some nice information about any security changes that this requires and that makes sense because okay so well this is deploying and you see that it's building the docker image for me which is pretty neat let's go back to our presentation and move on to the next the next part of the demo which is the monitoring aspect we're gonna use cloud watch dashboards and cloud watch alarms for monitoring and cdk today has a pretty neat API to define these dashboards you can actually programmatically create these dashboards widgets and grids but it is still pretty involved like if I want a nice dashboard I would need to understand all the metrics that are emitted by each resource and create alarms for them for like with like reasonable values and and I I really don't want to do that what I'm going to show you is how we can use a third-party library that someone incidentally it was me published two pi PI that does that for us just demonstrating to you that what reusability is about right for infrastructure it's it's a very powerful idea okay so I'm going to do a Tamiya cdk monitoring and we've got this this library here called cdk watchful and if I go into it watching your CD K back since 2019 that's nice it's a nativist construct climate it makes it easy to monitor CD k apps and you see that it's using JSI I similarly to what we use in the CD K to publish this library to them all to pull languages so it has support for Python and for typescript which is great and the way works is you instantiate the watchful construct and then you basically call watch dynamo table watched on the function and you pass in these resources and you get this nice little dashboard with alarm set up for you which is which is great cdk watchful is using this concept called aspects and we'll talk about in a second okay so I'm just gonna use this one so pip install cdk watchful is my first step terminal so I'm gonna bring in this library and then let's see is this still working yeah okay so I'm gonna go to my euro shortener stack here and I'm gonna import this library from cdk washable import watchful and then I'm gonna go to my main stack because as I showed you earlier I want my monitoring system to be part of my main stack right like it's an integral part of my application it doesn't make sense to deploy those separately it could may set it make sense to deploy those separately in certain circumstances it's your choice we're just gonna add it to the main stack so I'm gonna do watchful self-monitoring I can even specify an alarm email so I'm gonna specify Jace's email here why not and listen let's look at this API so I've got watch lambda function and watch dynamo table oh it's pretty nice even add you know specific alarms and sections to my dashboard but there's another interesting API here called watch scope and what this API is doing it let's let's look at our docs for a second watch school's watching scopes watchful can also watch complete cdk construct scopes it will automatically discover all watchable resources within that scope recursively add them to your dashboard and configure alarms from them which is pretty neat because that will save me a lot of time and I really do want to watch everything in this stack so I'm gonna just just do this right I'm just gonna create a watchful instance and tell it to watch this stack and it'll automatically find all the relevant resources we'll see that in a second okay are you done yeah okay so supposedly we're generating some traffic we can't really know I can check out that we have some um maybe I can you know just for the sanity I'm gonna do a little test here and see that I have my little easiest Fargate cluster default defined yep that looks like it's doing something I've got run ten running tasks which is what I asked for and so the next thing I'm gonna do is I'm gonna do cdk diff to see what's gonna happen after I've added this monitoring system and it looks like it's doing what it needs to do right it created a subscription for for Jason it created a dashboard it created alarms for me I'm just gonna deploy this this thing star means deployed oh sorry not diff boy star means deployed all the stacks and obviously my traffic generator hasn't changed so it's supposed supposedly it's shouldn't do anything ok so while this is deploying let's go back and recap and then we'll go we'll come back it's over so what have we seen here first of all we've seen far get is awesome like I can't stress how how much heavy lifting is being done for you to take this little docker image and just like spin up ten instances of it running as easily as this it's all working together within the cdk workflow so any changes to the docker image would automatically reflect an update it is awesome we talked a lot about reusable constructs this is the main thing that we want to you guys to take from this talk that this idea of being able to create arbitrary abstractions share them reuse them publish them to package managers we talked a little about API is encapsulation mental models and then in when we when we saw the monitor and we saw how to use a third-party concert library we found it on pipe I think about this idea of like this little cloud and a library or a service in the library we also saw that it's possible to use JCI which is a technology that we use to publish a CD k to multiple programming languages for any other library ok so let's see if our little yeah I even get a little link here for the dashboard that's pretty neat boom cool so we've got our little mapping table this is Dynamo table monitoring we can even see that it it started getting some traffic here we see the backend function getting some invocations we see the API gateway getting overall calls latency perfect this is what we need Jason's gonna wrap up thank you all for coming to this session today I hope you enjoyed the demo a lot and I really had a lot of fun building this with a CD Kay and I hope it inspired you to get your hands on a CD Kay and explore it yourself for next steps in terms of learning about the CD Kay and exploring it on your own I hope that you will all go check out CD k workshop com this is a little online interactive workshop that we built and it's available in typescript and in Python you can go and get your hands on the CD K and learn about the main concepts and have fun building a small little app and of course we're on github we're very active on github and git er we'd love for you to connect with the development team let us know how your experience is going with a CD K if they're features missing or any issues you've hit we'd love to hear from you yeah thanks so much for coming one of the things we talked about a lot in this session and showed you was how easy it is to create your own constructs and so we're really curious about what you guys are building with the CD K and we encourage encourage you to like share it with your team your company publish it to the O click one of the open source package managers we think this can have incredible impact on how people builds cloud applications where we can't wait to see what you guys are gonna build with it so yeah thanks so much for coming

Channel: AWS Online Tech Talks

Views: 47,153

Keywords: Infrastructure as code, yaml, CDK, AWS, Webinar, Cloud Computing, Amazon Web Services

Id: ZWCvNFUN-sU

Length: 58min 13sec (3493 seconds)

Published: Tue Aug 20 2019