AWS re:Invent 2018: Infrastructure Is Code with the AWS Cloud Development Kit (DEV372)

everybody thank you for coming to dev 372 infrastructure is code with the AWS cloud development kit my name is Jason Fulgham I'm a dev manager for the AWS cdk project and we also have a lot ben-israel here with us today Allods our principal engineer for the cdk so in the next hour a lot and I are gonna take you on a tour of the cdk and show you this new software development framework for defining your cloud infrastructure as code our session is going to be split up into four parts today we're gonna start by giving a little bit of context and talk about our journey through infrastructure management talk about some of the challenges that developers face and some of the best practices that have emerged then we're going to quickly give you an overview of some of the main core concepts you need to know to work with the cdk and we're really gonna spend the bulk of our time in this session and demos so we've got a pretty fun little demo application we're gonna build from scratch using the cdk and we're gonna see how the cdk really changes the developer experience around defining your infrastructure and finally we'll wrap up with some recommendations on where to go to learn more about the CDK so I'll turn things over to a lot to talk about infrastructure management Thanks hey guys welcome thanks for coming we're pretty excited to be here and talk to you a little bit about this project we've been working on before we get into coding which is basically what this thing is about let me tell you a little bit about my journey around infrastructure management so I used to work for amazon.com before joining AWS the retail website and as you know Amazon is an AWS customer and we use AWS like anyone else and my team initially had a pretty simple service we had a bunch of ec2 instances consuming messages from an sqs queue it's a pretty classic architecture for an fern for a retail team it's it's quite scalable it can handle a lot and the way we manage our infrastructure was basically go to the ADA base console log in and we had a pretty pleasant experience you know click through some buttons and change some knobs and provision or queues in our fleet and whenever we needed to create another environment we had a little document and you know people followed that little document and created these as a couple of resources and that model sustained quite well for quite a long time but then two things happened well gradually two things happen one is we needed to provision more and more environments I'm sure many of you run into this you expand to a new region you need test environments integration environments development environments and so many people needed to create these these resources the other thing that happened is that there were new services coming in and we wanted to leverage them DynamoDB came out Kinesis came out our surface became more and more complex and relied more and more on those resources and our little document became a big document and people started making mistakes and it took a long time more than we wanted and so obviously as a coder I was like let's find a way to you know codify this problem and and automated and so I ran over and picked up the Ruby SDK which is really awesome and started writing a little ruby program to provision my resources and that turned to be pretty useful like we could literally throw away this this document and now everybody that needed a new environment just ran this program and voila you got yourself a stack deployed so that was a huge improvement and it worked really well until we actually wanted to change things right and so when you want to change things you can just create those resources you actually need to update them and I find myself dealing more and more with that aspect of how to provision my infrastructure right because some people had environments in different stages so this code that needed to take into account the current state and figure out what's the next state how to update what happens if it doesn't work how do you roll back you know it's all kind of okay for development and testing but when it comes to production it becomes not kind of okay to miss things like that and so at some point we realize this model doesn't sustain anymore and we found out about provisioning engines which I'm sure most of you are familiar with research provisioning engines are very powerful tools that let you think about your infrastructure a little differently instead of thinking about what set of operations I need to issue against the the surface in order to provision my resource I can think about which what is my desired State I define my desires say using a configuration file CloudFormation which is the provisioning engine for eight of the u.s. uses a file called AWS CloudFormation templates terraform is a very popular provisioning engine used in by a DBS customers and they use HCl as a configuration file format but essentially they work the same way you hand them over this desire safe configuration file and they look at your stack and determine what are the steps that I that need to be executed in order to reach that desired state which is exactly the problem that I was trying to solve at that time and it's it's it's pretty magical right like you really think declaratively about where you want to be and all the heavy lifting is it happens in the provisioning engine rollbacks updates it's really cool however at some point I find myself spending full days of copying and pasting llamó files and that didn't feel right like it didn't feel like what I need to do in order to you know create like reusable things or create multiple instances of something and and I needed a better solution and and my intuition was that this file is not something that humans need to write like it's basically a big list of resources and prop it felt like an assembly language more than a more than actual programming language and so obviously I went over to the internet and found these really awesome community-driven projects that allow you to generate CloudFormation templates it's a pretty natural thing to do right like when you have a big big configuration file and there are a couple really awesome projects a few really awesome projects troposphere is probably one of the most popular ones it's in Python it has this really nice class library that allows you to basically interact but not interact but like produce at CloudFormation template through code and so back to code and that's where I'm feel comfortable at least and I can define my infrastructure logic again I can create for loops and if statements and basically create a mental model of what I'm trying to do instead of actually you know copying and pasting JSON files we still using the desired state model we're still defining what we want and not how to get there but what happened is it instead of copying and pasting Jason and yeah no I find myself caught I found myself copying and pasting Python code we had other people in our group that started using Pro totus fear and they had similar you know they're similar infrastructure that they needed to provision and it was really hard for us to actually share that stuff we couldn't the the thing was that we that the framework itself didn't have an intrinsic capability to create abstractions and to me this is what software is about it's about creating abstractions and not being able to create these abstractions was the main motivation for us to start talking and thinking about the CDK and that's when I moved over to AWS to work with Jason and this really great team because I thought it was a problem worth solving so let me hand it over to Jason he's gonna tell you a little bit what we came up with and where we are so this is such a typical story that we've heard from so many of our customers we've experienced it ourselves as developers I think the the it is really that we have this wealth of cloud services we have hundreds of these different cloud services we can take advantage of and our applications are using more and more of them and the configuration for these cloud resources begins to come become more and more complex and so developers really need more powerful tools for managing that complexity and we really really need powerful tools that let us take these patterns and naturally emerge as we put all these cloud resources together and turn them into these reusable shareable abstractions so that is exactly why we started building the AWS cdk I'm gonna tell you a little bit about some of the main concepts in the CD Kay and then we're gonna get into some code so the CD K is a software development framework for defining your infrastructure and your cloud resources so that code we call your CD Kay application we'll see that in just a minute within your CD Kay application you define all the infrastructure stacks that you need in your system so that could be one infrastructure stack it could be dozens of infrastructure stacks across multiple regions within each of those infrastructure stacks you define what we call a tree of constructs and construct is really the key abstraction it's really the core of the cdk programming model as we'll see throughout this session and at a high level a construct is really some configuration of cloud resources so it could be a bucket or it could be one cloud resource or it could it could encompass dozens and dozens of cloud resources so just like we saw before with troposphere we're still building on a software development framework we're getting all the goodness of using our programming tools we've got in our IDE s we're factoring tools navigation tools inline documentation content assist so we've got this really rich environment for building this code just like any other code we're still building on cloud formation so we've got this powerful provisioning engine underneath and we're working with this model of desired state configuration we describe the outcome and code that we want our infrastructure to be and we rely on the provisioning engine to actually make that happen and get us there so the we talked a little bit about the programming model and that's really what makes a CD case so powerful is this ability to share and create these abstractions and the cdk is really built from the ground up around that concept but in addition to the programming model we also ship the AWS construct library with a cdk so the ad based construct library is a set of constructs specifically curated for making AWS infrastructure really easy to define and we're going to be using the construct library throughout the demos so you'll be able to see what kind of support is in there and the construct library really gives us a lot of flexibility to easily define our infrastructure so setting up permissions for different resources configuring event sources and much much more and finally the cdk is very new project we're in Developer Preview we launched earlier this summer you can find all the source code out on github of course it's fantastic time to get engaged early in the project we think it has a lot of potential to really change how developers define their infrastructure so we'd love to see engagement from the community of course so I turn things over to a lot and we're gonna get into some code ok so hello world right we got to start with that that's what we came up with it's a cue you send a message to the Q lambda function consumes the message from the Q and writes the contents of the event to clog watch logs pretty simple architecture we're going to talk a little bit about cdk stack say decay applications cdk constructs the concepts that make up the CDKs model will show you the command-line interface called cdk toolkit and that command-line interface allows you to interact with cdk applications and walks you through the workflow of working with these data Kay applications and will obviously work with the IDs contract library because we need to provision those resources without further ado let's open our terminal here okay starting by creating a new directory and we're gonna initialize the project using a command called cdk in it it's basically a little templating engine that allows you to bootstrap you know cdk projects it's important to emphasize that those projects are just regular idiomatic projects and every like in like you do in every programming language we've got current as jason said we've got current support for typescript javascript dotnet java python is coming up and we plan to add more languages in the future so let me just kick that off quickly with a typescript project and we're gonna use typescript for this demo but you can create the exact same demo and the exact same application and one in each one of those programming languages okay so we've got our new project directory here and it comes bundled with a bunch of useful commands if you're familiar with typescript you'll know that we need to compile types we've to JavaScript in order for to execute it and so we're gonna run the watch command which basically looks at files in the background and compiles Emma just open a new table tab and I'm just gonna run the watch command here in the background so I don't have to think about compiling the code all the time okay back to our original tab and I'm gonna kick off our ID we're gonna use vs code which is a really really awesome ID for typescript but naturally you can use any IDE that that works for you which is one of our one of our main goals to basically bring infrastructure code to your comfort comfort zone if you're using Java as you run your develop your your you know your programming language and your team you'd be able to use Java to provision your infrastructure if you use typescript you can use typescript and our goal is to actually get to a point where we have as many programming languages supported so that you can stay in the same environment and use the same tools that you're using daily for your infrastructure okay let's see what we have here so it's a it's a it's a regular typescript project I've got pack adjacent file with some dependencies ts config for configuration and this is the entry point that the CD can get created for us and it comes with this boilerplate example I'm just gonna delete that for now because we want to create our own little one and is that big enough yeah and so this is this is an empty CD K application you can see the application here you can see that it contains a single stack as jason said you can create applications with multiple stacks and reference resources across them and they're really crazy architectures with this model and we've got the hello CD k stack which extends CD k dot stack and it's an empty stack to prove it to you I'm gonna use this command called CD case synth for synthesis synthesized and what siddik a synth is doing is takes your app and synthesizes the CloudFormation template form from your application and so naturally because this stack is empty I'm gonna get an empty template from it okay so first thing we want to do is we want to define a queue right let me switch over to to our browser and this is our documentation website you've got you've got lots of information here we're actually trying to figure out the right organization that would make it really easy to find ourself but there's a lot of information about concepts the construct library patterns how to use CloudFormation the low-level CloudFormation components and there's obviously reference for all the constructs in the construct library and so if I want to use the find es qsq for example there is the AWS sqs construct library so every eight of your service would have a contract library in this in this library and it's it's still not complete you'll find many gaps if you if you hit this now and we're working on making it complete and make it as awesome as we can but it's it's pretty extensive and Jason's gonna talk a little bit about things he can find there already there's really awesome programming models there already ok so we want to install this library it's it's just a regular library so it would be a regular library maven or a nougat or in pi PI or an NPM for our example and so I'm just gonna go back to our console and I'm just gonna install that library so npm install AWS cdk AWS excuse right and now that I have my library installed I can import the sqs module into my code and let's explore that just a little bit so I want to create a queue that I'm doing a new dot and you can see that this yes the library has a queue construct a queue class basically and when trying to create it I can see that this class accepts three parameters parent name we sometimes call that ID and properties this pattern is going to be repeated everywhere in the cdk because that's the pattern of creating constructs you can even see that my stack has these three parameters because a stack is also a construct these three parameters allow the cdk to basically bind your construct to what we call the construct tree you build this tree top down so basically start with the application then you go to the stack and then you basically end up with like the leaves which are the resources and and so the first parameter is a parent and almost every time you would just want to say this because you you're creating this contract in the context of this Const to the construct the parent contract in our cases the class is this stack second parameter is the name and that names maps to the cloud formation logical idea of the resources created underneath that tree and it's a local name it's not a global name like logical identities but it has the same purpose the purpose is to be able to identify this resource across updates of your stack and so I'm just going to give this a name it's not the cue name it's not the physical name of the resource that's actually allocated by CloudFormation when you're deploying the stack so let's call that hello cue and we've got an option an optional property bag here that we can pass we're not gonna use any any of those but I just wanted to show you you can see all the inline documentation and information about how to use these these properties and they're all strongly typed and all those good things that you get from codes from object oriented class libraries ok so I'm just I'm just gonna create a default queue and and to be honest one of the things we're trying to do is to make sure that you're not gonna need to specify anything that doesn't make sense that you would need to specify like smart defaults we call it and we can we go a long way to make sure that you know you get the most concise and efficient and safe development experience when you use the construct library ok so I've created the queue let's call CD case synth again I'm gonna go back to my terminal and see what happens neat so this is a CloudFormation template if fully legal CloudFormation template as a single resource of type sqs queue you can see the CD Celso embeds this metadata entry that allows you to map the path of the resource in the path of the construct in your constructor e to the actual CloudFormation resource so we can you know the back-trace really easy to find it project there are any problems and and yeah I've got my little cue let's just deploy this first and then we'll add the lambda function so I'm going to use cdk deploy to deploy this and it's going to synthesize my templates behind the scenes and basically kick off a CloudFormation deployment for this so you've seen this many times I'm sure and in the meantime let's bring in the lambda library so I'm gonna oh it's already done so before we bring in the lambda library let let me quickly do oh sorry let me quickly check that I actually have a queue here I'm just gonna use this CLI cool so we've got this cube provision for us now bringing in the lambda library and I'm also gonna bring another library I'll call lambda event sources which gives you some nice api's to bind lambda to different event sources I don't know if you're familiar with the CloudFormation experience around that but every event source in lambda it's kind of like Sam right like where all the event sources that have the same semi same API okay lambda sorry I'm like lambda event sources and okay same drill bring in the lambda module and I'm gonna bring in the event sources module two they're gonna use it real real soon cool and now same thing we did with the queue right like we're gonna create a lambda function cool three three arguments for the constructor this and Landa actually has a few required properties like your code right and so the ID even tells us it needs to know some stuff right you can see the ones that have question marks are optional the others are required so we've got code here as required we've got handler we've got runtime and I think that's it like that's that's the minimum requirement for lambda functions and so let's start with the runtime will write will write a will will write our code in JavaScript so we're just gonna use node and now for our code so if you're familiar with cloud formation and Confirmation doesn't deal with the runtime code it's basically just your infrastructure but with the cdk we believe that we wanted to create an experience where you can create constructs that are self-contained right the runtime code and infrastructure code can be bundled together in two libraries that people can reuse and so we have this concept in the cdk called assets which allows you to reference local resources within your project and bundle them into your construct and so first of all let's create our little directory for lambda code and then i'll show you how we bundle we bind them together okay so it just created this lambda directory and index file oh that's a directory no sorry index file and we'll ride out of this stupid lambda function exports handler async event so that's a that's a that's the lambda functions handler signature and we're just gonna log our event to the console nothing nothing fancy here it is like ten times longer than any hello world you've ever seen right but still it's like you'll see the the template soon okay so we've got the we've got the lambda code and now all we need to do is tell the cdk hey use this directory called lambda as your asset as your runtime code and it's it's actually a pretty pretty low level it's more low-level than we want it to be like we actually want to provide an even higher level experience for you to be able to reference your your code much more naturally for every programming language but at the lowest level you can just tell it take this directory zip it up put it in s3 reference it from my CloudFormation template do all the hula hoops that you need to do in order to make this code work and so I'm just gonna put lambda here right that's all I need to do last thing I think is the handler handler okay and learns the name of the handler right like it's index dot handler okay almost done now we need to connect the lambda and Q right like we need to actually get the lambda to consume from the Q so in order to do that we need to be able to act talk to the Q and talk to the lambda function and so I'm just gonna assign those two local variables and if I'm exploring the API I have for my lambda function it's a pretty rich API for example it lets you create metric objects and assign them into dashboards or create alarms from them like there are a lot of really cool things and we're thinking we have a lot of really cool ideas also like not oh not everything is as hearing still but it's there's a lot you can do with this already aside from all these we've got this add event source method which allows you to add an event source to your lambda function presumably and the the signature says I need an eye event source and then there's some some documentation that says you can use the event sources library blah blah blah and so we're just gonna do that new event sources dot and you can see there are only three event sources here but there are like 12 different event sources for lamda and we're just working on those and would love your help and pull requests and Plus Ones and all those okay and then I need to just press in the queue and that's it basically this is my this is how I connect a lambda function to an sqs queue and let's uh let's synthesize this for a second Oh before we synthesize this actually wanted to show you another command called diff diff is pretty cool because it allows you to take your current application and compare it to your deployed stack and see what will happen if I'm actually gonna deploy this so let's do that for a second and before I show you the diff I'm just gonna run deploy because it'll take a second okay so we see the different formation we see that there gonna be a bunch of resources added to my stack we see the lambda function here we see the event source mapping that maps the lambda function to the queue we see the lander role in policy and we also see a bunch of CloudFormation parameters which are part of the the asset magic that basically binds those assets to our to our code and if i take a quick look at the actual templates that's generated let's do a JSON version prefer Jason sometimes you see this is technically what I would manually need to write if I wanted to create this hello world example with what we call raw CloudFormation and so there's a lot here that it's not trivial and one of the interesting things is that you can see that it's created an anion policy for me for my lambda function that allows this lambda function to consume from this Q right like that's required like there's no way for this lambda function to be connected to the Q without without this policy and for those of you who have dealt with anything at AWS that's probably one of the biggest pain points at least was mine okay I think we're done let's see yeah let's go to our lambda console and see what we have lambda console you've changed the icon I think this past month okay so we've got our lambda function and we can see that it's connected to our Q we can see the permissions here also with all these nice set of actions to consume the for it to consume from the Q and if we open the sqs console let's try to actually send a message I think the official one is like that No okay send now the comma is gonna blow up the demo okay i'm back to the lambda function i'll open the cloud wash logs because that's where i expect this to be and we've got a law group still not still working yay hello CD k cool so quick recap we saw how to initialize CD k projects using CD k in it as i said those are just regular projects there are no magic to them and we can add more templates if you feel that we need more templates we saw how to use the CD k toolkit in order to interact with your application and help you with the cdk workflow you write your code you use this toolkit to synthesize diff and deploy our application to tear stack and I'm gonna hand it over to Jason for another really cool demo about writing constructs so we've seen the basics of using the cdk now we've seen the cdk CLI we also consumed some constructs from the construct library so in the second part of the demo we're gonna take things a little bit further and we're really going to look at not only consuming constructs but how we create constructs and use our own custom constructs to shape our code so we are gonna create some custom constructs in this demo we are also going to learn a little bit about the permission system in the abs construct library and talk about how our runtime code in our app in our lambda function actually communicates with the infrastructure we create in our infrastructure code so you can see from the architecture diagram we're gonna refactor our code a little bit we're gonna pull out this cue recorder construct and then instead of just logging to cloud watch logs we're gonna pull in a new module from the construct library and we're going to start writing data to a dynamo DB table so let's go back to vs code and the first thing we're gonna do is pull out this this new cue recorder construct class so this is just like any other code of course we're gonna create a new Lib directory and a cue recorder type script file and then the first thing we need to do is import the cdk framework of course so that we can access construct and all the classes we need and then we're going to export a new class called cue recorder that extends cdk construct and let's go ahead and fill in our initializer and we're gonna follow the same pattern that a lot showed you earlier and we're passing in our parent that's of type cdk construct we're also gonna pass in our ID or our logical name that's going to be of type string and we'll just pass those up to the superclass so this is the simplest construct we can write doesn't actually do anything yet so let's go and pull some of our code over from our main cdk application so let's grab this lambda function and also this code that configures the event source and instead of having all the details on our main cdk application we're just going to shift some of that into the queue recorder class and we can simplify our main application a little bit there so let's go grab these imports you don't need those anymore okay so our codes looking pretty good now we just have one problem down here we have to fix so before we could just directly reference this queue now we're gonna have to pass in the queue to our construct so we're gonna follow the same pattern that allod showed you earlier where we have these property objects that we pass in as the third argument so let's create a new interface and we're gonna call it Q recorder props and the only thing we need to put in here is our input Q of type sqs Q so now let's update our constructor to take in a new props member of type Q recorder props and then we're just going to reference that down here and pull out the input Q cool so that's all it took just like any other code working with our infrastructure definition is now just as easy as we're factoring any other code so let's go back to the main cdk application and actually use this construct so we're gonna create a new Q recorder and we're gonna pass in this as the parent in the construct tree where we want to attach our new construct we're gonna call this our Q recorder and then we have to pass in the input cue that we just saw and of course that is just a cue so go ahead and just update this real quick after our refactoring and while that's running let's go ahead and add the dynamodb table to our system and then we'll change our lambda function to start writing our data to that table so going back into our cue recorder construct so we're gonna put the DynamoDB table in here so we can keep all that complexity kind of wrapped up behind this cue recorder interface and just like we saw in the first demo the first thing we need to do is bring in this new module the DynamoDB module from the ada basic construct library so we're gonna npm install AWS CD k this DynamoDB and after we've added that to our project we can go up here and import it into our code oops so just like we saw for all the other modules in the construct library and then it should not surprise you that we're gonna create our table in the exact same way we created our other constructs so a new dynamodb table we're gonna pass in this as the parent in the construct tree where we want to attach this new construct and we're gonna call this our cue recorder table and we got to pass in one property for a dynamo table let's assign this to a constant two so when we're working with dynamo the one attribute or the one parameter we need to pass in is our partition key and specify which attribute is our partition key so we're going to create an attribute with name ID and type is just going to be a string and we're gonna use an enum in the dynamodb namespace to grab that string attribute type so that's all we have to do to add this table to our system the next thing that I want to do is jump over to our lambda and we're gonna replace this really simple console logger with some code to write to DynamoDB so we've already got this we're just gonna pull it in from the clipboard and then we'll explain what the code does but we've still got a really simple lambda function and we've got 19 lines of code here we're pulling in the dynamodb library from the AWS SDK and we're using the dynamo DB client to write all of the records in the events that come into our lambda function we're just going to write all those records to a dynamo table so the the interesting part of this code here is that we have to have our runtime code able to talk to the infrastructure that our cdk application code is creating and what we do we use a technique where we pass in the table name as an environment variable so a provisioning time we are going to get CloudFormation to wire all this stuff together for us so that our lambda function has this table name environment variable set and is able to just work with that table name without having to do anything else fancy to figure out what was the actual table name the CloudFormation provision for us so that's how the lambda function knows which table to write to but let's go back to our cue recorder cdk code and talk about how we actually supply that information to the lambda function so let's go and explore the function API a little bit right here on top you can see we've got this method called add environment and this is gonna let us specify any environment variables that we want we want to pass to our lambda function code so we are gonna pass this table name environment variable and then we're gonna set this by grabbing the table construct and just asking for its table name and even though we don't know what that name is now when CloudFormation actually provisions all this for us it's gonna wire everything up correctly so that this table name environment is set with the actual table name the physical name and the table that's created so we're pretty close now we've got our code refactored out into a construct we've got this new DynamoDB table and we've got our lambda code updated to write to this table so we're very close but we're still missing one piece we haven't actually granted permissions for our lambda function to be able to write data to that table of course permissions are always locked down by default so we have to explicitly grant any permissions that we want to add to our system so one of the things that I think is really really exciting about the database construct library are all these grant methods oops so this is how you explicitly give permission for different principles to work with different resources it's going to be very clear syntax it's going to be very explicit and we're gonna get a nice tight lease privilege policy that only grants permission for specific operations for one specific resource so security best practices very easy and very clear in the code we're gonna use the grant write data because our land of function just needs to pump data into this function and then we need to pass in the principle that we're granting this permission to so we can do that by grabbing the execution rule in our function so we just grab our function context and ask for the role and that's all we have to do to wire up this permission the code is really simple it's very explicit it's very clear what's happening and we really didn't have to dig deep into some of the assembly language of the CloudFormation template to do this so let's go ahead and take a look at C D K diff now that we have updated all of our code and see what's actually gonna happen when we deploy this and to save a couple of seconds I'm gonna go ahead and kick off the deploy as well so we can see this new table that we created we expected that we can see that our policy our policy document has changed a little bit a lot explained how we had permissions to work with sqs before and now you see that we have some new permissions added here so we now also have permission for the the role that's executing our lambda function also has permission to work with just a few actions in the dynamodb API just for that one table that we granted permissions on so very easy to lock down our permissions nice and tight and we can also see this lambda function down here that we had this new environment variable that we specified let's give this another minute for this deploy to finish and let's go back to our slides and do a quick recap so in this second part of the demo we saw not only how to consume constructs but how we create our own constructs and use them to simplify and shape our code we talked about the permission api's the grant methods and Atos construct library and we talked a little bit about a technique for enabling your runtime your application code to be aware and work with the infrastructure that CloudFormation provisions for you so let's see how our deployments going we finished so let's jump over to the SQL console again and let's go ahead and try sending a message and with any luck the message will be picked up by our lambda function and written to our DynamoDB table let's see let's go ahead and send two messages we'll see if both of these make it over to dynamo so over in the dynamo console we've got our table that we just created and if we look at the items sure enough we can see hello cdk1 hello cdk2 so the demo is working and we have our messages flowing from sqs to lambda into our dynamo DB table so I will turn it over to a lot to take us through the last final part of the demo Thanks so we saw how to organise your application using constructs and you can do a lot with that like if you think about it complex cloud applications consist of multiple components and infrastructure is becoming part of the application it's not an external part and so being able to model those mental models using object-oriented techniques is a very powerful capability that we see customers do pretty amazing things with the other nice thing about constructs is that you can use them to share infrastructure components there are just regular classes like any other class that you share and so you could publish them to package managers whether they're internal to your company or in your team or or to the public package managers so I spent some time a couple weeks ago over the weekend writing this cute little construct library called Twitter cue tweet cue I published it to NPM it's it's pretty cool it would basically give you an interface of the queue that contains tweets from a Twitter search so you just specify your query and this Twitter queue thing it has a few moving parts in it it has CloudWatch events that triggers the lambda function a lambda function that pulls Twitter checkpoints through a dynamodb to make sure that it's not repeating any tweets and then publish them to a queue and so you can consume the queue naturally as a queue but there's there's a bunch of things happening here behind the scenes and so let's just plug that queue into our little hello world application it's kind of it's gonna it's gonna plug in pretty cleanly and and there's some documentation here you can install this module it's an NPM module let's do that okay and p.m. installed the star of the demo CD k tweet queue and I'm gonna go to my main file import tweets Q from CD k to HQ sorry equals cdk tweet q oh and let's see the API here new tweet Q tweet Q and it's a construct surprisingly so I'm just gonna bind it to my stack tweets and if I'm looking at the set of properties here again a bunch of optionals a bunch of required ones the two required ones are the secret arne the secrets manager arne a RN that contains the twitter credentials and so we we've actually uploaded before so and I'm not gonna need to do that I'm not gonna eat you uploaded I'm just gonna paste in a little R in here so that contains my Twitter credentials so I can actually hit the Twitter API and the second required parameter is the query so I'm gonna use hashtag AWS cdk and you're gonna need to help me fill up that table with by hashtagging your tweets and I'm just gonna basically replace this cue with the tweet cue because a tweet cue if I'm if I'm looking at the tweet cue definition you see it extends sqs cue so it's basically an sqs cue as far as everything polymorphically is concerned so I'm just gonna I'm just gonna do this I don't need this guy anymore and let me kick oh sorry let me kick off the ployment here and I just I just want to quickly show you what's going on here sorry cdk dip is probably more so this is basically what the this little one line of code or three lines on four lines of code added to my stack like it's added a cue a table role of policy a function that permission cloud watch events rule and event source mapping and it modified our policy by the way this policy view diff view is hopefully going to change we have a poll question four you know nice really UI for showing you exactly what permissions are going to be changed in your stack and so the let's go back to the slides for a second the point here is encapsulation right like it's it's it's a very fundamental principle in software design and we were really excited the CD cake is enabling that around infrastructure all I needed to do is basically bring in this tweet queue class and behind the scenes this class had like lots of implementation details and quite a lot of resources it could be hundreds of resources behind this this one little class it provided an API that's worked against a specific mental model that they designed right like it's not about the resources is about what this thing is actually doing and it plugged really nicely with my existing application because everybody is speaking in the same these construct language right so while this is deploying I think it's gonna take a little while because of all those resources Jason's gonna do a little summary and tell you what more you can explore so I'm gonna recommend some areas to go explore more for the CDK while I'm doing that please help us out and hash tag abs CD case and some tweets my first recommendation is definitely to go explore the AWS construct library remember that this is the curated area where we define the experience for AWS infrastructure and there's a ton of really awesome functionality there already if you want to go build application with containers check out our ECS Fargate an ECR support if you want to build a service application we've got great support for Amazon API gateway a dubious lambda and a davis step functions as well we've also got a nice support for code pipeline code build code deploy so you can use a cdk to manage your whole build and release infrastructure as well - there's a ton more there's great support for ec2 if you want to work at a lower level manage V pcs very very easily security groups cloud watch metrics alarms and dashboards so tons to go explorer if there's something missing in the library raise an issue on github or better yet send us a pull request we'd love to hear about it so we can help prioritize based on customer feedback and we we've really focused on typescript today my second recommendation is to go explore some of the other language support for seedy cave so we've kind of hinted a little bit at this idea where the cdk is really available not only in typescript and javascript but also in Java and c-sharp we've got some experimental support for Python and the repo already that you can check out we hope to include that and the official builds very soon as well but I think one of the really neat things about cdk is that we're able to take this programming model and this ad based construct library and we're able to map it consistently into all these different languages and keep it in sync across the different languages and we're able to do that by using another open source project we created called J SII so that's another really interesting area to explore and then so of course we're really just scratching the surface in an hour of what you can do with the cdk they're already support for a lot of pretty advanced use cases so if you for example have a big application that runs in multiple regions that's easily supported with a CD k if you have a big investment in cloud formation templates already you can take those templates and embed them into your CDA application without having to port them all over the code if you just want to start migrating over to the CD K and there's a ton more check out the docs and ask us questions on github and then finally as some kind of higher level resources if you're just looking for some easy ways to get your hands on some CD K code and learn there's a great workshop online at CD K workshop com this is a nice guided workshop it's gonna help you build a pretty cool application using Amazon API gateway and AWS lambda to build a neat little serverless application you can also find our Doc's linked from github and we'll have the source from this sample application up on github as well later today and of course if you want to reach out and engage the developer team we'd love to hear from you on github or git er or Twitter and there are a couple of related sessions as well that'll have a little bit more cdk content in them so dev for one one infrastructure as code AWS best practices is available three more times this week this is a chalk talk so it's going to be a little bit informal and lots of chances to ask questions there'll be a couple of Engineers from a TBS developer tools there and showing off some cdk code as well dev three to seven beyond the basics advanced infrastructure as code programming on AWS there's a chance to catch this session on Thursday this is going to be an overview of some of the advanced features that we've added to a tube-based cloud formation in the past year so that'll be a cool one and last but not least dev 3:09 CI CD for server lists and containerized applications this is probably the first session I'm going to check out as soon as they're uploaded to YouTube phenomenal speaker and it's gonna have a lot of cdk content in there and really something for everything or something for everybody with CI CDE serverless and containers I mean catch that one later today or on Thursday let's check the tweets yeah okay this is the moment of truth we'll see you gonna be an untied clam table yeah flawless demo before anybody a CD Kay yeah that's a good one thank you all for tweeting thanks for coming we're gonna hang out here we've got some cdk stickers the first ones ever we'd love to talk to anyone on the getter channel who talked to us and is maybe here and we would love to get to know you and thanks so much thank you [Applause]

Channel: Amazon Web Services

Views: 24,978

Keywords: re:Invent 2018, Amazon, AWS re:Invent, Devops, DEV372, AWS CloudFormation

Id: Lh-kVC2r2AU

Length: 54min 47sec (3287 seconds)

Published: Wed Nov 28 2018

Reddit Comments

[u/[deleted]]

I thought this was the point with cloud services, the ability to "loan" some computing power from someone.

How is this different from that?

Genuine question