From the Field NZ | Pipeline as a Code with AWS CDK

[Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] hello everyone my name is eduardo robelo i'm a solutions architect based in auckland new zealand and this is the frontier field is a new zealand technical show where we bring our solutions architects that work with the customers across the country to share their experience online for us and here today i have two special guests coming back to the show with a really hot topic in the market right now welcome to the show rattan vish hi dwanjo hi rodo how are you pretty good thank you i hope you two are great too we are on lock down level four in auckland so everyone is isolated in their homes so that's why i have them in the show live today so then we can you know talk about everything that has been happening in new zealand so rattan vish this is a really hot topic in the market especially with our customers doing infrastructure as a code here in new zealand what we will see here today with pipeline as a code using cdk all right i'll take that uh question uh eduardo uh so um as you say it's a it's a it's a pretty important topic for a lot of our customers infrastructure as a code is no longer an optional thing we see [Music] and we're just looking at the ways in which we can make that make our developers more productive this is something that has always been our top focus at aws um and cdk's is our attempt to to do that um so in this we will be covering what is cdk uh how do you how you use it um and um some some of the foundational aspects i'll do a basic demo showcasing that you know how to use cdk and then we'll go real deep where wish actually covers how do you actually build an end-to-end code pipeline using cdk live code today that's pretty good yeah yeah so so just uh for for the people out there um we'll go from i would say right from level 200 all the way to level 400 uh in today's session and we're really excited that uh hopefully this will resonate with you and you will pick a thing or two from from today and uh start using it in your day-to-day uh uh projects absolutely we always we are always learning something right so every day we pick a trick and tip here and there on a specific article or a blog post so vish from the demo that you built there is anything specific that we should be paying attention that we will see during this session yes i mean as i'll be building a pipeline in this demo and you know pipeline has some waiting times as and when it goes into deployment stages so i have pre-recorded my demo but what i'm going to do is i'll walk you through each and every steps and i try my lip syncing skills here um so that's one thing second thing is i'll be i'll be building a lambda code and api gateway but i'll be fairly i know walk you through a bit faster mode and when we reach the actual pipeline itself that is where we slow down and walk you through line by line how we can build that pipeline right so i'll slow down there and basically uh put more focus on that part rather than focusing on the application itself fantastic and i know from our our previous conversation before we started this show we had we have a really special guest here today who who you are bringing to the show yes we are really really grateful that uh the creator of cdk uh himself is joining the show today uh so why don't we bring a lot into the into the conversation fantastic let me see here's already and there we go eli welcome to the show hello everybody hi lad great to have you sure yeah thank you so much uh i uh just for everyone i i only reached out to you live literally three days ago um i think him and uh i i wasn't sure that as such short notice he will be he will be able to join so ella i'm super grateful that you have uh you know um uh uh you know joined us today and uh to share share your your knowledge and and the whole approach and that how you build cdk and um yeah so we're really really excited to have you thank you for joining us cool thank you for having me so we start the conversation with a fire shot right right then so which questions do we have that uh elad can help us understand a little bit more the background and the future of cdk yeah i wish there was a real fireside chat but here we are it's like twitch far side uh so so a lot i'll just start with real basic thing for for everyone out there who is watching the show today uh what what is aws cdk really um so the cdk is uh as a software development framework it's it's a library basically for software developers to be able to to define cloud infrastructure using normal familiar programming languages the idea behind it was that this this this industry of defining infrastructure of infrastructure as code has kind of evolved mostly from like the operational world and you know like people installing software on machines right and data centers and like that was basically the evolution of it and at some point it got to the it got to the kind of like this inflection point of complexity and so if you're using you know tools like cloud formation or terraform or you know other tools for defining infrastructure using um at the desired state configuration uh you probably already understand why we built the cdk because as you as you try to like model complex systems using these configuration files you basically run into needs that are not easily addressed using these these formats right um even simple things like an if statement or or a simple loop at the you know if you want to express some some business logic that represents your infrastructure on one hand and on the other hand the ability to model your system be able to like describe how your system looks like is is usually pretty hard with those uh with those tools and so we looked at this we looked at these as these existing tools and we said hey we actually have solutions to all these problems we've been writing software in the past 50 years that's addressing the exact same problems right like creating models expressing business logic and so we said instead of like trying to reinvent the wheel and and force this you know this paradigm to like become software we're just going to be able we're just going to move over to the world of software and enable people to use the tools and the or the or all their skill sets and tools and patterns that are used from software development to uh express how the infrastructure of their applications work uh looks like and so this is basically kind of like where we came where we came from in terms of the cdk and i think the other interesting thing that happens and that's a very natural kind of evolution of that of that approach is that as you move over to as you have all these like extremely powerful tools for creating abstractions and defining business logic then you start creating then you you basically start going up the stack in terms of like abstraction level and that's where the cdk becomes super interesting in my mind nice nice yeah i really think that that's a game changer aspect of cdk um so did you have this problem of your own you know were you part of a project team where you were doing your own cloud formation or something and you felt the need to to build this yeah yeah actually that was the kind of like the story behind um the the birth of the cdk was uh story time tell us the story it's actually a very classic amazon story and i think like it's a really beautiful example in my you know my mind is like the innovation culture at amazon um uh we've been working in uh a in a product on a product that uh you know kind of like the serverless system event based kinesis all the good stuff um large-scale amazon style and deployed to multiple regions multiple environments and and you know as we were like designing these at the system and created the cloud formation templates for them uh we realize it's like we don't have the right tools to do that like copying and pasting cloud formation templates is not you know what we learn as software engineers this should i still have scarves over here one of those things that smells bad and you're like okay this shouldn't be what i'm doing um and so we started like basically hacking on some some internal tool or generating cloud formation and there were like also open source projects um like um um um like oh sorry i can't i can't remember the one jaws serverless framework yeah i mean there were some internal so some external uh really nice open source projects for like um generating cloud formation from code and and and so but but i think like the interesting part again was this idea of being able to create abstractions because one of the things that we really that we really needed is to split the work in the team to multiple you know multiple people and every person was responsible for a different part of the system and we wanted that that part of the system to be isolated and tested in you know as as its own and then integrated together into to form the bigger system and then deployed into multiple environments and so like this type of activity which is very very common in software development right like when you develop software this is how you work um we needed the ability to do that with our infrastructure and so we literally created this internal tool and it was really a game changer for our team like it worked very well it was easy for like junior engineers to understand it it was like it had a lot of good smell in that sense and then after we launched the system it was like ahead of schedule it was like very a nice showcase of like this i of these ideas um we went over and wrote this uh this thing at amazon called press release i don't know if you're of course you're familiar with it but i don't know if everybody's familiar with it which is this beautiful um uh format for describing the future so you're basically writing this one doc one page document that basically tells the story of your launch in two years three years right like you're basically kind of dreaming and it's a really fun thing to do and usually like writing is like amazon you know reinvent 2020 whatever 2018 i think it was launches this product we didn't have that name yet and so like describing how this would work for customers and it was really a good kind of we were at the right time a right place there was some interest in like solving that problem but no existing solution and so like when when we brought this like internal tool to the leadership and we're like look at all these examples right like and i i we were literally just like walking through the code base in the prfaq meeting um go to some real world usage and it was easy well it was really easy like everybody was like yeah this is this looks like it's really solving a real problem for for you it's it's clear how it can solve problems for customers and then we started the journey and it took probably two years from that point to like actually release the cdk in ga which was i think a year and a half more than i wanted to release this in six months and then like it took two years so yeah that's the that's the story wow that's that's such an inspirational story really um you know um having having a problem and then solving it at a scale uh for our customers is truly aws way an amazon way so so really inspiring yeah and not only this the the story about city kate but how the process also helped have the idea spread across the company right and have the opportunity to innovate be on behalf of your our internal team and also to our customers which is beautiful yeah i think i think the other uh interesting property that we have internally and i know that many companies don't work like that and which is completely you know actually something that we're tackling today nowadays but amaz at amazon there's no separation between ops and dev like the team that develops a software also operates it like there's no even like role separations like the software developer is responsible to maintain to to provision and maintain and operate their system and so i think that was another reason why we you know there was this connection right between the world of ops and the world of application development because we were like the same people right yeah so those small two pieces of teams focusing on a single feature working as a team you're a little bit robotic but yeah we could get it so elad we got a question uh before we jump on the the presentation from from our viewers so how enthusiastic was the leadership from the get-go was it a huge sell from from the beginning or just after the pr faq i'm not sure i understand the what does it mean a huge sell maybe i'm not but i i think leadership like generally leadership with amazon is driven by customer feedback mostly and so initially they were like yeah this looks nice it seems like it solved your problem i can see how it can solve other people's problem but let's let's let's see how it works and so like we got some small funding to get started and we started with ball rolling and like the amazon has a very very strong kind of bias for action cultures like release it put it in the market and especially in open source uh and you know cdk is an open source project it's also it's also kind of aligned with how open source works right like you put something that's very early that's very kind of even experimental initially and then you slowly mature it and bake it with customer feedback and customer inputs and in that sense it was it was a it was a nice kind of organic growth uh and i think the other thing that happened at some point actually didn't happen from the first from from the day from day one was there was um some point in time where we collaborated with an internal team with a with an internal uh kind of platform team to build a cdk construct for the amazon deployment system the internal you know the deployment system that's used across all of amazon and that was kind of like the killer app for internal adoption and so once that was released we started seeing like thousands of amazon teams adopt the cdk because they really needed that capability and they see they offered a really good experience and it was like a really good alignment there and so that was another way for us to like see this feedback loop right and as the feedback grows and as you know we see more customers using it and um then leadership is more enthusiastic so it's very it's very much like trying to align to working together right [Laughter] cool i think uh one quick question so some of my customers already use cloudformation but they are exploring the ways to go into cdk and they see the value in it so what is your advice and you know making those converting those existing cloud formations into cdk so that they have a single cdk repository where they can manage um so first of all there's some good content out there that talks about migration from cloud formation i think one of our team members wrote a blog post about this uh i think a year ago or something as we released the cloud formation include which is uh but generally speaking i would say first thing you could do and it's almost like a super low hanging fruit is use this construct called cloud formation include to include cloud formation templates into your cdk applications you basically kind of import the template as is into your cpk application it doesn't mean that your template now is written in code but it means it's part of your cdk application so it's kind of like becomes this first class citizen and you can even you can you can even edit it in your code so you can like import it into your code and then you can say add this property to this resource get this attribute from this resource you can like start to interact with it in a sense and so i would say the first thing you could do is do use these cloudformation includes and then additional resources and additional infrastructure use proper cdk in a sense so you kind of get this hybrid model and then what i would do is i basically migrate in parts i would basically say okay let's take this you know these bunch of resources maybe maybe they're not like stateful resources or persistent data or stuff like that and i can like start to migrate them from cloud formation to become cdk code and the reason i'm i'm saying i would do this manually is because cdk has this concept of levels of abstraction and and we call them l1 l2 and then l n or l3 and sky's the limit but at the lower levels l1 corresponds one to one with the cloudformation spec the confirmation resources and so in the sense you're not getting an abstraction you're basically just getting support for writing cloud formation so in a sense it's kind of like a cloud formation generator at that level but the complexity of using you know aws and you and integrating with between resources and you know iam and and all of that stuff is still there you still need to understand all the details of how those resources work how every property works and so in that sense you're still at the very kind of like very low level or the lowest level that you can get um and then what we have in the cdk which is i think one of the biggest you know where we where we inject most of our effort and most of the value is what we call the l2 constructs and those l2 constructs also correspond to aws resources so you have an l2 construct for bucket and an sl2 construct for sns topics and l2 construct for lambda functions but the apis are higher level apis are intent based we call them so you basically express your intent using object-oriented apis properties you know declarative properties so you're saying for example i want to connect this sqsq to my lambda function you say lambda dot add event source q and and as you write this one line of code this the l2 constructs will interact with each other um define the i am policy for the lambda function and the resource policy for the queue and the event source and all the all that magic all that you know all that all these like undifferentiated details are going to just going to be implemented for you and so when when i'm you know proposing to people to like write cdk i'm saying you you this is where you're going to get the most value and so if you're migrating from existing cloud formation the only real way for you to actually go up the layers is writer code again in the cd so that kind of migration is um and the same defaults from the constructor o2 really helps too when you are migrating from cloudfront or starting from scratch there is send defaults following some really good principles less privileged all bake it in for you on the l2 construct and that really helps to speed up the agility of the software development and the infrastructure development which is great for everyone so yeah the community can contribute some ideas came from aws and how we work with our customers but it's open to everyone to you know put their ideas over there yeah well sometimes we think we we like to think about the l2 layer as like an implementation of the aws documentation in code ah interesting basically all the adwords documentation of how to use a service is now implemented for you in code express really nice with nice apis on top of it um and so when we built these l2 sometimes we're frustrated we're like oh my god just you know doing this simple task requires so much details and so many like you know loosely coupled screens that have to match with each other and like but then we always tell ourselves okay this is the last time someone needs to read the stock nice sorry written just to be mindful of the time so we do have a last question here from the viewers after a few years of cdk running if you could change one thing in cdk today what that would be nice question i think one of the interesting things that we um we're still struggling with in the cdk is streamlining the connection between the runtime code and the infrastructure code right like because one of the value propositions of the cdk is you're going to be able to use the same programming language when you write your runtime code in your infrastructure and that's huge i think many customers love that about the cd can one of the reasons we're shipping the cdk to multiple programming languages like java and python.net some of those languages are like very far from like the ops and devops world and the reason is that we wanted uh application developers to to feel comfortable right and that was one of the reasons you know we have those languages supported um and the thing that i think you know i'm i wish we would do better uh and we're definitely in that trying to do better is to improve the connection between the two phases of execution if you think about it right like there's basically the phase of the execution that defines your infrastructure and the phase of execution that's actually running as the your your system is like and these connections are still pretty weak um they require quite a lot of like a boilerplate and and tweaking and some build tricks and some some languages that are easier some languages are almost impossible and so i feel like that's the thing that we i feel like we should have invested in more from from the get-go but it's actually not a simple problem like we're like we don't really have a clear still a clear way to do that so yeah but but improvements are coming for sure and i think city is paving the way for that to happen and have a unified uh at least framework for infrastructure as a code in the future i love it so i'm a big fan yeah all right hey thank you i wish that we could continue to have this uh far side virtual far side chat i love it um and all the great insights that we're getting from you uh hopefully we can probably do another show where we can have you for a longer time i know you've got a busy day ahead of you um so once again thank you so much for uh taking time and and joining us um yeah all the way from israel really really appreciate it all the way i came all the way to new york more than welcome yeah thank you would i would have loved to come to new zealand actually it's uh it's such a beautiful place so thanks uh for having me and uh i'll stay tuned i'll catch up on the show thank you thank you thank you thank you cheers all right guys that was amazing wasn't it i mean having a chance to actually ask questions uh you know from someone who created it and and we could get into the story of how did he go about creating it uh so edwardo i hope my screen is visible now it is we have these lights over here a cloud development kit and some logos python javascript yep yeah so i believe that a lot has already covered quite a lot of edge which is fantastic so i just want to quickly go over a few slides before we get into a live demo of actually building a simple cdk app you know very very basic before as i said earlier before vish goes into the details and advanced topics of creating a pipeline as a code using cdk so yeah as uh lad mentioned cdk allows you to define your cloud infrastructure using one of your you know favorite programming languages python javascript typescript whichever you like you use that and and support for golang is coming pretty soon um so under the hood if you look into cdk every app that you build in cdk is made up of multiple stacks you know one or more each stack is really you can link it directly to say a cloud formation stack and every stack has got one or more aws resources um and the second thing that cdk provides you is a cli which you use to bootstrap a basic project you know and uh do your synthesizing of your uh code into a cloud formation template and finally do deployments and those things i will look into into that very soon and finally this is where the magic of cdk lies as elad was touching on that we provide you a set of construct libraries which for which has got either you can use uh you know have got one-to-one mapping if you are comfortable using cloud formation and just want to use those you can use what we call l1 constructs um or if you want to have uh you know where we have automated few of these things with some best practice defaults you can use l2 constructs and then we have got lt construct which is really where you have got what we call cloud uh patterns you know um which are combination of multiple aws services stretched together in a best practice way so uh this i i don't have too many slides and this is probably uh after this i've got a few more slides but let's get into the demo so i will switch to uh my terminal and what i'll want to show here is i haven't got anything in this directory at the moment i want to create a simple cdk project but before i start doing creating a simple cdk project let's have a quick look at how do you get started with a cdk you know so if you go look for aws cdk documentation you'll you'll find this page which nicely talks through that how do you get started with a cdk app uh some of the prerequisites that you need to have how to configure your uh aws cli and um and the main thing is is what command do you run to you know get uh cdk installed on your machine so i'll just use that i do have cl uh cdk already installed but i'll i'll still just uh show you that um how do we get um you know cdk on on the machine uh so i put my password and and yeah so so that's that's pretty much all all we need to have in terms of cdk and uh i'm skipping a step which which we'll go through uh in his uh in his demo later on uh so once i've got aws cdk installed i'm actually good to go to set up a project first project so let me do that by just say cdk init and then i can specify the language of my choice you know it could be python or or it could be typescript which is what i want to use today so while this is happening it may take about a minute or so you can see in the left side we've got the project uh structure coming uh a base setup um and while that is happening i just want to quickly touch on few the common commands that we will be using uh to uh as we interact with cdk so the first is cdk synth which um converts your code um into a cloud formation template cdk diff helps you see the difference between multiple deployments uh and then you use ctk deploy to basically deploy your code to to your aws account so once i've got my project set up and and before i look into the actual setup of my project i know that for my project i'll be needing few of the construct packages that uh so i'll quickly add them into my project so that i have them ready when i start to actually create them uh create my application so i need aws ec2 i need another one aws acs and there's one more that i need for aws ecs patterns and i can show you where to go to find a catalog of these um what we call constructs so yeah so while that is happening let me uh quickly go to where we find the documentation for these uh constructs so we have got an api documentation for aws cdk which has uh which provides you all the list of different contacts that are available and what they do there's another place where you can actually find this is is this one which we call construct hub so here you can just type you know uh construct name and then you can search through this pretty handy to to actually find the constructs and the documentation and some code snippets that you can then use in your code so let's say if i look for ecs patterns i can find one here which shows me that what uh what it does once it loads uh and yeah so you can use this or you can go to construct hub and find the details um either either will work so hopefully all the packages that i needed is is here so i'm good to go so let me show you what's the basic cdk code structure or project structure looks like so the bin folder is where you have got cdk demo dot type script this is the entry point for your project really and as i was showing earlier in my powerpoint take that the parent is a cdk app and then inside that we've got multiple stacks so this is the place where you can actually inject multiple stacks so for now i've got just one stack cdk demo stack which comes from this lift folder and it just provides me some documentation or a help text to suggesting that hey um if i which account do i want to deploy the stack in so i'll take all of that help text out um i can specify these accounts or if i don't specify anything it's just going to take the account details from my profile you know but for the sake of just specifying it i'll just be prescriptive and say that hey use user accounts which is a default account for the cdk so that's that's pretty much what i need to do so i just say it's an app cdk app it has got a single stack at the moment which is going to be deploying to one of my accounts uh that i have got set up and now let's look into the the place where we will be actually doing the coding and uh [Music] looks like one of the packages that i've tried to install hasn't uh yeah looks that's okay uh so this is this is where we will be coding most of our you know uh thing so actual application will will go here uh so i will start let's say this is where i need to i want to create a vpc first so i will add one of my package which is this one i know that vpc construct is available in that package so i'll say my vpc equal to uh i'll just instantiate an object of that so as elab was saying earlier you can either use a a low-level construct which is which starts with cfn so um cfn vpc uh which is just going to create a single vpc resource and nothing else right so so that's that's uh for creating a vpc every construct that we use in cdk takes three arguments pretty much so the first one as it shows me here that it is asking me what's the scope of this which is really what's the parent of this which stack this resource belongs to and in most cases this is going to be referring to the parent in which it's running so i'll use this um then it asks me the identify for it so it's just a logical identifier i'll say it's my dev vpc and then um and the the beautiful part of this is look at this if if i'm not uh providing one of the uh required parameter for for this it's asking me is highlighting that hey i've got a problem here and if i hover over it it says that an argument for props was not provided so so this is where you provide all the digital parameters that this construct requires to synthesize a cloud formation out of this code and uh i can do this uh control space to see that what are the other props that it actually needs so anything which is question mark these are optional props i don't need to specify them but it needs a cider block and if you have used if you created vpc using cloudformation you would know that cider block is a required field so i just need to provide that so it's just a parameter that it needs to take and yeah so so once i provided that um it should be happy with it um yeah i'm finding something unusual here which do you do you have an idea that what's going on here is my cd cannot set it properly awc core is is not there could be could be the vs code itself for attempt so sometimes when you install a new package you need to he open the file so then vs code can load the packages so that uh no it usually has been pretty good with that so um so yeah so something something interesting uh going there uh let me try putting that package um hopefully that should fix it um anyway so uh that's that's how you actually add um you know a vpc and uh if my cdk synth runs uh let's try that yeah seems to be okay with that uh perfect so uh once i do cdk synth which is converting this code into a formation template i go into cdk out folder and i can see that hey there's just single resource that it has created which is vpc but this is not a fully functional vpc as you if you've set our vpc you would know that we need multiple subnets you know you need uh internet gateway route table all those uh 20 different things that you have to really do to have a fully functional vpc ready for you so how do we do that and this is where the magic of level two constructs comes so let's try doing the same thing with the level two construct so i'll say my vpc and this time instead of using cfn vpc i'll just use a level two construct which is vpc and same goes here i need to provide the the scope uh then a logical identify for it and and and that's that's all this one needs you know so so the magic of this is that it has got all the quite a lot of defaults embedded into it so i don't have to provide any uh props and this time let's say if i do a cdk synth and look into the code that it has generated as you can see that it is now using a default slash 16 range um it has created me a subnet which is on slash 19 range uh it's a public subnet it has set up the route table accordingly and and so on so i won't go into the details of everything but as you can see that it has generated 626 lines of nicely formatted cloud formation template for me how cool is that and that speaks a lot to the development right yeah yeah with a single line of code and this this is where the magic comes you might be thinking all the network folks out there that hey but i don't need 10.0 16 range i want a very specific site range that is available or that has been allocated to for this vpc cool sure so how about we just provide some props here and one of the props that it takes is cider so i can say that okay cider if you want to change there you go you use slash 19 range and and that's it um then you may have a requirement that you know uh i also want to have a very specific subnet range that i want to use and this is where uh some of the amazing power of you know coding this into higher level language like this comes is that you're getting autocomplete you're getting inline documentation so i don't have to go refer to aws documentation to understand hey what is subnet configuration i can i can actually look at this parameter this props and it says that you know subject configuration allows me to provide the you know the cider range and those things and here is also an example so how about i just copy paste this example because i do want uh say multiple subnets um i want a slash 24 subnet um and which is for public subnet i want a private subnet and a subnet for my database and then i can decide that i want to change this name so i'll say that this is my web subnet this is my app and this is my database a very standard three-tier you know architecture and a vpc for that and guess what i am all set to actually have a fully functional vpc uh the the cloud formation for that generated with uh with different zoning all of that done for me with few lines of code so as you can see now if i scroll all the way up i have got slash 16 range for 10.19 my subnets are on slash 24 it's a web subnet and so on so it's it's all done for me with literally five lines of code with the ability to actually go and check the inline documentation um have auto complete and any any of those errors that that i was getting so i i'll stop here from from a cdk high level demo perspective um because which is going to go drill into a lot of code as we go through how do we create a code pipeline but again just a basic structure is is you have got a bin folder where your primary app or application sits this is the entry point um then all your stacks are inside a lib folder it doesn't have to be in lib but that's how the project gets structured by default you can move it around um and cdk.out is where you see all the uh cloud formation that is uh that cdk generated for you uh so um before we move on so yes we saw how we can create a new vpc using uh cdk right suppose in my environment if i have a vpc already uh present can i refer that as part of the cdk but that's most of the most yeah as a developer most of the times vpc is already created by say my network team and how can i then use that as a reference yeah so there is a construct for which allows you to do a vpc lookup on the basis of some tags or or you know by a name so you can use that construct to get get you know specific ppc and then use that into your into your project so um yeah so that that would be a way to do that uh does that answer your question yeah yep yep so that's just possible and easy to do it yeah absolutely absolutely possible i mean as you asked me this question i just wanted to quickly touch on that i did import uh one of the other package which was for ecs you know uh so let's let's let me let me quickly touch on this that if i want to create an ecs cluster and those of you who are not familiar with ecs it's one of our container orchestration platforms um and let's say i want to create a cluster of um an ecs cluster and you know i want to refer my vpc in that so for that all that i need to do is um similar stuff that what we have done with the vpc in this case uh we will be using ecs and then i call the cluster and this is again another l2 construct that that we have got available uh so same thing goes here my this is my dev cluster let's say um and one of the props that it needs is that hey which vpc it belongs to so if i check the all the other parameters that this this needs i can say vpc and then very simply just refer the ppc that i've created up that's it so this way i can i can basically refer uh the resources that were created before into my other constructs cool so that's uh that's really the cdk demo part uh as uh we have been uh showing you and hopefully uh we have we have um you know show you that cdk is really a game changer and not because of some of these capabilities that we have been touching on that hey inline documentation or a complete ability to write this into your favorite programming language i mean that's that's really a cool aspect but um the the biggest differentiator that that we see is the ability that you can create your own abstractions and and why i think that's such a game changer is having worked in many large enterprise organizations i see that one of the ways that many organizations try to achieve their compliance or you know standards uh across the board is document what they call cloud patterns or best practices in a wiki document or somewhere and uh the adoption of that is generally poor i've seen a and reason being is that hey you're you're asking developers and all these guys who are time poor to just go on a wiki document and then follow a set of instructions manually how about if we enable them to have all of these approved ways of doing things or building an application through a code package that they can just employ and import into a project you know how about if they get a bunch of approved libraries that are available to them um that they can just add into their project and as long as they're following or using those they it's it's an easy approval process or easy uh you know way to meet the compliance that this is the this is what constructs uh library provides you that not only you can just use a set of constructs that aws is giving you however your organizations can create your own construct on top of it with your own internal best practices or you can create an l3 pattern which connects multiple services together in a way that that's again follows your organizational best practices so let's say that your organization's best practice is that every st bucket must be encrypted you know so you can embed that into a construct that you create and then make that available to your uh developers so so yeah so they don't have to follow a manual set of instructions to do that and and the way that we see that this is going to happen is already happening and will happen more is that with this approach there will be a central cloud excellence team or or architecture team which is setting up a framework in which uh people can create these reusable artifacts and and then different applications team can not only use these artifacts but can also contribute to this so again helping grow the developer productivity with with reusability and this is where the power of cdk lies uh the second thing that i i really want to touch on is uh quite often we see that in in organization there is this uh there exists a silo between different teams doing different things and sometimes this is by design the silo exists because the organization is set up in that way that there is a different team who is responsible for building infrastructure other one for code another one for you know config or or creating code pipeline uh and if that's by design i'm saying that you can still leverage cdk and you can make the most of it um absolutely fine however if this this silo exists because these teams use different tooling and different processes to do their things like if the infrastructure team is familiar with uh cloud formation and that's the only thing that they can use or if your development team are only familiar with python or typescript and they don't want to necessarily learn a declarative language of json or or ml uh and if that is the reason what cdk enables you is that bring this all under a single roof you know where your every the whole entire uh delivery cycle of your of your project is uh is managed through cdk you know so same uh tooling same coding language uh and and and breaking those silos and this is quite common uh what we see at amazon as ella was touching on is that we have a concept of what we call a two piece of size team where app application team ops team are they don't have us clearly separated boundaries that you know they only build the code pipeline and this team only does the infrastructure as a code everybody is responsible for every aspect of the application or the service that they are building and maintaining so yeah so that's uh that's really what i wanted to cover so just to summarize cdk is truly a really powerful tool uh i'll say that we we have built it with an intention to help uh developers be more productive i've personally been in projects where infrastructure automation parts especially takes so long um that some organizations tend not to invest into infrastructure as or as a code which i think is no longer an optional part this is something that every organization should do and there are there are tremendous benefits of actually automating your infrastructure uh so go give aws cdk a try um and uh we are listening you know in terms of provide us feedback go to our cdk repo uh upvote the features that you're looking for um and uh yeah and we'll continue to keep evolving it we don't think that we've got the uh as lars was saying that we've got everything right about it but we want to make sure that community keeps suggesting the features that they want to see into this and accordingly we keep evolving this tool over to you wish hey thanks nathan uh hi everyone first of all if i drop out apologies because i'm facing a lot of frequent uh intermittent issues today uh so here what we're gonna do is have a bit more fun we'll build a continuous delivery of an app by building a pipeline and let's see how we can deploy a simple application right so rather than if you can go to the next slide yeah so what we're going to do is sorry um so what we're going to do is as part of this demo i'll be creating a simple calculator application using lambda and api gateway and see how we can deploy that application into multiple stages using a code pipeline and what we're going to do is the entire pipeline itself we are going to create using cdk right and since my demo has multiple deployment stages and it has some waiting times so i have pre-recorded my video and i'm going to play that pre-recorded video as part of this session but i'll try to give that narration and step-by-step configurations what i'm doing that in this video and try my luck with my lip syncing so let's let's get started let's see how we can build a pipeline using cdk so in overall in this demo i'll be doing four main steps uh first of all i'll be creating a new cdk app as rethon showed using cdk initialize command and i'll be doing a environment setup using cdk bootstrap again i'll talk you through each step as we go uh here is an empty folder nothing in it right so we are starting pretty much from the scratch so what i'm going to do is i'll be initializing my cdiq application again i'll be choosing typescript tabs as my preferred language here and as soon as we uh start initializing as ritan said it will give us a boilerplate template or a or a base template for us [Music] as you can see it creates a bin folder which is a main entry point for your application uh it'll also create a lib folder where typically you create all your stacks and it also gives you a test folder where you can write your unit test and it pre-configures any testing tools like just again as a developer you don't have to stick to the same format you can change it in an easy way and if you want to change it that's where you go into package.json and change those configurations right i mean you go here and change your entity file if you want to and if you if you want to use some other unique testing tools like cinnamon chai mocha just go on the scripts and change those settings so you have that flexibility it's only that cdk in it will give you that starting point so as part of this demo what i'm doing is just cleaning out all the tests that come as part of the cdk unit so that we will write everything from the scratch right one important thing i'm going to add additionally as part of my cd application is called new synth style okay so in order to add that i'm going to open cdk.js file and add a new property and [Applause] add an audible student hello we still here yeah i can see everything yep go for it medium is paused for something yeah it pause a little bit maybe some flaky connections over here i think we lost him [Applause] let me check that was what happens with live demo folks so yeah skip with us just a minute and we are back yeah nothing can you hear us yeah i don't i don't think he'll be back i can't see him online anymore yeah it's frozen for everyone i can see people commenting on the chat that yeah i think he lost his connection yes sorry so there was something at my end suddenly i don't know what happened uh no problem yeah just as a curiosity uh the last few weeks we saw some really strange uh ddos attacks on the saps in new zealand we also saw some connectivity issues during the day but i hope i hope that's not the same but yeah let's go back yeah yeah we i i mean just uh today i've been praying uh so much to the demo god that it's a long demo please please keep the internet okay uh so sorry there there are some glitches but hopefully we should be fine uh carry on wish yeah sure uh so as i was saying in the cdk.json file i'll be adding a new property called uh i think if you can play a little bit uh behind and so bring it here a little even further behind because i think this is the important part yeah so what are we seeing is and uh the video quality is very bad maybe it's not streaming that well yeah it's it's much better now nothing so what i was saying is here i'm going to add a new property called new sim style and i'm going to walk you through why i'm adding this so in cdk we have two types of templates one is default uh also called as legacy but it's default today but this new template called modern template and that is going to be uh it's under preview at the moment and in order to use that that's where we have to add this property right and that modern template is required for us to create uh some of the modern features like creating a code pipeline and what it allows us to do is as part of your cdk bootstrap which i'm going to talk uh you know step you through shortly it's going to create all the required prerequisites behind the scenes so that cdk can perform that deployment for example you might have your container or docker images it automatically creates that eci repository for you it also creates all the required im rules so that cdk can perform those deployments and as i said since code pipeline is a new we have to enable that now once we have done that next thing is we will be doing what we call cdk bootstrap right in order to for us to deploy any cdk app into aws environment we have to do this cdk bootstrapping and this is where cdk will deploy all the click rates all the required resources into the target aws environment and one thing to note the cdk bootstrapping has to be performed between aws account and a region combination for example if i if i bootstrap in aws sydney region for an account and tomorrow if i have to deploy cdcap into singapore region i have to do that bootstrapping again in that region so command is simple you call cdkbootstrap command then provide permission in my case i'm giving administrator access but you can pass your own command as well and specify a content region combination in my case i'm not specifying that's because i'm passing the profile and it reads from the profile and giving profile is useful if you have multiple icons and i have a demo using multiple accounts today hence why i have created profile so that i don't have to remember my you know secrets and keys so using profile automatically you can switch easily between them right so i'll be executing this command shortly in this video and i'll show you how that new modern synth style is applied as part of the bootstrapping step and how the cdk bootstrap also automatically picks that account and region combination for it to deploy those resources so as you can see first line is saying that hey using new style bootstrapping that's where it is picking that property that we provided under cdk.json file and applying and as i said it's a aws account and region combination that's where it is doing that bootstrapping by the way all these seconds are deleted i only created for demo purposes now if i quickly go into my target account and look at the cloud formation that is where that cdk bootstrap will create a stack set called cdk toolkit right and if i go inside that toolkit sorry stack set under resources you can see cdk bootstrap has installed all the prerequisite requirements so that cdk can do its deployment activities later on so for example as you can see it has created a cloud formation execution role for itself it will also create a ecr repository or elastic container register in case if you have docker images or container images it will also create a staging bucket or s3 bucket to store artifacts as well as it will create all the required bucket policies so at the end it will create everything it requires so the cdp's cdk is ready to go ahead and deploy its resources right there we we completed it's going to take typically a minute minute and half one question wish um yeah so do we have to do cdk for all uh ctk projects sorry bootstrapping for all cdk projects yeah good question uh not required uh so you have to do cdk bootstrapping only when your cd cap maybe you can pause the video so you reque you need to do cdk bootstrapping only when your application has some acid code for example in my case i might have a lambda code or i might write you know some other asset code where i'll be putting some of the resources into s3 bucket or i might have a container images to run kubernetes or maybe containers on ecs then only we need to do the cdk bootstrapping but you if you have only the schema for example in your earlier case where you created a vpc which does not have any asset code that's and in that case you don't have to do cdk bootstrap got it right okay cool thank you wish yeah and as a next step as ratan said pretty much all resources has equivalent cdk constructs and i'm installing them for example pipeline lambda and api gateway which i'm going to use to build my project so that is there for me to use later on right so as we saw uh step number one we created a simple silica initialization to create a cd gap we did a bootstrap on cdk remember cdk bootstrapping is a content region combination right now here i'm going to rush bit uh faster because i think uh viewers are not that keen on seeing lambda code but focus is on pipeline so this is where i'm creating atlanta code it's the northeast lambda called calculator where it takes certain parameters and you know send the result based on based on operator parameter right and i've also i'm also adding a unit test here just to uh you know show best practices when it comes to development so whatever you write typically you have to write a unit test as a best practice right so i'm quickly bringing few unit tests and the reason i'm writing here so that i can build a nice story later on when i create stack right so just ignore it uh at the moment so in this step we created a lambda a simple calculator lambda and wrote in with this now we need to create a stack as rathan said a cdk will have one or more stacks right and typically you need to create separate stacks for a different indent in my case i'll be creating application stack to deploy my lambda and api gateway at this step step but later on i use another stack to create my pipeline itself so that you have nice segregation of stacks and as i said you typically in a boilerplate template you'll create a stack under lift folder and i'm creating a class called a file called calculator application stack and i'll be copying a few snippets here and i'll walk you through what i'm doing here right in this file first of all i'm importing all the required packages on the top that i need for lambda and api gateway and other other things on this stack created a class called uh application stack that is inheriting a stack class coming from your cdk core construct right and this is where we are creating that cloud formation stack in cdk next thing we are creating a lambda function and we are using there a function as a as a class which is coming from aws lambda constant as i highlighted there and if you remember rathan was mentioning pretty much every cdk class has a consistent parameter signatures right i mean scope or also we call it as a parent id or name and you pass the properties right let me elaborate a little bit more on scope so if you look at cloud formation the typical cloud formation you have an entire template under that you define a resources tag under that you create individual resources and their properties right basically it's building a tree similarly that in cdc also we need to build what we call uh constructory and in cdk you will have one or more stacks and each stack can have multiple resources that is where you tell whenever you create any uh you know any resource here hey this resource is part of what stack so that you can build that tree and cdk understand how to deploy this resource so in this case i am saying that hey lambda my lambda function belongs to my application stack that i have created uh here which i am highlighting so that this stack is responsible for deploying this right and second parameter is id or name it can be anything and i just mentioned here calc lambda is the name and after that you pass all the required properties for the lambda function so for example i'm saying hey for this lambda backend code is under app folder that we created earlier and entry point for my lambda function is calc lambda dot handler i'm using node.js 14 version here and specified certain memory size right this is how you can create a lambda in cdk similarly i'll be creating a api gateway again exactly same like lambda you use rest api as a class here coming from aws api gateway construct same parameters you pass the scope you can name it pass the name as anything that you need it's a string parameter and specify properties like what is the api name what is the deployment options and so on right and since i want to use this calculator application i want to create a method for my api gateway and that method will call my lambda as a backend that i have created right so here i'm creating a method of type get and using lambda integration class which is coming from aws api gateway construct i am referencing the lambda function which i have created on the top and saying that hey api gateway this is your backend method right and specifying few more templates to pass query string parameters for my calculator which will not focus that much um but one interesting thing i am doing here and i wanted to highlight here is i am outputting the api gateway url once we create api gateway as part of the stack deployment i am outputting that into a cloud formation right so that i can reuse it later there are multiple advantages this is how you can share information between stacks and if you see here i'm using cfn output that is level one construct coming from cdk and keeping it in a variable as i said advantages says as in cdk you'll have multiple stacks and this is the way you share information among them plus if you're building pipelines you can use this value to do further processing for example doing validation tests and so on and we which we're going to see that in action today now one interesting as i said i mean i want to follow the development best practices here and i want to write unit tests for every code that i write no right and this is where another great advantage of cdk comes into play so you can write unit test for your infrastructure as code i'm a big fan of this right you treat development best practices not only for your application code but also for your infrastructure score right so i'll be creating a unit test uh file here called application stacked test.ts and um pasting couple of snippets for my unit test and walk you through that as i said i mean it's uh it's a game changer to be honest because uh it's not only uh you know writing unit tests for your uh you know your resources but also for your entire infrastructure as good for example here i'm saying that hey my stack should have a lambda function which should have certain signatures uh for example i'm expecting node.js 14 for example a developer should write in this case and api gateway should have at least one get method here imagine you have s3 bucket or dynamodb you can test whether encryption is enabled or not or you can also bring your organizational policies for example vulnerability scanning static code analysis all those things you can bring as part of your unit testing so that your developers will get that fast feedback and your deployments will fail if something is not you know aligned or it's not written in certain way that you expect right so this is a game changer and uh imagine um typically in cloud formation even though it's a wonderful a wonderful way of infrastructure s code but you typically find out errors very late in the stage but whereas in cdk you can find out as part of your unit testing as well just to demo that i'm going to break a small signature here i'll be as a developer i'll be using world version of node like i'll change the version to uh let's say 12 here then 14 and quickly run npm run test so that i can show you how how the unit testing tools will catch those uh you know discrepancies and your build will fail and exactly same happens when you pass this to a code pipeline right and your developers will get that fast feedback so as you can see npm run test should fail shortly and it should say that hey expecting is node.js 14 but you're using a node.js 12. and it's a simple example i know that but you can do a lot of innovation here bring a lot of policy as code you can attach that with your unit test and do that next level of automation so which this this really enables organizations to to to do what we have been talking about that uh compliance such a code exactly exactly right and you will i mean developers nowadays developers even before putting into aws some of the times developers use local docker containers to build and try out things and they'll get that fast feedback even before putting that into let's say on the cloud right so that's how you can you can you know utilize those unit test as part of your continuous governance and continuous compliance nice nice yeah cool right so we we completed um three steps here we saw how we created a lambda function in node.js we created application stack and cool one cool thing we did is we also wrote some unit tests for infrastructure s code now let's do a main thing right this is where i'm going to go a bit slowly and focus on a bit more detail right so this is where we are going to create a pipeline that will allow us to do that continuous delivery and we'll do that through cdk and i'll be using uh the pipelines stack came under the lib folder and quickly copy a base snippet with a class and after that walk you through each line of code i mean we will be doing like a even though it's a pre-recorded video i have done line by line coding here so that we can go slowly and and walk you through the steps yeah sorry i had a question i mean so just just so so that we are all on the same page so far what you have done is you have actually built your uh web application right which is a calculator app yes and now is is the is the main part of the show coming which is how are we going to actually deploy that using a code pipeline which is also written in cdk exactly we saw we saw kind of a policy as code or test as a code now we are doing actual pipeline as code and whatever we built so far that is lambda and api gateway we are going to deploy that using pipeline into different stages nice good i follow you now it's kind of inception right so you write code to deploy the same code yeah so what i'm doing here is uh just importing all the required packages and created a class called uh piper which looks like we have lost wish yeah i think we lost let me check if his own on our internal chat sorry folks we're having a lot of connective connectivity issues here today uh yeah yeah let me see this is on planet unplanned cool but hey this is uh really quite unique i'm i'm not too familiar with that how in the cloud formation world developers would write a unit test do you do you know if there is any way that developers could actually do that yeah from a language point of view emo itself doesn't support any test runners and the benefit of cdk is enabling the ecosystem of the language itself right so we go away from a declarative uh uh only static language and we can have either a declarative or a multiple multi-process uh coding on top of that infrastructure that will be generated so the ecosystem around the language that you choose can help you define that part at least from my developer experience that really helps so this is what attracts me on cdk it's because i can use the same skills i have as a developer to uh infrastructure as a code which is fantastic and his back there we go your audio is really bad can you try to reconnect no yeah it is really bad could you close and come back to the studio yeah i really think that you may want to try it through your hot spot one time see if that helps while you do that what i'll do wish is maybe i'll just uh quickly uh take everyone through what are what is the whole end-to-end architecture going to look like in in the slide take is that okay wish okay so while which rejoins uh i'll just uh bring the the slide tick back if if you can stream my screen edwardo yes yes we are on uh and this uh if you could reconnect we will be waiting for you here cool so as you were talking that we have built this calculator app in cdk so it's part of the the cdk application the next that what this is going to do is build a cdk pipeline which is going to be another stack and the cdk pipeline will have or the code pipeline that we're going to build will have the usual stages like the source stage which is to go fetch the source code from code commit here or you can we can you can use github if you use that and then do a build stage which is where the cdk is going to synthesize this code and and spit out all the cloud formation templates and then we'll use that to actually do the actual deployment of the calculator app in one of the uh staging environment uh so so first into pre-prod and then into a into a production environment so this is the whole end-to-end floor that we are trying to build towards uh we're really at a cdk pipeline stage right now hey folks i'm back i'm audible way better so cool i'm trying uh my hot spot now let's see how it works nice okay yeah the damage online and yeah perfect so uh what i'm saying is uh so we will be creating a pipeline um again a class called pipeline stack inherited from a stack class here coming from aws cdk core and it's exactly same code as what we created earlier write the application stack but as part of the pipeline what we're going to do is first step is to connect to our source repository and in my case i'm using code commit which is uh i know git compliant source code repository from aws you can also use github and you connect pretty much the same way that i am doing except that you'll be using different construct for example here i'm using repository construct coming from code commit and i'm saying that hey connect to my repo by giving uh you know scope and name can be anything and passing my repository name and in github there's one small chain that you have to pass your github access token and you can keep that access token in aws secrets manager right so that's the additional thing you would do since i'm using code commit for simplicity so just directly connecting to that particular repo next thing that what i'm going to do is i'll be creating couple of artifacts or s3 buckets for me to use that as part of the pipeline again i'm using slightly older version of code pipeline here sdk or the construct and if you use the latest one um construct you don't have to create these uh it's automatically get created by pipeline itself now source artifact is required for code pipeline once it connects to your code commit repository it downloads the source code there so that it can use it for building uh as part of the build stage right and second artifact i'm creating here is cloud assembly artifact this is where once your build step is done a cdk application code is converted into cloud formation and it will be stored in this location as well as your any asset code like lambda code in my case so this will be a output folder from where you deploy uh changes to your target stages for target environments right now here is the actual actual part where we're going to create a pipeline so we created all the prerequisites for the pipeline so i'm just referring here a variable called pipeline and calling a new cdk pipeline class coming from cdk pipelines construct and signature should be pretty much consistent as we have been talking through uh this these demos so you pass the parent as a scope name can be anything and then you start passing properties for your pipeline right let's see what all the properties we need to pass so first of all i will be specifying what is my pipeline name looks like and this is the name you see in aws console once it is deployed and it can be anything um and after that you will be saying for the pipeline hey what is your cloud assembly artifact um and that's what we created on line number 16 right and uh if you look at the pipeline typically you'll have uh three stages in a pipeline as so stage a build stage and deploy as a first pass let's only focus on source and build let's not deploy anything at this point of time let's only do a simple things first so here i'm what i'm doing is creating a source action that is where my pipeline is going to connect to my repository in code commit so there is action called source action and you say add that and refer code commit source action as a construct and and provide what is your repository and refer that so here i'm saying that hey what is my action name it can be anything this is what you see in console and i'm going to show you that side by side once it is deployed right so this is the name of that action um and if you're using github uh instead of code commit source section you'll use github source action right and this is where i'm saying hey a repository is what we created on line number 12 on the top um just referencing that and saying that hey once pipeline connects to code commit this is where it's uh it's going to download all the files so that it can use one more thing you can also specify a branch in my case i'm using master branch but if you have multiple release branches and you follow that sort of deployment or release cycle then you can also give a combination of repository and a branch for this now next we are going to create a build stage right and for the build stage again we have a action called synth action and you have two options here one is you can use code build construct from cdk and build that code build project by your own or cdk pipeline also um you know has what we call simple synth action so basically uh it's a further abstraction created to simplify and make it make developers life easy so that you have to write less number of code here so i'm using standard npm cynthia because i'm using typescript you can also use yarn if you're using the yarn in your project so what it's going to do is behind the scene and edwardo and ruthen you're going to allow this it creates an entire code build project for you you just specifying few properties behind the scene it does everything for you in terms of creating build project using aws code bit right so as part of this you're saying hey what is my source artifact where my source files are and what is my cloud assembly artifact where my output file goes into and here you can bring your own build spec file or you can run build command as well you have flexibility there you can also bring your custom docker image if you have and specify that as part of your build spec file but for the demo purpose i am keeping it simple specifying couple of build commands here which is npm run build and npm contest that's it single line of code there and couple of properties and it should create an entire code build project for us and at this point of time i'm just committing and pushing all my changes to changes to repository [Music] right now one thing to note here is whenever i mean as a i think rathan already covered this whenever we do you know this we write this application code you can do that cdk synth so that you can find out if there are any typos or any errors in your code and you can find it out and um i i i do that all the time um right that tells me if the code is um built correctly or not and if everything goes well it should show me a cloud formation template and also you can show it and under cdk.out folder in your local local disk right and as you can see i already created around 550 lines of code and we pretty much you know wrote three four lines of pipeline code with three few properties now one thing to remember here is even though we are creating a pipeline and pipeline will listen to a basically a source repository right but our pipeline itself is not deployed yet so you have to do at least one deployment so that your pipeline is there in the target environment right that's what we are doing here we are saying hey cdk go and deploy my application code into my target aws environment and as part of that it's going to show me what all the changes it's going to deploy there you can also do cdk diff if you want to find out what is the changes between previous operations and the current one you can do that and i'm just saying yes and this step where i have speed up my video so that i'll cover that five six minutes of you know deployment uh times to save some of our time to cover as part of this demo right so here you go the pipeline is already created everything is green and that is where we called pipeline as my cal pipeline and that's what we called in the pipeline name there we created a couple of stages first one is source as you can see it has created a source stage it also create it's connected to code commit repository that we have specified and as i said it also created a build stage but with simple one line and few properties it created an entire code build project for us that means you as a developer no need to worry about creating end-to-end build project and i'm just showing what what it has done behind the scene by going into the aws code builder there and if you look at here what it has done is it's uh it's created a code build it also executed the command which was specified as part of the build command there and since it is a cdk app it also knows that it has to do the cdk synth even though we have not specified it has done by itself and it created that cloud deformation template and it puts that cloud formation template into the cloud assembly artifact that we created right and it's going to create both cloud formation template as well as any other asset code in my case i have a lambda code and in your case you might have also your container images and it will store that as part of the cloud assembly artifact quickly highlight so we only created two stages so far one is source and build but it has created a special stage called update pipeline or also known as self mutate right and this is one of the exciting stage that i wanted to highlight to all the viewers here so this is a special stage created when you gets created when you create a pipeline using cdk and what it does is cdk knows that you're creating pipeline through using a code and that the pipeline is going to change you might add new stages new actions and this stage will make sure that your pipeline is automatically updated to reflect those changes that means your pipeline is self-mutating itself based on a change you make on a pipeline code so usually here to simplify what pipeline is doing apart from deploying your resources like lambda and api gateway that we're going to see shortly it's also making sure that it stays updated itself how cool is that i'm super excited on that particular stage so what we have seen is we just now created a pipeline um using cdk but if you remember we just created a source and build stage we haven't done any deployment but our ultimate aim of this demo is to deploy that lambda and api gateway or our application stack right so that's where i'm just referencing here my application stack that i've created earlier which has my lambda and api gateway right and as i think as part of the previous talk rather than mention that some of the best practices where you have multiple stacks in a cd application right and even when when you deploy applications across multiple stages like prod or non-prod even though it is in a same aws account its best practice to deploy that using a separate stack set or cloud formation stacks right so that's where i am creating here a pipeline stage class and simple as that i'm just creating a new instance of applications that means whenever somebody creates a new stage i'm just giving a new instance of my application style that means you're getting new instance of lambda and api database so we'll see that in action as soon as we deploy and uh if you remember when we created the application stack we also uh outputting or giving back the api gateway url for further processing right so here i'm just referencing that and storing it in a variable and see how we can use that in code pipeline later on right just to summarize we created here a stage class and whenever somebody uses the stage class we are making sure that he will get here she'll get a new application stack right and let's see now let's work on the deploy stage as i said i want to as part of this demo i want to deploy into multiple stages so let me call first stage as non-prod and simply i'm creating a new instance of my pipeline stage that created on top and passing parameters again scope as this and name could be anything i'm calling this as known non-prod and now i'm saying hey pipeline add a new deployment stage or add a new application stage to a pipeline right so i'm saying add application stage and passing that non-broad instance that i've created on the top similarly i can repeat that for the production instance for example and say again create a new pipeline stage call it as a prod and add another stage for my pipeline so with simple two uh four lines of code we are adding two deployment stages using two different application stack instances right so again as a developer i have full control because i'm using my own preferred language here simplified coding and you can use all your object oriented uh you know principles here now i want to take this two a little bit further and see how we can add additional actions to a stage so you can do a lot of uh you know in a stage you can add more actions for example you can write a validation test you can run some of the shell commands and so on so let's see how easy it is to create few actions as part of a stage right so as a steps next step i'm going to demonstrate using couple of common and flexible actions there are many actions available by the way in uh code pipeline construct so i'm using couple of them as part of this demo right so here i'm saying that hey uh as part of the stage add actions and first action i'm going to use let's let me call it as validation test or smoke test and as a developer i come across this particular action and i find it very very useful called shell script action right very flexible you can run any shell commands within this with only passing few properties behind the scene it creates again end-to-end complete paint project for you code by database code but it will create you to execute those commands right and i'll show you how simple these uh children's shell steps is in this new action coming from 3d code pipeline right so action name can be anything again that will be visible on aws console once it is deployed and here i'm showing that how you can read those stack outputs if you remember as part of application stack i'm i'm exporting my api gateway url right so i want to use it later on this is how you refer that in your pipeline if you want to use it for further further processing so i'm just saying that hey pipeline stack output read from non-broad stage that's where you have api gateway url created as part of your non plot stage and after that you can run any commands and in this case i'm writing a very simple girl command so this is this is a uh so it looks like we can actually pass the parameters between stacks yeah exactly yeah right yes so so not only you can structure your cdk application in a way that different stacks are responsible for different things um and and then you can pass parameters between them exactly and the best practice is you create stack based on your intent right you might have a stack for your web let's say you have a a web application and you have a monitoring using cloud watch you have a back end using dynamodb and let's say front end as your s3 bucket and lambda and api gateway so you can create one web stack which contains s3 bucket your lambda and api gateway and your database or persistent stack for dynamodb and monitoring stack for your cloud watch now you have to share some of the information between them right and that is how from each stack you can export whatever the information you need and you can reuse that in other stacks nice complete separation of concerns and exactly and here we are taking one step further that we are reading that stack output within the pipeline because pipeline is deploying multiple stacks now stacks can share the information yes but i also want to use that stack output in my pipeline itself so that i can run some code build project so here if you look at the line 59 what i'm saying is hey pipeline read the output from the stage that we deployed stage is nothing but a stack created right so we created a new application stack part of that stage and pipeline is reading that api gateway url and you can do further processing on that nice that's super cool yeah and here i'm running a very simple curl command and i'm not uh testing for status code like you know um 200 or um something like that but typically you would like you know against a url and test for certain status code um but let's keep it simple i'm directly doing a curl on api gateway url and saying then it is fast right and finally you you have very flexibility uh here to run access in different order so you might have multiple actions within a stage and you can see in which order you want to uh execute right in here i'm saying that hey run this shell script action once the deployment action is completed which is once the stack is deployed right and i'm going to quickly do the same thing so that i again watch it through that because i'm really really passionate about shell script action and the flexibility the flexibility brings to the table so i'm adding again a smoke test or the validation test in production but this time i'm going to rather than getting it from the non-profit stage i'm getting the url of the api gateway from the broad stage and that's how you can have multiple stacks going through pipeline but pipeline is still able to get that output as part of this right right and i wanted to showcase another thing right another action so before non-prod and prod once your non-pro deployment is completed and smoke test is passed i want to bring the approval gate here and see how easy it is to create an approval action right again there is a manual approval action available in cdk and just couple of properties it will allow you to have that approval stage in typical world you might do it a little bit more in this you might have a sms topic sending an email to your qs qa team or testers and they test and approve it for this demo i'm using very simple step i'm going to approve in the console itself but just to show you how how easy it is to inject actions as part of the stages with only a couple of lines of code right and again i'm saying here is a run order is after my shell script action so it waits uh till shell script is completed and then executes it so we are pretty much good there uh we have what we have done here is we did a deployment to two stages um non-prod and broad and we also added a few actions here like shell script actions for both the both stages as well as we introduced approval gate between uh between the non-prod and plot right and i'm quickly doing a cdk synth again as i said so that i can capture any typos see if my cloud formation is getting generated yes it is and we are good to go and uh i'm just pushing my code into the repository if you remember our pipeline is already in our target our target uh account right that means this time as soon as we do a commit and push to our code repository pipeline should automatically you know start executing right because we already have that hooks between code commit and code pipeline so if i refresh quickly here the code pipeline should trigger there you go it's already started executing and i will be speeding up my demo a little bit because it's going to take some time and come back when when we hit that manual approval stage there you go our pipeline is created here and before i walk you through each entire pipeline what we have done let me quickly manually approve that gate so that we can start the production deployment so if you remember this is a stage we put between non-prod and prod for manual approval right so i'm just going to approve that but then i'll walk you through what has happened as part of the pipeline deployment right so if you look at here this is where we created uh we already had a pass source and build stages and if you remember that was a special stage comes with the cdk pipeline called self-mutate and as i said earlier it kind of knows that we have additional stages for non-prod we added further actions it automatically self mutated itself and updated itself to reflect those changes as you can see there it has created that non-broad stage that we have specified there and also it created a deploy stage which we have specified in line number 53 and it also added a script action shell script action which is we call it a smoke test which again creates a code build behind the scene and finally it also added a manual approval accident which i have just now manually put a mini table right similarly it also identifies stages have been added for uh stages called rod is added with the respective actions and the pipeline has automatically updated itself to reflect that so if you look at this i have not touched anything in console in order to configure my pipeline imagine doing this uh in a in a console and we know that pipelines keep changing right and you will always modify add new stages actions i have to go always i know back to the console manually configure it come back but here everything is happening automatically right yeah he's frozen to me let me see on the internal chat here yeah today's been the day folks we have been connectivity issues during the whole show apologies about that and yeah once again nothing new can you hear us and he dropped okay let me check on the internal shed yep he's coming back in a few minutes uh and uh why while we wait for rattanvish we do have a interesting question on the chat and i would show you under the buzz with that so imagine you are on an interview and someone asks cdk versus terraform which one would you choose or what are the benefits between one and another there is support for uh terra pharmacy decay or not what would be a specific answer to that yeah so it depends right i mean if i'm since i come from a development background i choose cdk as my personal preference so both has its own preference for example terraform might be uh some of the uh you know regulated customers might think teleform is gives them the contingency so that they can target multiple different clouds right whereas cdk will allow me to do that write everything based on language of my preference and you know what cdk is there's a cdk project happening with terraform as well so that means there will be a cdk support for terraform and once it is yeah once it is there um i was just reading other day an article once it is there that means your cdk becomes like a developer uh language to build iac no matter whether you use cloud formation under the hood or whether you use terraform right so again it's a personal choice if you ask me as a developer i would love to go with cdk because you know um handling with the large yaml file json files is tricky whereas writing in a in a language of my choice is much more much more easy easier yes absolutely so from the cdk ecosystem there anything that uh besides the cdk pipelines that you have been you have been showing to us right now anything that people should be paying attention that like is coming up or is under development that you could you know just drop a hint for them yeah a lot of as i said i mean um i just mentioned i'm using one of the slightly older version of cdk pipeline and when i built the demo but the new machine is also available so we are making a lot of improvements where you will further simplify that as uh that was mentioned as part of your his talk that hey we want to take away all the all the you know behind the scene things and abstract away things for example in this case i was creating source artifact cloud assembly artifact you don't have to do that even with the new version so that means you just express your your intent hey i need a pipeline it should work like this and behind the scene it does everything so i think we are heading in that space and another thing to look out is level three constructs right and um especially i look i mean i work with a lot of uh banks and uh financial services you know industry customers and uh regulatory requirements is that when they move their high risk uh workloads into cloud that has to go through the approval process and by having those patents first of all it speed up the development process because you're reusing the patterns created by your center of cloud excellence or cloud foundations teams so that means you're using but imagine those patterns are already gone through the approval process and you know checked all the compliance that means from weeks of approval process you are doing that in minute because it's already approved and as part of your pipeline you're using those you know constant level three constraints again i'm saying uh since it is an open source community i could see a lot of lot of uh you know innovative level three construct is gonna come into that uh coming to that area and you can watch out that area and you can reuse them fantastic and definitely working with the financial industry they have a different level of governance right and being able to write your infrastructure using a specific language it helps you integrate that with the wider ecosystem of projects and security right how what what kind of uh partners are you seeing uh on the financial industry that they are using cdk for is usually like a web app or a backups application or easy integration what is the most common uh project that they use i mean from my experience again from my customer exposure so a lot of customers using in the microservices world right and and a lot of even though in the banking industry a lot of digital transformation is happening and banks needs to build those loosely coupled components and microservices are coming really really uh handy in that regard and they they create smaller teams who are responsible for from implementation to deployment to operations that's where they are using cdk and um yeah so that's one example i can that that's great to hear it it's definitely something to keep your eye on and uh we are always releasing new patterns on the cdk patterns so yeah the hub is already live you can have a look on that and we are back right then joining us again connectivity issues mate ah it's uh it's one of those days i'm sorry guys yeah no it's just fast forward maybe till till the end we are almost there right so here uh a quick summary what we have done uh so far as uh rathan mentioned if if you don't have cdk installed you need to do that i didn't do that because it was already installed otherwise so rathan showed how to install that and once you install a first thing is to create a cd cap using cdk init command and you can choose a language of your preference like typescript that i have chosen cdk bootstrap is very important step and you you perform that with a combination of aws account and region and after that we saw how you can create multiple stacks we created application stack to hold a lambda and api gateway that we that we wanted to deploy and finally we saw how easy it is to create a pipeline using cdk and we first tried with creating a simple steps stages like source and build we saw how you have you can deploy the cdc application first time so that your pipeline is in the environment but what we also seen is you can add deployment stages corresponding actions fairly in a simple manner right using couple of lines of code and we also seen a flexible actions like shell script action uh as well as approved like you know manual approval action right and finally we saw that um once you deploy your i mean push your code how the pipeline automatically self mutate and does that there you go so end to end pipeline is green so here you go we did a pipeline as code without touching the console everything from cdk and it deployed my application into multiple stages uh with that uh written over to you great uh wish so that was a wonderful demo uh quite quite in-depth i mean i would say that that was probably 500 level you know some of that went over my head as well but perhaps that that because um uh it's it's uh it's it's a great whole lot of detail and it truly shows the power of cdk there um so the next thing that i believe if we can share my screen if i'm still sharing uh let me share my screen as well so yeah so one of the things that we wanted to to do today uh which i think um do we still have time wish uh for showing a multi-account uh set up what do you think lots of lots of connectivity challenges today because of that we have obviously run out of time yeah maybe i'll take a minute to talk rather than demo so uh but i i think uh one of the questions might come is if you go into the next uh slide this one so the part of the demo what we have seen is i deployed multiple stages using a pipeline into the same account right but it's also common and i've seen that in many customer places that you keep your pipeline in a separate account they might call is a call call as a tooling account or maybe shared services account and your non-profit broad or you know test accounts are on separate aws accounts and a lot of times i get this question hey can we do that using cdk cdk pipelines absolutely yes you can do this there is no changes pretty much the same step as i showed already as part of the single account only couple of small changes happens is how you do the cdk bootstrapping if you remember we do a cdk bootstrap provide a content region and then provide a permissions right so when you do that on so let's say target deployment icons for example uh let's say here in the slide you have a beat icon i'll just quickly bring that slide sorry the video so you can just touch on that part yeah that's the really important part so i'll just touch that if you um because i think that's a very important uh aspect so let me do that quickly yeah so right yeah so this is the first account that i have where i want to store the csa pipeline from the tooling or shared services account and it's pretty much the same as what we did earlier right you are saying cdk bootstrap provide what permission cdk can have in my case it's administrative access then you're specifying your content region combination let's say nothing differs here but where it differs is on the target accounts where you want to deploy resources let's say i'm deploying lambda and api gateway into prod account which is separate aws account sits outside this devops second of mine that is where a slight change happens and i am going to highlight that so this is where you do the bootstrap in in this fashion so pretty much the same and you start with cdk bootstrap you again provide permissions provide account and region combination for their product account but if you notice in the middle i am specifying trust to my devops account right so that is my devops account i'm saying here is hey my devops second can assume a role and deploy resources into my prod account or in other words cdk is creating across a control here and providing my devops account to deploy into the prod account so that's a subtle change you have to do it yeah that's part of the good stuff you in a usual way if you're not using cdk for example you may have to do this manually right uh trading across account role and everything yes and if you are doing that in cloud formation uh there are a lot of lot of steps in it yeah and i i never got it and i never got it working uh for the first time because i made some one or other mistake when i am doing referencing right because you have to assume a rule and allow other account to uh you know assume that role and uh deploy the resources not to mention the indentation on the ammo that and sometimes so yeah so so that's uh that's really the main difference right between a single account and cross account which is uh to make sure that uh the devops account has got uh right privileges to do the deployment into uh non-prod as well as pro account one another small thing is when you're creating the stage you pass environment variable i think you covered that already as part of your entry point the bin folder you pass the environment where you're specifying what is the content region right similarly when you're creating stages within the pipeline you can specify what is my target account and that's how you say non-broad stage is going to the non-prod account and plot stage is going to the product one exactly yeah yeah so that's fantastic yeah yeah i think just mindful of time folks uh uh we are almost closing for two hours now uh and before we close up there is anything that uh both of you would like to share with everyone like you know a heads up or keeper eye on something so then we can close that nice demo in a really fashion way yeah i mean really what i i would like to say that um some key key takeaways is um as as we have discussed and shown you that how you can model your infrastructure in any one of your familiar languages um so so give cdk a try uh even if you are a big terraform or a cloud formation uh fan uh just take cdk for the ride you know and see that how how it fits into your development processes and everything um which was showing that one of the coolest thing about a cdk is that you can actually do unit tests on your infrastructure as a code which is which is really incredible i mean third is uh cdk pipelines allow you to to apply the same development principles that you apply to your application code you can apply the same to your uh pipeline you know so writing in the same language um so what we're trying to bring here is that you're not just writing your infrastructure in the same language as your application code but you also your code pipeline is getting built into the same uh way and finally cte constructs we've shown you the the the incredible power that cdk construct spring in terms of improving your productivity uh ensuring that how you can use these reusable components to speed up your infrastructure record as well as create your own reusable components which you can share within your team within your organization to meet your compliance requirements and those things so that's really the main four if you have to say that one of the key uh strengths or key uh takeaways from this session is these four oh sorry go for it you're good yeah pretty much a second what you said and plus come and join us on open source community and help us building new construct let's build some innovative level three maybe level four level five construct as well fantastic well i just need to thank you folks for being uh with me in the show and also the challenges of connectivity during these two hours but yeah it was a great presentation i hope everyone that is watching uh you know picked up one or two tips and tricks from there the video will be available uh just uh after that and yeah thank you everyone have a good night from new zealand bye-bye cheers thank you [Music] [Music] you

Channel: From the Field NZ

Views: 78

Keywords:

Id: Ci9ZFIA6iUg

Length: 123min 25sec (7405 seconds)

Published: Mon Oct 04 2021