Exploring the CDK for Kubernetes - AWS Online Tech Talks

hello everyone and welcome to this online tech talk on exploring cdk's cdk for kubernetes my name is palvin mystery and i'm a senior developer advocate with the aws container services team i worked in aws previously as a specialist solutions architect and as a manager within aws security prior to aws i worked as a security consulting leader at global enterprises you have my email address and twitter on this slide so feel free to contact me for any questions on cdk cdks aws container services or security in general so this is a level 300 tech talk so basically understanding of aws services like amazon eks and awps card cloud formation is expected first we'll quickly understand the fundamentals around aws cdk in the context of amazon e key ks or amazon elastic kubernetes service next we'll do a deep dive into the cdks project followed by an overview of cdk's plus package and at the end we'll use this feature in action to build a simple app within the eks cluster using just a few lines of code with cdk cdks and cdks plus the talk is structured such that you easily be able to reproduce code and get hands-on with these tools the goal of the tech talk is to leave you with an understanding of cdk cdks and city cage plus and how can how you can work with these tools to improve your operations and build workflows for deploying kubernetes applications we'll end the tech talk with a q a session so before we get started and actually dive in we are making some assumptions one that there is a high level understanding of containers and kubernetes in general specifically what a container image is and how to work with container images kubernetes yaml and so on and so forth we also assume a basic understanding of working with aws console and aws services like aws cloud formation and amazon eks so cdk or cloud development kit and amazon eks let's take a look at the evolution of the tooling and processes in infrastructure provisioning it probably started with manual deployments where we used wikis and playbooks which sometimes were outdated i'm sure most of us can relate to some of the projects where we had to use it the next step was scripting everything in bash and it worked until the complexity was too big because bash is not built for creating complex deployment frameworks so it was hard to maintain an important next step was to abort the introduction of infrastructure provision engines such as aws cloud formation or terraform and others i mean these tools were would hide the complexity of state management rollbacks drift detection error management and so on and so forth many companies and developer communities adopted the new infrastructure as codeway and the adoption increased rapidly and this led to growing size of templates which were hard to maintain in json or yaml format so the next idea was to use generators write code that generates json or yaml document from classes and methods of programming language and on top of that idea new tools emerge such as aws cdk or polumi which provides tools and framework to create custom abstractions for cloud infrastructure so aws cdk or abs cloud development kit is an open source software development framework to model and provision your cloud application resources using familiar programming languages you can do this in different programming languages such as python javascript typescript java or c-sharp golang is coming too soon to cdk so with cdk you could be much faster than with previous tools because you work with your familiar language and concepts classes and method without switching context you also have all the tools support from the programming language such as autocomplete inline documentation tests linting and debugging and an important part is that you are able to build your abstractions and components of the infrastructure and application so there are three main components of cdk the core framework the aws construct library and the command line interface with the core framework you can create and structure apps that contain one or multiple stacks stacks are logical units of infrastructure which contains multiple resources and mapped one-to-one to cloud formation stacks it is a good practice to divide resources into stacks that have different life cycles that is you would create one stack for network infrastructure such as vpc another stack would have an elastic kubernetes service cluster and yet another stack would be the application that is running in this cluster we'll actually demo this at the end of this talk so stay tuned the current the aws construct library is a set of components crafted by aws to create resources for specific service this helps to decouple libraries and use only the dependencies that you need in your project it is also built with best practices and security considerations to provide good developer experience ease of use and fast iteration cycles the cdk cli helps to interact with the core framework to initialize project structure inspect differences between deployments and deploy easily to aws we start our project by executing cdk init that will generate a project structure for our specific programming language and we can start creating our app add stacks constructs and resources once we are done we have to build the project that is if you use typescript we use your command npm run build then we need to synthesize our code to cloud formation template we run cdk synth and this will generate cloud formation templates and assets and we call this bundle cloud assembly before we deploy we can inspect what we change if we deploy this cloud assembly which resources will be deleted updated or created so with cdk deploy we push the changes to the aws cloud formation service and from there the service will create crowd resources in our aws account so let's get hands-on with cdk and use cdk to provision an eks cluster from scratch okay so here we are in the terminal and i'm just checking the aws version and the cdk version installed within my cli um you can do an npm install cdk cdk to install cdk cdk doctor allows you to look at the configuration within your terminal or your laptop and i'm just creating a directory which allows you to create your app and then once we are in the directory i'll do a cdk init language type script this installs all the boilerplate that you'd need for your application and once we have that installed we'll open up the code which allows us to view the boilerplate which was created by cdk so taking taking a look at the package.json which includes all the dependencies that were installed by default we have a few um or installed already but to create an eks cluster we'll need a few more um we the starting point of the app is is within bin and and the lib has a typescript file which is your enterprise into the to the code so that's the one we we will edit but before we do that we'll just run npm run build which will build our code obviously we don't have anything in the code yet so when we do an cdk synth to synthesize the cloud formation it will create a default cloud formation script which doesn't create any resources for you so now we'll go ahead and install the libraries that we need one of which is most most important which is around eks but we also include cdks and cdks plus because we'll we'll use that later on and constructs so once that's installed we'll take a look at the package.json file again and that has all the dependencies installed and now we can add that into our into our code so we do an import star as eks from the it will be a cdk eks library and then just use that library to create the cluster new eks cluster with the construct scope and the name of the cluster which you can pass on here and then the props and one of the props you need is version so we'll create the version of the kubernetes that we need and we specify that through kubernetes version and we recently announced version 18 1.18 support um so we'll create that so npm run build builds the the code compiles it and then synth will show us the actual cloudformation template which has all the resources that you'd need to build an eks cluster so this is how easy it is to use cdk and create an eks cluster using cdk you can switch versions and add a lot more we will look into that as well and now i'm doing a cdk deploy which lets us review the security related changes and once i say yes it's going to create the cloud formation change set and run and deploy that into cloudformation um so let's take a quick look at my console and the aws console and we don't have any clusters yet but the one service i'm interested in is cloud formation so i'm going to click um and find out what's happening within and here we can see the creation in process and we have a few events that are kicked off and some resources which are being um which are being created so this creates your i think there's you have to wait for a few minutes um but once it's once it's created um it will let us view the cluster by running cube cutter commands and adding it to your um to your config your cube config see through the expected output as you can see on the screen now that once it's created it um it gives us two commands one is the config command which you can use to update your cube config on your local machine and that will add the context and you can switch the context uh it'll it switches the context and to verify that you have the contacts you can i can i'll be running like cube get nodes um and by default without specifying any nodes um you have two m five large ec2 instances um running version 1.18 provisioned with amazon amazon linux 2. um so we have that up and running now so you have a cluster up and running with just one line of code as you can see and we're going to use the github command line tool to create a repository which will allow us to include that change version and versioning around using cdk which isn't quite native to developers so i think this is the the cool benefit of using something like cdk against eks control which is a command line tool we recommend to create the cache clusters however with eks control um you don't have that general purpose language experience and you you don't have that experience that developers are used to so this allows allows you to follow the get workflow your github workflow and what i'm doing here is committing the code out to a new repository and i'm pushing it out to to this repository by the way this repository will be available for for your references again on github it's a public repository so you'd be able to refer to it so now that we have pushed pushed our code we can take a look at the documentation so it's the cdk constructs within c within for eks allows you to add different profiles like fire gate profiles and really specify and change your cluster provisioning so in this case i'm just copying a cluster sort of fire gate profile adding a target profile along with the managed nodes which have been added so i'll just i can just reference that add the profile and see what changes are happening so the one way to do it is again run the build and that would compile the code and cdk synth will show us what additional cloud formation stacks or resources have been added um and actually you there is a different command which cdk-diff which allows you to change look at the changes between the two versions and here you can see that we have fargate profile which is um which has been added so the ones highlighted in green is is the additions which are the differences to the previous uh cloud formation stack so actually we are not going to sort of commit this and deploy this we are going to revert back to what we had which is the two instances and just run with that to keep it simple so i'm going to run the build again just to make sure we have the the cloud formation without any changes if i do a cdk diff i should expect no changes to the cloudformation stack so that's great and what i can do is again assuming that you have a few developers responsible for your cdk cluster or any eks cluster using cdk you can manage the life cycle of that cluster using a mechanism around gate and have branches added and have a workflow around the change process so that there is that um rigorous sort of governance around how you are managing your kubernetes cluster which can be useful in terms of uh operating in a in a regulated environment where you need to demonstrate um the workflows and the change processes to your to your internal audit teams or external auditors as well so in this case we are just showcasing how to use github features and create a pr to manage the manage the code which has been added so really a ci cd workflow can be folded around this so that you can manage the code and the version life cycle within your within your cdk project so i'm just adding a github pr review and reviewing that and merging the code so that i can see that there has been a request to add a fargate profile and i'll delete the local branch so you can see those changes up in the repository when you refer to that repository as i've shown on the screen so for the pack you can see that there has been a few historical comments and changes to that to that cluster and using your your commits you can add the governance around how the how the cluster has been managed so now that we have a cluster running let's look into cdks we launched cdks or cdk for kubernetes in mid-may um aladdin nathan wrote a launch blog that introduced the open source project that lets you use programming languages like typescript python and now java to generate standard kubernetes yaml there's three important things that i think cdks helps customers with first of all cdks is a software framework that lets you go from code to config so using cdks you can define kubernetes apps and architectures using familiar programming languages and generate well formatted yaml for your applications every time this lets you effectively reduce or completely eliminate the need to copy and paste a template to do what a lot of folks refer to as yaml engineering using kubernetes cdks you can turn the best practices for how you define kubernetes clusters and apps into code libraries and share them easily with your teammates across your organization or in the larger community so we think this is a really powerful way that people will be able to share different paradigms and best practices for their kubernetes clusters and then finally we think this is really important point that cydicates is projects that started by aws but it's not for aws so cdk is for everyone it runs locally and it generates kubernetes yaml that you can deploy to any cluster anywhere and this lets you standardize across on-premises or any cloud so let's talk a little bit about the ammo and we usually yaml everywhere in our lives like kubernetes uses yaml it is the data configuration and definition layer for kubernetes clusters so using yaml is tried and true for us yaml is great because it's easy for humans it's easy for humans to read and to understand and to use it in a simple way so it's universal so you can use yaml for almost every system for declarative straight and it does a great job of being declarative of representing the desired state of the system and that's part of the reason why we all like it as a part of kubernetes we can represent the desired state of the system and then use a state machine within kubernetes to actually bring that system to the desired state so yaml is also very static yaml is very easy when you have a few things to declare so if you have a simple web app or a few services on a cluster maybe setting up a config mac yaml is very straightforward it's easy to open a text editor and just type the the code but as you grow as you start to run lots of app on your cluster um and use kubernetes across on in your organization yaml can get really hard and we also see this when the job of defining and managing apps in kubernetes gets passed down from ops to dev so when developers start taking over management of clusters they have to do a lot of things that is not necessarily intuitive to them in the way that they write code so they have to do a lot of boilerplate and it means that it's manual with a lot of copy and paste either from a community repository or from inside your company and it's fundamentally not solving the issue and that's where general programming languages come in um so they are what developers are writing so they're familiar with them and in kubernetes community there's a lot of people writing and very familiar with golang and in data science community there are people familiar with python and these languages are also imperative which means that traditionally they are not great for defining the state of a system but they are really good for defining business logic and complex dynamic applications that we need to write it's a lot of great tooling and things that you can do with gp general purpose languages that you can't necessarily do you do in declarative yamo and that's where um cdks comes in the cdk for kubernetes kind of bridges imperative languages with declarative yaml and so one of the criticisms is that there are ways to use imperative language on kubernetes today so for example you can write custom controllers in go that directly call the kubernetes api but that actually is harder for most people to implement because you have to own the state machine and it's hard to find state in a complex system like kubernetes the established community tooling for defining state and maintaining state or cube cattle for example are very good at holding the state within the system and bringing that state to a desired place and so re-implementing that logic as a developer actually is really daunting unless you are an expert and the reality is many people using kubernetes and developers are less familiar with kubernetes and this is a daunting task for them to take on if they just want to get advantage of imperative language definition so cdks can give you both and it simplifies the writing process and allows you to produce best practice yaml for configuring your clusters and applications let's go ahead and actually see it in action because this is the best way to get a feel of how it works so in this demo to give you an overview we'll be creating what is referred to as a chart which is basically the codified application architecture in the language of your choice in this case we'll use typescript within this chart we'll include a couple of basic constructs for deployment and service so let's take a look okay so let me create a a new directory for cdks app which is separate from the cdk eks cluster that we created and go into that directory and what we do is similar to cdk we have cdk init typescript and it creates a boiler spoiler plate for us for the actual cdks project so here we'll just close that cdk eks app but we have the cdks app open now with the boilerplate and it includes an import of the kubernetes typescript so the library and it it has all the constructs within within the kubernetes library for us to import and we'll do that um soon uh in the main.typescript file that's the one you would you'd add it edit i'm just going to add a label pod info to it and um exactly as you define uh your structure within yaml you we will use those um constructs from kubernetes that has automatically sort of imported the deployment construct and that has a spec which we'll build upon and i'll sort of add more options to it so i'm just going to paste a spec which is with a container and three replicas um to create our deployment with the image pod info which is pretty commonly used for for a demonstration by stefan prodan and we just use npm sort of npm run build to build the build the code so next i'm going to do a cdk synth but before we do that let's look at the this directory which will actually generate our yaml so right now it's empty but before we do cdk synth we need to do an npm run build so i'll do that and that should give us the ammo it runs some tests as well and this is expected it fails when we run the test using some of the checks within the test but we can delete that but what we can see here is the yaml has been created so within a few lines of code that we presented um cdks has created the ammo i'm just removing the test so it doesn't give us any more errors from our package.json okay so yeah here we are with the deployment and that's worked fine we've removed the test and what i would like to do is actually add a service as well so we'll pull in the service construct from kubernetes that we've imported the js which comes by default um you can also import any other constructs you want but in this case we'll stick to the standard kubernetes constructs and use and create a spec around the service and again i'm going to just create a load balancer which is going to provision a classic load balancer in this case on port 80. and use the target port for for a deployment which is i think 9898 and then we can specify the selector which is the label so now when we run npm run build it should provide us with the update yaml which includes both the service and the deployment types and we can go ahead and um and run or add it using cube control or cubo and apply it to our cluster just to to verify that yeah just to verify that we are using the same cluster that we created i'm just going to uh get nodes um it shows 1.18 again and checking the config current context it's the cluster we provisioned so i'm going to go ahead and apply the definition and that's created a service and the deployment so we have the service created and with three or three replicas and i'm going to go into the console and look at look at the load balancer within the ec2 service and it shows us the dns name that we can pull out and see that it is up and running so pod info is presented to us and we can go ahead and use the app so this is really simple example of deployment and a service using cdks right so let's look into cdk's plus which is a kind of a follow-up release to cd kate's and it was uh following the launch of city hates in in may um aladdin elise launched cdks plus in july as a package within cdk its repository to enable high level constructs so city cats is a live city cats plus is a library built on top of cdks and it's a rich intent based class library for using the core kubernetes api it includes handcrafted constructs that map to native kubernetes objects and exposes a richer api with reduced complexity um we'll take a look at how uh within the context of what we just demoed to show how this helps within a simple app that we created you can head over to our github repository and try it out and you'll also find documentation for all the available constructs as well as a full api spec and we'd love to hear what you think is missing and if you can actively participate in development you can add your own ideas and help the community so without waiting any further let's take a look at cdk's plus in in action um so in this case we'll um in this demo we'll use that cdks app we created but change it to use um the cdks plus constructs okay so we're back in the in vs code and i'm going to i think i can check the package.json and it actually does install cdks plus by default when you do a cdks init so we don't really need to install the cdksplus here now but what we can do is remove the references or the constructs which we pulled from from kubernetes but import k plus uh from cdh plus library and having commented out the the previous code i'm going to um create a a new deployment using cdks plus and in this case i'm creating a deployment which with much simpler definitions uh props around the deployment so what we have gone and done is make it more relevant for developers so that when they are building deployments they don't have to worry about some of the details that goes into the normal kubernetes yaml in this case you can see i'm just adding a container reference container prop and it has [Music] just a quick reference to the image we want to use in this case we're going to use the same image and we're going to use the same port so that's our deployment and it's it i think i'm following a new library then um then before and that's why it's giving me those those highlights within the code but let's carry on and when i expose this uh so this is a much simpler interface to create the service so deployed at expose at 80 and that is kind of a wrapper around the service um that in the service construct which we used previously so here we just specify the service type as a load balancer and this is it so all the boilerplate for yaml which which needs to be created is created using simple few lines um so it's for a developer this is much easier now i'm going to troubleshoot the errors i'm getting as in the highlights in the code it seems like i'm running an older version of cdksplus so in the package.json i mean this is a good exercise for for you when you're dealing with um cdks plus or cdks um because they are they are in alpha um so i'm going to install the latest cdk plus version and for a good measure so here you can see it's updated the cdks plus version to 0.32 but cdks is point um so update cd gates 2.30 as well 0.32 sorry and yeah i think i'll have to go ahead and remove my node modules directory and reinstall the node packages npm install so that's what i'm doing now so that updates my packages to the latest version which i have updated in package.json so there you go it has updated my ammo and sorry so it has updated my node modules and now i can go ahead and delete the the yaml that was generated free previously and follow the steps which we are used to by now which is npm run build and that should compile the code and and then we'll let the uh this folder with the ammo and that's created the service and the deployment so that's cdks plus which simplifies the constructs okay so now using what we have um so we created a cdk eks app which had which which helped us create an eks cluster using the eks library but then separately we created a city cates app which using the cdks plus we create uh created a service um deployment uh service which is exposed on port 80 so how about using or or using that chart within the code of cdk so right within one piece of code you can have the chart that was created using cdks but you are using using your cdk more familiar cdk constructs so keeping it all within one piece of code so let's jump right in okay so we are still within cdk's app but what we'll do now is move into the cdk eks app but before we do that we want to really use all the code that we have from the cdks except the app constructs which are not needed so we don't need to initialize the app we can copy that into the lib folder as a new file um and i'm going to name it as my chart um surface file and name it as my chart dot a yes type for typescript and paste it there and because we are not using the app construct we'll just remove it and there we have it like just within within one piece of code like we have we have merged the two codes that we created and now we want to pull that into into our main code which is for the cdk so import cdks from cdks and import plus or my chart from from my chart so so that's that allows us to pull in the chart that we created and reference it in in this piece of code so we're going to add this function which is a unique like feature which we've added recently to add cdk's chart we name it my chart and then create a new instance for for the chart as a new cdks app so when we initialize that and then pass it to my chart with the name my chart so this is it so this is a one liner which allows you to add the chart that you have created for your application into the cluster and this is great because you can go ahead and your application within the cluster separately and then using a single piece of code you can just add it to your cluster so a single piece of code to maintain and manage for your for your application um and build it from scratch so um so i'm sure i'm we're we're sure that the developers would love this experience when they are both managing creation of clusters and the code or the teams who are separate creating eks clusters and building applications within eks or kubernetes they can work together using a single piece of code so here i'm just following the experience we have seen before around npm run build and making sure we are in the same cluster and deploying that to our cluster all right so whilst this is building i'm going to take a look at the console and we don't have any load balancers created yet but using this time i'm going to look at one of the new features which was announced around importing helm charts uh within the kubernetes construct so this was a a new feature recently at um by matthew bonig and that was merged by lad so if you are to review any new features this is a good place to start in terms of the helm chart which was added and this is how you can go about understanding what went into that change so that's a pull request that matthew created and that enables the cdks to use hem charts as an input [Music] to find out more i'm just going to go in and look at some of the docks around it i mean we're going into the cdks github we have examples directory and that should provide you with some examples to play around with and create some of the code around custom resource definitions and ingress and pod info which is one of the examples i'm using right now so um so yeah but to dive deep and and understand more let's go to the to the website cdks.io and um the docs which are pretty comprehensive so um that has the new feature added to there and there you go so this is um this is a new feature in terms of adding helm charts to cdks that's something you can try out okay so we'll refresh and hopefully we should have the load balancer created by the code and we go ahead and take a look at that and it has the app up and running as we intended so we can go ahead and follow the workflow around creating a branch with all the updates that we have added in this case we have added a cdk's chart to our code and we can follow the the usual sort of get workflows and get hub workflows around managing our code and updates to that main piece so hopefully this provides you with an overview around what is the value that um cdk cdk's cdks plus provides you in terms of managing your eks clusters and and the workflows um so you can go ahead and use this repository as a reference um to learn and use the documentation but i'm just showcasing how you can have a workflow around this so hope this is useful so we are nearing to the closure of this tech talk but before we do that a few features around importing specific kubernetes versions into cdks these are the that's the command that you'd use cdks import and kubernetes version uh you're interested in or setting a api version for a specific object or using a custom reef resource definitions um so easily you add crds to your library which are not uh which are outside of of kubernetes so um so custom crds you can add those to to to cdks as well things to know before you can with the cdk and cdks project are in alpha and uh constantly going through a feature release cycle and updates so keep an eye out on the repository and it's built for entire community so it can work with any cluster anywhere in terms of kubernetes and we're adding more support to different languages we just added support for java but have plans for go and net um so we love your contributions so open an issue or a pull request at aws lab cdks on github and to recap quickly what we went through we look at look at aws cdk cdks cdks plus and how they can use they can be used in conjunction to manage your kubernetes and eks workflows so feel free to contribute take a look at the the repository and the project and here are some resources which would help you understand these projects so feel free to connect before we close i'd like to thank elad and nathan for their vision and contribution towards the cdk and cdk explorer city kate's projects and our fellow developer advocates per hood and adam who have great content around um around eks and cdk in general so do follow them on twitter and ask any questions to me or them and they'd be we'd be happy to answer any questions you have but right now we can open up for q a and thank you very much for attending this webinar so hope you found it useful and i'll open up the questions thank you

Channel: AWS Online Tech Talks

Views: 2,259

Keywords: AWS, Amazon EKS, Amazon Web Services, Cloud Computing, Kubernetes, Webinar, cdk, cdk8s, containers

Id: AkPsCnEgWbY

Length: 49min 5sec (2945 seconds)

Published: Tue Oct 27 2020