Introducing Amazon EKS - Brandon Chavis & Arun Gupta, AWS (Beginner Skill Level)

hi welcome everyone we are extremely stoked for the attendance at this hour I know everyone's probably wanted to grab the Rolly bags and get out of here so we we debated the merits of just spending the 35 minutes that we have today for just QA but instead we decided to go with kind of a middle ground and really narrow down the slides give a minimal demo and then give a lot of time for discussion because I'm sure there's plenty of questions based on what my colleagues at the booth have been saying so a quick introduction I'm Brandon Chavez I'm the product manager for eks and I'm Erin Gupta I'm a principal open source technologist responsible for the open source containers and serverless strategy so yeah let's go ahead and get started all right cool let's we're gonna get started brandon doesn't have a twitter handle I do have a Twitter handle so it's alright you know you can tweet me and I can pass the message to him I don't know how to use computers well you're a product manager so what is e KS e KS is a managed kubernetes service from Amazon we introduce this at reinvent last year what do we get as part of this managed service if you look at kubernetes cluster it consists of two pieces one is a control plane which is where your masters are running and then there's a data plane where your worker nodes are running worker nodes is where your containers are running so as part of eks which is elastic container service for kubernetes that's what the eks stands for we give you a managed control plane as part of that you know it's a fully highly available setup what that means is we are running I know if you're picking a region say us us to where the limited preview is available today then when you are asking for a eks cluster if you do have access to a limited preview then we spin up a master in a highly available configuration that means we're running it in multiple availability zones so to be specific 3 availability zones in a region in addition your HDD is also running across three different availability zones that's fully available the HDD itself is encrypted backed up and all regularly done for you so that you don't have to worry about it and we also provide you full upgrades in the sense that if there are kubernetes versions coming up newer versions coming up we provide you an option of automatic upgrades or manual upgrades and then also in terms of if you look at how the master need to be created if you look at the documentation it talks about the instance type that should be defined based upon the number of pods in your cluster so the worker instance type needs to be scaled up and down just to be a little bit more optimal so we take care of not just managing the cluster for you but also upgrading the node instance type and the version upgrades as well this is an extremely humbling statistics for us you know CN CF runs a survey every few months and then they seek what is their primary choice of container platform and in addition where are you running that container platform so this is really a shout out to the community that is in a fantastic work up until Amazon announced eks there are a variety of ways by which you can run kubernetes cluster and that all is existing in the community without really a lot of support from Amazon in that sense but we are a very customer obsessed company 90 to 95 percent of our roadmap is driven by customer demands so our customers have been telling us run kubernetes for me you know I mean we would love to we manage our clusters but there is a this is a undifferentiated heavy lifting for us and we would like you to run the cluster for us so this number is just an indication that our customers were really asking us to run the cluster and that's the reason we announced the manage chromatic service now we did build this service thinking through this you know very deeply and as we do with all of our services so let's go through the core tenets of why we build this service and how are we looking at it so the very first tenet itself is it's a platform for enterprises to run production grade workloads some of our most innovative some of our customers who are extremely large and very innovative have told us that we would like to run a huge enterprise workload on this cluster and that's sort of one of the biggest cannet over here that we want to give you the illa tees that you are aware of coming from AWS whether it's reliability whether it is availability whether it is scalability whether it is performance we want to give you all the elytis when you are running a kubernetes cluster at an AWS scale the second core tenet is about providing a native and upstream kubernetes experience one of the reasons among many why kubernetes is so popular is you can run it on your desktop you can run it on premise you can run it on your different cloud no there's a variety of ways you can run it and then in you feel yearning upstream kubernetes you are used you are learning a particular scale you are using cube coral s cube card or not cube CTL if you're using cube coral then you are applying your manifest to the cluster the quick and debate about cube cradle cube CT later at the AWS booth but you know how your manifest is designed you apply that manifest using cube coral that is exactly the experience we are bringing to kubernetes there is no forking there is no internal branch of kubernetes being maintained we want to bring your upstream experience straight up to a kubernetes cluster running as part of geek ass the third tenet now a lot of our customers they would like to leverage the integration with different AWS services not required because if you are running kubernetes you know in a variety of ways then you may want to run it as is but if you need integration with AWS services we will provide a seamless integration so think about services like cloud watch think about services like cloud trail private link VPC networking all of those integrations are going to be provided to you out of the box which allows you as a customer to not worry about those integrations and that's just only a start we would love to hear from our customers from each one of you so stop by the AWS booth or in or during the Q&A time shout out your request we would love to take that feedback what is it that you would like us to work upon and what are the other integrations with different AWS services that you would like to see because that is what is going to drive our roadmap and last but not the least this is the Kenneth that I am the most excited about working in the open source team so Brandon is part of the service team I am part of the open source team eks service team is going to actively contribute to the upstream experience as I said earlier there is no internal branch or Fork of the kuben is upstream up here so whatever patches fixes issues that gonna be applied or gonna be done in an upstream manner and we are actively hiring so this is my pitch if you want to talk if you want to work for Amazon in the kubernetes open source land come talk to us we would love to talk to you and come work for us so let's talk about it a little bit as an eks customer what do you really get well we have you get an e KS cluster how do you create an e KS cluster there is a AWS CLI by which you can create a cluster of course if your account is white listed then you also get a console through the console you can create a cluster as well so you literally create create console create cluster and it creates a cluster for you and in terms of cluster as we talked about the control plane is all fully managed then once the cluster is created you bring your worker nodes and other worker nodes are provided as an army and we'll talk about that in a second but then you apply a cloud formation template which then attaches your worker nodes or your data plane to the control plane so that kind of makes your entire eks cluster you can launch add-ons on the workers because the master is in our account the worker nodes are in your account so we provide an opinion to duay of running a kubernetes master but on the worker nodes you can launch any add-ons that are completely in your control so you can do what makes sense for you the color is I think it's dimming the last day of coop con I guess the color is tiring and so in the screen I am last but not the least the whole purpose of spinning of this cluster is to launch workloads which is exactly the applications that you have been building so eventually you can start launching workflow so the process is very simple in that sense let's take a look at it what do we do for you behind the scenes to make this happen though so you say create a cluster behind the scenes we create three now it's a highly available control plane so we spin up three masters for you we create a highly available HCD as well so and again it's an OP in it approach that we have taken the masters and the HCD are not co-located that goes back to Amazon's philosophy of reducing the blast radius so that you know there are six nodes running over there if a single node goes down is either the HDD or the API server controller scheduler kind of gets impacted not the whole thing that's an important part of it we also provide I am integration because if you're in the AWS LAN then you love that identity and access management using I am so you can actually intenta Kate with the cluster using a happier Authenticator that we'll talk about a little bit later we also do certificate management for you the worker nodes are registered in an auto scaling group and so are the masters so that in case a node goes down it can come back up again now that concept is a bit different from auto scaling your cluster which is again a very native upstream experience so if you have set up a cluster then you can use all the native kubernetes concepts that are available to you and last but not the least of course we set up a load balancer in front of your master so that all from your perspective what you get is an API server and you can talk to it directly now setting up and managing a kubernetes cluster it requires a bit of an effort over here ok so what we do is we give you a console you create a cluster and we give you an API server URL and then you can set up your highly available cluster the data plane across multiple availability zones you bring your workers in you get an e KS control plane and then you can use your standard cube cut' also like if you're using 110 you can use a 110 cube card all to talk to the EES cluster and start deploying your applications and if you're running your workloads the deployment be two replicas be two replicas set whatever it is all those are now registering in your worker node let's talk a little bit about the eks architecture so how do I provision eks worker nodes now we provide you a pre-configured army as part of that essentially you get a Amazon Linux two based army and on that army we have the right darker version the cube cuttle version the cube cubelet and all the components that you need figured for you now again that's an opiate way of running army or for the worker note but we understand the customers would like to run their own operating system they would like to package additional components a lot of our customers have told we would like to bundle additional components over there so the way we build those economies are using Packer scripts and those Packer scripts are gonna be GA as part of the eks GA itself so what that means is you can use the same Packer script to create your own army using your own operating system and bring those armies as part of the cloud formation and then attach them to the master so it kind of gives you full control and configuration and ability to customize the army that is running on the worker nodes awesome thanks Aaron so I apologize for flux kicking and my machines confused it says it's 557 a.m. Seattle time so now you won't have to stay up too late because your circadian rhythm is disrupted I know I'm not gonna do that but anyway so let's also talk about a couple of the open source projects that are that a company eks so the first is the hep toi M Authenticator this is an open source project that basically allows coop cuttle to also pass forward I am identity information with every coop cuddle call so we chose this approach because it is open source this is something you could set up on your own cluster if you wanted to but it's basically prepackaged with the eks cluster that you get when you create it just by default so what we wanted to do in this case is bundle I am with eks but not make you use I am for all the authorization so in this case I am is just for authentication and we actually tie into the native kubernetes are back for authorization so look let's look a little bit at how we do that so step one of this process is with coop cuddle using the hep Toa WS Authenticator it looks in the same credential chain where your AWS CLI does so if you have your database config file if you have environment variables if you have like an IM role set up because you're using it from an instance it basically looks for that identity whether it's active profile or like in the config file and passes it forward to the kubernetes api server on the back end we have a web hook essentially that verifies AWS identity just says yes or no this is a valid AWS user for this particular interview us account if you're approved if you're a valid AWS user the actual verification of what actions you can perform inside the cluster are validated against the rback system so for as an example by default the cluster creator gets system master privileges right so you have cluster admin privileges so you can of course do everything we need to do get pods you can describe pods you can control everything on the cluster but if you have an additional user in your account that tries to run the same commands they'll be rejected because they are not in the our back system so again that kubernetes action isn't passed back to coop cuddle and we'll take a look at that in just a little bit so an interesting component of this is that during the preview one of the things that was a little bit maybe controversial is we packaged our own coop cuddle we compiled it with the hep T Authenticator and it was a is basically just a superset of coop coop cuddle functionality and you could use it with all our kubernetes clusters but in order to use eks you had to use our custom coop cuddle because it contained that binary in it however as of the release of 110 with the support of pluggable authentication providers you can now just configure your coop cuddle to reference this external Authenticator binary and use the native upstream 110 coop cuddle and that's how my clusters configured today in the demo so we'll talk about that another open source component of eks which we are quite fond of is our CNI plugin when we were talking to customers early on about what should eks do for you there were a lot of requests to solve some of the problems with some of the existing networking approaches in kubernetes Kubb net for example kind of limited your cluster size some other options or maybe difficult to debug or difficult to get visibility into so with our scene I plug-in what we wanted to do is just bring native VPC networking to kubernetes pods and so the CNI plug-in itself is a couple of different components there's an eye Pam daemon inside of it which is responsible for basically calling out to the VPC saying give me IP addresses and there's the actual CNI plug-in itself which does all the wiring of the interfaces and you know dropping those into the pod namespaces this is the the end goal here is to just give you native EPC IP addresses so you get a flat routable network your pods and your instances all share the same VPC IP IP space this is open source on github there are quite a few contributors and there's a lot of work going on on this right now it's still alpha you can test it out in cops today and it is provided by default on eks clusters and this will be something that will consider GA when the service is also GA so let's look at architectural e how this works the CNI itself is deployed as a daemon set on every instance in your cluster and so what happens is when a workload is then deployed or scheduled to your instances the C&I plugin again makes that call to the VPC api's so it makes an ec2 associated address call and associated secondary IP addresses to your e'en eyes on your instances so after that happens the CNI plug-in itself actually wires in these secondary IPs it creates the virtual Ethernet pairs it puts them into the network namespace for the individual pods and the end result is you just have routable VPC IP addresses that allow you to hit your pods directly so there are some interesting caveats around this approach which we always like to talk about there's of course limitations on how many en eyes you can attach to instances right so if you're using like a t2 micro you can attach to en eyes and like 6 secondary IPS per and eye but as you get up to larger instances you might see where you have you know 15 en eyes and 30 or 40 secondary IPS Paree and i so generally the pod density that you can achieve on a given instance type tends to scale pretty well with larger instance types so it'd be great to take a look at that let us know what you think and test it out going back to some of arune's earlier messaging we are very intent about keeping eks upstream and vanilla kubernetes right i know when we announced this there were some questions around what are our intentions with this and we think by becoming a certified distribution of kubernetes you can see our conformance test results on the CN CF github page we really hope this communicates our intention here we intend to get certified for every version of kubernetes that we qualify on eks that was easier said than done early on though because there were some challenges to passing conformance tests for us and we think this is kind of one of the interesting problems we've solved here so kubernetes has this assumption generally that the masters and the workers run on the same network right and they have direct access to each other and this is because there's a bi-directional data path right so your workers reach out to the master to access the API and the Masters and vote connections into the worker nodes to support data flows like exact logs proxy all of these things treat the couplet as the server so in a situation where you're running PKS and you have your masters and workers in different VP C's this presents a unique challenge because we could ask you to open up security groups so we can reach into your instances so we can invoke that connection to your cubelet but that was kind of nasty feeling so we decided not to do that so what this looks like to enable this cross account networking and to support exec and logs is first of all for the API access from the workers to the masters we give you a network load balancer in front of your eks masters this is really nice because we have amongst other things or static IPS on the network load balancer so this solves some of the problems where if you've ever experienced like a classic load balancer that scaled up and the actual IP address behind your dns name changed and it messed with your certificates or something like that that's no longer a problem so API access traverses the static IPS but in situations where the masters need to reach out to the workers what we actually do is we create elastic network interfaces in your V PC but they're actually attached to our masters so if you look at the details of one of these Ian's eyes you'll see that there's a different owner for the attachment and a different owner for the actual eni object itself so this e and I essentially just gives us a route into your V PC and this is TLS encrypted so anytime you invoke executive IRS's this cross-county and i and it is a encrypted data channel so a little bit more about that we configure every kubernetes cluster every e KS cluster with the with with its as its own CI so we're using the built in kubernetes CI mechanism and we generate public and private keys upon cluster creation when couplets check into the cluster they issue a certificate signing request and we return a server cert that it installs and then nothing super fancy here but we do have a certificate rotation on that so you can ensure that the cert that the couplet is running is up-to-date and secure so that about wraps up our slides we're doing ok on time and I'm going to switch over to a demo real quick what I wanted to do just run a describe cluster and so one of the food King do you Oh what nope let's do this I'm really nice background I want to shoot this to show this how about that cool so this is my describe cluster it's important to know the eks api's themselves they're actually relatively minimal we don't want you to have to interact with our api's more than you would have to interact with KU cuddle right so you can keep your native kubernetes tooling and the additional overhead from eks is really really minimal we wanted to keep this a very thin ship so the things that are important to look at here there's some metadata on this cluster we can see that there are two subnets assigned to it and this means that there are subnets in my account where the U KS masters live in the you know the U KS teams accounts this is where it creates Ian's eyes and this is where it expects the worker nodes to actually be running so these subnets are the ones that we have access to in order to invoke connections into your worker nodes there's also a AR n for a role where this role gives us the ability to do things on your behalf in your account so for example when kubernetes needs to create a load balancer we use this role to assume those permissions and then take those actions other interesting things perhaps desired master version 1:9 we only specify to the minor version because we're automatically going to keep this on the latest patch version that's applicable to our environment to make sure that there's no outstanding CVS or security issues there are there's a security group assigned and this is a security group that controls ingress and egress from these control plane and is that we create and then finally the master endpoint this is perhaps the most critical detail right this is how you reach that kubernetes cluster you actually configure your coop cuttle to point to that so if we do a coop cuddle version something that is interesting to point out is that this is the upstream 110 coop coop cuddle client and it's actually pointed at the hefty o AWS Authenticator and I'll show you how that how that works this is backwards compatible so the server version here the KS cluster is running 1.9 and there's no problems with those working together as far as I know and so let's see if we do a coop cuddle config view the interesting part here is the username for my eks cluster references this these arguments here references external heavy Authenticator yes command so every time a coop cuddle command is executed this binary is executed as well almost every time what else was I gonna do let's see this so I have a cluster running do I have Wi-Fi there we go so I've got a few nodes checked into my cluster here we're running an actual eks produced ami this is the ami Arun was talking about this is re KS optimized ami we maintain this and we release this continuously as there are patches needed on the underlying OS or to the couplet or any of the basic alert docker Kubla any of the binaries that we packaged in that OS let's see what's running on my cluster here so so far I have a couple of Redis pods I have we've scoped running that I deployed via helm we can do it some logs from this particular Redis container we can also see that helm is working and helm works the same way that coupe kettle does where it references that external binary and we can see that we have a couple of things running via helm let's see if we can deploy data dog via home go have that running now goodly shrimp is an excellent name awesome so helm is working there so minimal demo things work this is kubernetes and if we go back to our slide I mean go back to my awesome background we can talk about uses as a prompt for our Q&A so we'll thing work on eks generally the answer is yes but if you have specific questions about these things we'd love to answer them for you in the few minutes that we have left if you have a question so you want to manage the certificate rotation for the kubernetes CA yeah today only we manage that for you so you know for you to provide certs maybe via something like ACM is a goal for us long term but there's not a quick solution to that right now yeah I agree there is a some difficulty in that yeah I think that's a brilliant question the question is you know if I'm running kubernetes cluster by myself on ec2 instance let's say using cops versus I'm running eks cluster what is it that I can I cannot do I think that is the thing where I'm pretty stoked because we are providing a hundred percent upstream experience where anything that you can do on a cluster created using cops or any upstream experience you should be able to do it on the cluster creatively using eks however in a cluster that you have created by yourself you have full control of the master so for example you can specify the API servers which is you can anything that needs to be done specifically on the master you can do that in our case because the master or the control plane is running in our account so you don't have access to the master those ec2 instances don't even show up in your account so you cannot SSH or do anything over there I think that's sort of the biggest reference that I would say that you need to think about and if you don't care about you know how the master is managing all those things workers are in your accounts if you run need to run a daemon set then you can easily run it in your worker nodes right something else to add to that for example is we don't have we don't have API feature or half of features enabled on the masters so for example if you want to take advantage of alpha features you would need to still run your own cluster for the time being right so slight differences there would you like us to who's a core maintainer here wants to talk about Windows support we will be supporting Windows as soon as it's something that is a beta feature for example there's been several customers asking us about Windows it's clear it's important to the community so I don't see why not the question is how are we managing provisioning I ops for ED CD nodes we are using so everything that is in E KS is built on AWS today and so HDD instances are just ec2 instances running provisioned dialects volumes behind them I think the question is kind of you know what happens if you start to run out of AI ops but we'll be monitoring for that and the answer is yes ah check out our virtual couplet pull requests as of a few days ago this is we've written a far gate plugin for virtual couplets so this is actually kind of an experimental approach without going into too much of a rabbit hole here you know the concept of removing the node from kubernetes is kind of complex so we are hoping to start getting the community to think more about this virtual couplet approach and we think it definitely has legs there's some collaboration between us and some other folks in this space we'd love to see how this works out for you and of course there's some work to do to make sure Fargate can support the full scope of kubernetes api is which it doesn't today but we think eventually we'll get there and this is probably the right way to go for us right now at least it's one option for us and just to add to that far gate we announced at arraignment last year and essentially forget is a way by which you can run your tasks which is the easiest language our native orchestration technology or a kubernetes part eventually that you'll be able to run in future without being able to worry about the cluster itself so you say from your perspective you just want to bring up parts back and say run it which is a deployment or whatever it is so at reinvent we announced the GA of Fargate supporting ECS which is our native orchestration technology and then we also announced our intent for Fargate plus eks that's going to be coming up in the future will I be able to specify which Flags the API server will use or cannot use and that's the approach that we have taken wherein is opinionated approach and we are constantly talking to our customers and what works and what doesn't work for them but at this point those flags are not exposed let me actually come back to you there was another hand with that yeah that is an incredibly aspirational question his question was when are we gonna have this GA so soon and if you want to sign up for the limited preview you can go to aws.amazon.com slash ETS 2018 the timeline for GI is 2018 now you can sign up we are constantly onboarding customers white listing them so yeah well I mean HCD managing EDD by itself is not for the faint of the hearts so at this point what we are looking at purely is to have HCD behind the scenes not be even exposed over there but that's a good customer feedback do talk to your account manager eventually that's what helps us decide what our road map is going to look like and what are some things that we can do let me actually yeah yeah whoa would you use marriage at CV itself or would you run your own queue Brandy's masters I want to go to the back of the room and see if there are any questions over there yeah well there yeah now the question is around cube - I am that's what they're using so with happier Authenticator that Brandon talked about we give you the ability to authenticate to your cluster cube - I am as much as the prominent it is you know that's sort of one of the primary ways how customers have been using to provide a sign and I am role to a pod and the capabilities to the pod now there are issues with cube - I am there are other approaches like Kay I am but we don't like some of those approaches the long-term wrote the way we are thinking is something like spiffy inspire that was talked about you know in the earlier keynotes that kind of provide a portable container identity so that's sort of the approach we are looking at it there is no strong recommendation from us and I think one of the cool root benefits is because this hundred-person upstream experience whatever is available in the upstream you can leverage all the innovation as part of that for eks as well so the question is pricing pricing will be available when the service is ga-oh so the question is do I have to use I am for authentication right so for other IDPs the solution would be to federate with I am because we're an AWS managed service I am is the de facto standard for us question here is it going to be possible to use calico as a network supplier that's actually a great question as part of the kubernetes uses CNI and we have our own AWS CNI plugin that is fully open source so that's the cni plugin that we're going to use although we are working with calico very closely and as part of you know I mean if this was like a three-hour session we could have gone into all the details of he cares but calico would be available in eks cluster to give you at least the firewall capability but you don't have the capability to choose the networking plug-in by itself right so the question is Ian eyes impose a resource limitation on your nodes so is auto-scaling aware of that in this case the scheduler we've written an admission controller to make sure that the scheduler is aware of how many on based on an instance type basis how many total IP addresses are available and this is just another resource constraint so the goal is to make sure that the scheduler if it tries to place a pot on a node that doesn't have sufficient IP address capacity will fail and then if you have something like this ei that's auto-scaling reactively should work based on that so the question is how's our support for online resizing of PVCs with EBS so that's an interesting question we haven't done anything unique in that space is just the same volume provision that's in the open source space we have talked quite a bit about the container storage interface this week and I know there's a bunch of efforts to pull the volume provisioner out of cork kubernetes so this seems like a nice opportunity for us to contribute some to that and try to address some of the issues in the volume provisioner in the future no CS I use it not even an alpha feature at this point in upstream itself so how it evolves over there we're definitely watching that space anything in the back yeah last person yeah so the question is you know the users can use custom ami for the worker notes as I said earlier the way we are creating those ami is is for example using Packer scripts so those Packer scripts are going to be open source as part of the GI so you can take exactly those Packer scripts so that you understand what components and how the army needs to be configured so you can bring your own operating system bring those armies and then essentially the way those worker nodes or those armies are connected to the master is using CloudFormation template so you will just alter the call formation template plug in your own armies and attach them to the control plane like that's actually a great question you know will he provide ability to provision you know other managed services three days back actually I'm glad you asked that question three days back we wrote a blog about open service broker so take a look at that effort that was started by twelve honoree and that allows you to provision AWS services as first-class kubernetes resources now at this point the blog talks about how you can use open service broker to provision sqs and other services but that's running on a cop's cluster that you manage but again that's an effort that we are closely watching we are actually contributing back to it so once eks goes GA if that is the feature that matters to you do let us know and we will start integrating that as part of ETA I think there's quite a few services that are available in the open service broker API already there's s 3 RDS I mean SQL was like six or eight of them right yeah yeah yeah the question was no dotto patching so that's not something we will have by GA you'll still kind of have the you know you bring your own node approach but we will of course release armies continuously so it could be as simple if your apps are all stateless right just updating the launch configuration for auto scaling group and rolling those out so there's also an SNS topic you can subscribe to to be alerted when there's new Ami's available so unfortunately we can keep going you know for as long as you want and I mean there's there's a lot of interest we would love to keep going thank you very much if you would like to know more about eks and the roadmap and what's coming up we would request you to stop by the AWS booth get your badge scan we'll put you on the list so that you can get the latest and the greatest updates about e-cash thank you so much everyone you so much [Applause]

Channel: CNCF [Cloud Native Computing Foundation]

Views: 10,183

Rating: 4.9512196 out of 5

Keywords:

Id: FmubBIw_wSs

Length: 37min 34sec (2254 seconds)

Published: Sun May 06 2018