Kubernetes: Changing the Way That we Think and Talk About Computing • Brian Dorsey • GOTO 2015

thank you all very very much for coming quick context what is this talk so at one level it's going to be a kubernetes overview so if you know all the kubernetes things you can probably find a better talk to watch that said I also want to talk about kind of context of where this kind of fits in and touch a little bit on when you have this collection of containers spread across machines where do you put your stuff where do you store your data things like that so let's get started so hi I'm Brian Dorsey you can catch me on the Internet I try to get my name everywhere turns out there's lots of Brian dorsey's in the world but I'm gonna try to get my name in all the places this is go to Khan so I thought we'd do a little choose-your-own-adventure and we'll how many people have started a container themselves ok we're skipping ok I won't put you through containers I'll buy a new way containers all this stuff one thing I think is really relevant is these I just want to kind of touch on containers are these isolated layers of the images that we run the containers from are these isolated collections of files and this is just a really really powerful thing you know that we're all very excited about but one of the things I think is is interesting is how it changes the timing of where you run into problems and so in some ways this pushes a lot of the integration problems up into your continuous integration your your build system and I think that's really powerful so you can kind of thinking think of it as early binding rather than late binding in your production machines so I think that's interesting so containers are awesome we're all in let's run lots of them everywhere and that's where kubernetes comes in so it's I think we're at this interesting point where we have something new that we can add to our our architectural decisions our problem-solving when you have a problem in trying to work through things so we've got so we've got physical computers we can go grab a bunch of computers and we can connect the network cables and do them we can use virtual computers virtual machines and the API is there look almost exactly like the virtual machines right you've got give me a computer put a disk in it connect these two networks together that sort of thing or we can jump all the way to platforms as a service stuff and give me your code and I'll promise to run it and scale it but you have to write it in a particular way to match you know Heroku style or App Engine style or you know whatever the platform's the service gives you and I'm super excited in this like in the last year or so there's many many options now for this this middle ground of when you want to treat a whole group of machines as one logical computer and you want to run your containers on lots of the machines treat them as one but you want to be working closer to the actual software you have so that's where at and that's what kubernetes is it's one of these container schedulers orchestrators these pieces so it is Greek for helmsman I'm going to probably do this wrong but I think the original Greeks is probably so one in the room is something more like kubernetes but it came out of the west coast of the United States so it's goober Nettie's sorry it runs and manages containers it it is inspired by Google running lots of containers in production and now that it's been an open source project for a year and a half it's it's grown into meeting the needs of a much wider group than just kind of Google but it's definitely inspired by patterns that we know work at Google you can run it in multiple different cloud environments you can run it on your own computers it supports of course docker but also you know rocket and you know the app see stuff and that sort of stuff 100% open source open source for reals if you will it's up on github the issues are there like you can go watch people argue over the code and the focus is managed applications not just machines so this is the slide everything at Google runs in containers there's probably little asterisk there somewhere but but basically everything that you've ever interacted with Google wise is running in a container somewhere in the data center so all Gmail search maps including if you use things from Google cloud platform so virtual machines those are also actually running in containers so it's containers kind of all the way down and we launched probably more than this now but over two billion containers per week that's carefully worded we launched two million containers a week and so you can think of kubernetes as a toolkit for helping you deal with the things you run into in distributed systems regularly so there's a bunch of stuff that comes up over and over and over how do the pieces find each other how do you log consistently how do you debug how you keep things running how do you figure out where the things should should run to be most efficient one of the big reasons to do something like a cluster wide scheduling system is to drive up efficiency and reduce costs and you know just better easier management of the things all this stuff and all of these pieces are built into kubernetes and help you run your applications so let's start with a cluster and this could be and and I want to kind of break this up because most of the talk I'm gonna spend on kind of using kubernetes but there's this the if you decide like if this talks inspire you like yes I'm gonna go play with it you'll get to this startup part and this can be challenging so kind of separate out the getting a cluster running from using kubernetes cluster so you need to get a cluster somehow you can use cloud resources you can use your own computers you will need an overlay network for the networking and you'll see why in a few moments you can run it on some raspberry PI's there's some great demos online of people running kubernetes on raspberries it runs in lots and lots of places and you know kind of gratuitous usual graph you've got a master it speaks rest HTTP - you and the command-line tools and the web UIs and then there is a process running on each of the nodes so the virtual machines or wherever you're running this that is the cubelet and it talks to docker or whatever your container technology you're using and make sure that everything is in the state that the cluster wants it to be in and just to be concrete so you can grab the slides later you essentially export you know an environment variable to say this is the provider I want to use and you can do the what I find kind of awkward curl SSH pipe to bash where you could save that and look at the file and decide to do it yourself and we have lots of recipes in fact arguably too many recipes for how to set up a cluster and we offer a hosted version as well that I'll talk about so what it looks like to actually use this so now we're switching over into the kind of using mode so if you have an image somewhere this is for example the standard nginx image that's up on docker hub you can in one line run two copies of that image somewhere in your cluster and not even need to care or know where those guys are so let's actually just go do that okay so we'll go to demo mode this is this is live hi so in the top here I have let's actually copy this out this will be exciting did you you know when you get into demos you're like I should have done that a different way that's me right now okay so I ran that command and immediately we could smile okay good it goes to pending and then running down below and in the bottom half I'm actually just running the command-line tool for Communities cube control you know watch loop so I'm actually just running it once a second three times once to show at the top which pods are running which replication controllers and which services and so we're going to see this over and over throughout the talk but basically at the top we've got containers that are running so I just started two containers there and those are good to go and so we can we can say actually wanted to have more of those so we're gonna scale this controller rough kitchen controller it's called my nginx and we want I don't know for those something like that so we talked to the the master it says oh hey four is more than two I actually only have two running let me start the new ones and get up and ready to go so that's a very quick kind of feel for what it looks like to actually use this and and these these containers are up and they're running the configuration of what's supposed to be the state there is now in the server in in the cluster and so if something happens to one of these containers and they crash they'll get restarted even if I'm asleep and it's constantly and we'll talk about this more but it's constantly checking today I've asked for four of them and how many are actually there and the like so this is a pod of whales and this is a pod of containers I guess so if you guys are here for the previous talk a pod is a collection of containers that are tightly bound we are we strongly believe that this is a very very valuable abstraction to add into the mix what this gives you this is the unit of scheduling in kubernetes this is the the atom one of these always gets scheduled on one machine and from the perspective of the containers inside the pod all of the other containers are on the same logical host so they can talk to local host networking they can do IPC they can see the same volumes the same disk all Yume's but other pods on the same host can't see the same things so this gives you some really nice properties for example if a whole bunch of your containers are your pods you listen on port 80 you can schedule multiple pods on the same physical host that all use port 80 because every pod gets its own IP address in the cluster so every pod has its own IP address you the pods can talk to each other directly over IPS if you want to and so that's this idea it's we have this little unit and if you missed the talk I highly recommend seeing the the previous talk on container patterns so check out his slides online so you can use this for any scenario where you have containers processes that need to work together directly they need to be talking to the same files to the same you know localhost sockets that sort of thing you know this example here is you know for example if you were going to have a you know a node.js app or a static file serving app and you wanted it to pull out the actual contents from somewhere else live as it was coming down so you have these containers that are the images are static but when they start up they pull down and kind of bootstrap the latest content so this would be an example of kind of separating those concerns the concerns of pulling out the data and serving the data so I'd be one example so the way these pods talk to each other every pod logically has its own has its own IP address within the cluster and they can speak directly to each other and we essentially you know in most environments do IP tables and router firewall you know router forwarding rules to make this all happen but you can also use flannel we've open V switch and if you're running in AWS or Google cloud the cloud environment can help you out and make this fast and like I said that means you don't have to worry about conflicting port numbers between your pods so how do you see so if we do this and they can all talk to each other we've constructed this this network this logical network within the cluster how do you get things make them available outside the cluster and you do it with expose and expose creates a service so in kubernetes so service is kind of an overloaded word but a kubernetes service is basically a load balancer that is aware of exactly where all the pods are for that service and knows how to route traffic specifically to them so if pods are these ephemeral pieces they come and go our containers we don't have anything long-lived in them we can just reschedule them put them on different computers keeping track of their IPs is a challenge but the service has a static IP address and a dns entry inside the cluster and so this is the primary mech for kind of service discovery between things and kubernetes so it looks something like this you you basically have a service definition and then it's automatically updated with where where the pieces are because it is part of the kubernetes cluster so that you have this combination of stable service endpoints and ephemeral pods coming and going the service can in fact even exist before there are any pods connected to it so you can have you know you won't ever get any response back but you can you can find out its IP address and connect to it and so clients are always in most cases talking to a service endpoint rather than the the actual pods directly and we stitch all of these things together in kubernetes with labels so labels are simply key value pairs that you can put on anything so we can put them on pods we can put them on services any of the parts of kubernetes you can put a label on them and labels are the only grouping mechanism in kubernetes so you basically make a query against labels to do things so for example if you wanted to draw a dashboard of how things were going on with your front ends you have your pods you can go ahead and set labels say ok this is a type front-end and we'll do a selector query and get all the data for the type people's front-end and you could also have a different graph somewhere that was like oh all the version twos we actually want to show on one graph and they might refer to the same pods or different ones but all the version two labels will come back and show and this is actually how the services are connected to the pods as well when the the service has a selector and when those labels match that's how it knows which pods to actually route traffic to so this you know very nicely addresses this need to connect things without having strict coupling of one person owns the other side or what have you or needs to know about it ahead of time and this is how replication controllers work as well so replication controllers are primary concept in kubernetes and it's it's this control loop that you set up and give it a template for how to create a pod and how many pods should be running and under the covers when I ran this command over here it actually created both the original two pods and this replication controller implicitly when I made that run command and the replication controller is what actually does this work of making sure there's always two or always four pods running to do the work so they basically live in a loop where they look at the real world look at the desired state and make whatever changes they need to make to make the real world look like that so this is a very nice decoupling of needs there and this is primarily for long-running processes so if you've got things like web front-ends you've got caching servers you know things that are meant to be always running log collectors things like that and it looks kind of something like this it it checks and says okay I'm how many do I actually have got three I'm supposed to have four let me start one and then just sits there and does this loop logically kind of forever so this is what gives you fewer pages at night or at least if you write software like I do that you know it sometimes you haven't tracked down all of the failures this will kind of keep it keep it going while you sleep so what if this looked like so I did this this one already but we can we can just we basically have a big stereo knob where we can tell the replication controller how many we want to have when I have to and it will say oh now we're supposed to have two and it will go down and this gives you this this direct control of how many of each of these are supposed to be running and in lines and boxes it looks something like this you've got the actual controller template add the actual labels on the pods and in this case we would have a service as well connected to that fact I think what I'm gonna do here is I'm gonna clean this up a little bit and we will play with some other fun stuff as this goes and then when I change it to four it notices that and adds the the four new pots so we can use this just as a straight scaling piece but we can also use it for some other fun patterns so I don't know if you heard of a canary pattern where like you've got a bunch of workers and then you get a new version of your app and you want to try it I mean you've tested it already right but you want to try it in production right and so you want to have some small percentage of your traffic go to the canary and you want to watch the logs really carefully and see how it works and then you want to go to rollback if something bad happened so you can use replication controllers to make it very easy to do a canary rollout so we have the same kind of setup we've got our primary version here ready to go and then you start version 2 with a replication controller if it's set to 1 and you're good to go and so if you are happy with that then you can kind of leave it be you can add more remove them it's very easy to pull that back out and you if you decide you like it you can do what's called a rolling up trick update where you add more of these so and this is such a useful thing we've actually built it into both the the front-end tool and now there's a deployments piece built into the master API so you can tell the master you want to roll between versions and that looks something like this and we're gonna do a more involved demo here so basically it goes you have two knobs and so you you end up turning up one and turning down the other bit by bit and eventually all of your traffic is being served by the new piece so let's let's look at that for real awesome okay in fact okay so we have I'm going to show you a guestbook app because this is you know what we have to do and so this is a PHP app at PHP we're actually using my sequel and some memcache and make make a whole app here and I'm gonna go ahead and start the I'm gonna show you these files in just a moment but one so the the first version the first command lines I was showing you we're basically interactive talking to the cluster right run-run me this image it just make me some stuff but in production that's not really what you want you want configuration checked in safe somewhere right so I'm gonna actually create some items from a front-end service and we'll get that and also the front-end replication controller so what these files look like so this is a Hamel doc I don't know is that visible up there that basically you know it's sorry it's very very simple the key part is this is our name we're going to expose port 80 from port 80 and we're going to the selector so the Cori we're going to use we're going to look for all the pods that are labeled front-end and this next demo app name so I've given these labels and set that up and create this service that was the first command I run and so we have a new service down here that's called front-end one extra little fun bit in here is I said taifa load balancer and this kicks off and tells Cooper Nettie's that we want this to be an external outside of the cluster load balancer and since I'm running this on container engine in Google cloud this actually means we're gonna configure the Google cloud load balancer which is basically Google's data center level load balancers which is super fun so I can be on the same load balancer that runs Gmail and YouTube and actually get that to distribute the traffic to my VMs and then the other file we ran here is the replication controller and so this contains how many replicas we want to run of our front-end that's here and the selector so that we know which pods are part of our group it's actually the same as the service and the template for making a new pod when the time comes so whenever it needs to make a new pod for whatever reason it will use this template to create it and the key bit right here is we're pointing at a docker image in the Google container registry and it's it's in this project named next demo 42 the image is named PHP guestbook and the version so : then it's tagged docker wise v1 and that's this name it uses port 80 so we've done all that bit and down here we can see the front-end load balancer got set up with an IP so we should be able to hit this IP address and this is the dangerous part of the thing if people can read the small bit down there hello I can say even hello lid if I can spell it correctly yay and you can hit that same IP address and put script tags in there and all kinds of fun stuff and see how that that turns out so we've got this and then we can say a cube CTL so actually let me I'm gonna go ahead I've got this little extension that will refresh the page every second so that's gonna start flickering in the background and updating as other people are or adding things and hey looks like emoji works and so we can actually scale the replication controller named front-end and set replicas is equal to 6 for example and we can start new replicas get them in serving traffic and and everybody in the audience and myself who's sitting here refresh the site just City it stays up and keeps working we can drop it back down to three have fun with the whole deal so that's neat but as soon as you get something working there's always a feature request right and this is kind of a dull site we want to make it way better right so I got the request to make the site better and my design skills are perhaps lacking but I've got a new version of the site that's come out of my continuous integration system and in its pushed up into the container registry and it's called green so we've got this guy here and so what I'm gonna do is try to remember the exact command line which I won't so we're going to we're gonna do a little trickery so I'm gonna run cube control rolling update the front end the update period here which is cut off the screen I'm setting to one second which is not a normal thing to do in production so the update period is how long to wait between steps here set to one second and then I'm gonna rolling upgrade to this new image called green and what's happening here now is it's managing the two dials for us so I've got the site up and running and I'm slowly changing the traffic over to the new version of the app while while it's running and it looks like I've messed up the view there let's see so we can see the the in progress controller that it made has a big long name and the the new pods have this name just kind of make sure that they've got stuff going there and we've got all kinds of hacking going on on the site fantastic some some of the things worked and some of them don't and we're basically just turning these these knobs one up and one down as we go and then when the last one goes away we should by the time the last one goes away we should see the new version when we need actually switches depends on you know kind of which the puzzle is connected to it hey we got the new version so that's amazing all right thank you so the thing that I find really awesome about this besides my incredible design skills is that so we you know we migrate our live site up right but if we decide we don't like this anymore we can actually migrate back using the exact same process so since we've got this isolated filesystem these containers under the covers you know forward and back is actually exactly the same process so if you decide later that you don't like you know you don't like that you can actually switch this back to version one and you know we can rolling upgrade that back and I won't make you like watch that but we can come back to it later and hopefully you guys won't completely destroy my my account there so also just recently added and this is still in beta we have auto scaling and auto scaling right now is based on the CPU usage of the pods so you can basically set up a scale target which says you know if too much too many if the pods are over 50% we need to increase the number of pods that are running in our cluster and so what we do is we're actually using C advisor I don't know if if you're familiar with that to like get all the metrics out of the running containers pushing that up through this app called heap ster which aggregates all of these and then we actually know relatively close to real-time exactly what's going on with all the containers across the whole cluster and so we can set rules basically that says hey if you know we have pods that are you know over fifty percent which our rule says we need more pods and then instead of me turning the dial we have a robot turn the dial and you know so we can actually have the scale target drive how many pods are actually running in our cluster so the currently CPU is the is the one that's there we're definitely working on more more options for you know the actual thresholds to do that and including you know pushing your own metrics through hipster and using that to drive the auto gatling so that's their and all kinds of other fun stuff there so you guys have probably noticed that almost every demo of containers is is using some like very simple like web server or stateless app or other things like and most of these demos were as well technically my my sequel is actually backed by something that lives but I'm going to blow it away but but almost all the demos are are stateless right and and I have questions about this and also I think as an industry we're trying to kind of resolve this right it's it's clear that there's these really funny they're great advantages to keeping all the state out of your containers you can then treat them like these disposable things but you build apps to keep track of things right so at some point we've gotta like manage this data and we really care about it that said I think this is I'd love to hear anybody has a different opinion I think in a cluster of all these ephemeral containers we have to keep the state out of the container or at least out of any one container at a bare minimum and and probably just out of all the containers so if that's the case what do we do here and I'd like to share three of the patterns that I've seen so far and some examples of people using that and I'd love to hear if you've run into anything that doesn't fit into one of these patterns that would be very interesting but before we get to that docker has volumes and kubernetes has volumes as well and they're kind of it's kind of a superset of the things you can do in docker so when you've got this pod it has some local disk available if you configure it that way so you can do things like a temporary empty directory you know that just you know lives and dies with the pod so you can kind of have scratch space you can if you want to you can break out and put files on your host machines it's up to you then to figure out which machine those are on is you get scheduled on different nodes or use some of the tools in Cooper nodes to make sure you always get scheduled on the same node and then you lose flexibility there you can also use you know NFS if you have to create a really reliable and fs system or you can plug into I scuzzy so you can do that if you've got a nice cozy system or if you if you're running on a cloud provider that has a block storage service you can use that so we have plugins for EBS and for our Google cloud block store so we need we probably need some storage so we're going to come back that's why I hit this here so we're gonna come back to these guys so I think the three big options for how to store state in a system that's made up of all these ephemeral containers is just you're already putting it somewhere you have a database team probably perhaps keep doing what you're already doing right like run your database the way you know how to do it and it's safe and connect to it from inside the cluster so one option is just straight-up it's outside the cluster you run the database you know as you know as yeah we're not gonna make the job anyway so you just run the database the way you've been doing it so that's one option another option is that you take something that really wants to run on one computer all the time and you could have swapped the storage out from underneath it so that by the time it comes up wherever it comes up it thinks it's on the same machine all the time and so this is where having a block store it set up in your environment something like Ceph or you know the the block stores in your cloud provider are really really handy because kubernetes knows how to take a block store and connect it to the hosts and plumb it up through to the container through the pods before it starts the pods so you can actually and that's the way this my sequel was running you can basically set up a volume that is always available to one pod at a time and and make that work so you can do that and you know there may be some latency things but it works in a wide variety of cases and it's a good way to adapt systems that already exist and then the third option is to do something that's it's very aware of this world of things coming and going that's cluster native so I'm thinking of things like Cassandra react it's it's very new but cockroach DB things of this nature which are already distributed systems they're aware of the fact that nodes come and go in a kubernetes environment the nodes are going to come and go faster than they're used to so there's there may be some extra things you want to do and we've got an ongoing example that we're working on with data stacks in the examples folder of kubernetes on how to do some of that plumbing but you can use databases that are aware of the fact that they're on multiple machines so I think that's your three options so basically like keep it out of the cluster adapt it to the cluster or use something that's aware of being in clusters another option here in the kind of cluster native space is called Vitesse I don't know if you guys have heard of this it's an open source project from Google it's the system that's been serving all the YouTube database traffic on my sequel since 2011 and so this does all the hard work of charting and setting up replication and things for my sequel and they have kubernetes configurations included with that so that may be something to look into there's another kind of state so if we're going to keep all the configuration out of our of our containers right how do we get them connected to our secure services to our databases to our api's we need some way of getting the the actual stuff in there and one way of doing it is environment variables kubernetes has decided to go with something that we think is a little bit more secure environment variables are very introspective all on purpose so we have this concept of a named secret and basically the cluster admin can configure named secrets and the if you have permission the the the pod templates so the replication controllers are the pod creations you basically say I want to mix this named secret into my file system in this location so that I I know that in you know slash wherever is my password every time I start out and it's up to the cluster environment you know the cluster admin to configure what passwords are actually there in that spot so this lives all in tempo fess it never hits a disk on any of the nodes if you turn the machines off it all goes away but this is a mechanism for getting that configuration of secret information in there we also have environment variables and labels and you know that kind of stuff that you can use for other config so kubernetes was open sourced in June 2014 so we're approaching a year and a half old it hit 100 this year we hit one one last month and we have a hosted version of this that's called container engine as part of a Google cloud platform it's already in use by several platforms of service folks so the Red Hat openshift folks actually tore out there kind of orchestration layer and put kubernetes in under the out of there they've been extremely active contributors to the project also Dias and Stratus and I think there's another that I'm forgetting that are that are using kubernetes building kind of a more user friendly or sometimes more adjusted to certain environments platforms and then there's multiple distributions so if you want to run this on your own machines core OS tectonics you're gonna hear next from Asus folks definitely stick around for that on how kubernetes and maize O's fit together I'm working towards a 1/2 release so I just wanted to quickly show if so separating out this like setting up a cluster from using the cluster I think it's kind of hard to tell whether you care enough to set one up in like your environment until you've used it a bit so I strongly recommend using container engine Google container engine to set up a cluster to play with wherever you plan on running it in long term it's definitely definitely the easiest way to just get a cluster of and by easy I mean you hit this button you give it a name and pick where you want to run it so we probably want to run it in Europe pick the size machines how many of those machines and hit the button and three four minutes later you have a cluster that's ready to go this is my recommendation for how to play with kubernetes wherever you plan on running it long-term communities is completely open source for realz please if this is something interesting to you take a look at it fill issues if you run into any trouble when you get started I'm trying to play with it if you've got use cases where it doesn't work for you for whatever reason we'd love to hear about that the reason kubernetes is in the place it is right now is because of the open source community the that's grown around it it's been amazing so please please join us there and I'd like you to think about like the applications you have the things you're doing right now and where they kind of fit in the abstractions you know do they run on physical machines do they run on virtual machines is it time for a cluster of containers using paths or different parts of the app actually be a more natural fit in different parts of it you know a really common pattern we see for Google cloud customers is to put part of the app in App Engine just that just as HP requests boom boom boom boom boom and then another big part is not going to be off on virtual machines so think about your applications and how how they fit and I really think you have a new kind of architectural choice a new option at least let's say option to think about so thank you all very very much that's another talk and we have questions excellent first questions about replica controllers that actually two questions one is a replica controllers like a single point of failure and why is the replica controller separate from the scheduler or is it actually not an actual separate component okay so let's do the first one of there is a replication controller single point of failure the replication controller is a logical it's a logical thing it's it's part of the kubernetes cluster it's actually implemented at as a single process on the master so there is a master that is a single point of failure that's implementing the replication controller that said you know the the whole back-end for that is at CD so there's a strong deterministic consensus protocol behind that and there are definitely people already running multiple masters you know in standby mode so you can have you know multiple processes ready to do the work on different physical machines there also if something does go wrong since we have the the kind of detailed management pushed out to all of the nodes if you lose the master for some period of time the current configuration just stays all the networking keeps working the services stay working if individual containers die they get restarted the system stays going it's just that you can't change it until you have a master to talk to there so the second question was the controller separate from the scheduler but I presume schedule is running in the masters yes so it is it is separate so and very much on purpose because it's pluggable so we have new controllers now so for example we have a jobs controller that is specifically designed to help you do batch workloads so if you want to make sure that a thousand copies of this container run even if some of them fail like to don't care which machines they're on the batch job controller is there for you so the replication controller what it does is it just it's actually a process that's just talking to the kubernetes api so you if you want to make the d special phase of the moon controller that does more process pause when the moons in a certain phase or what have you you can totally do that using the same API is that replication controller does so yes scheduler is separate and when you request to kubernetes give me a new pod then the scheduler kicks in and decides where to put that and it's mostly kind of spreading the work out right now but as we get more metrics plugged in through the heap stir work they're actively working on doing this kind of multi-dimensional tetris game of finding out where the best place to put a pod is for memory disk you know CPU all the things next question is related to the presentation itself the source code of the application and the command line calls that you did not a part of the slides as well or they are in the slides and I will in fact so I will do a live pointer but also kind of add them there as well so this is the kubernetes github repo eleven thousand stars turning without some good minutes and there's an example folder here the the example that I did was the guestbook example okay so you can you can run this guestbook example yourself I think that this example might be slightly newer than the exact version that I ran and and the the amazing update to the design is not here okay next question someone asks how much manpower how much time is actually needed to get really up to speed from zero to have kind of enough kubernetes understanding to run as a set up and run a successful computer setup so how much time is do you need to actually kind of get started with kubernetes I think there's a two parts of that question I think the more complicated part is actually getting the experience to kind of run a cluster in production well there's there's lots of challenges that are really outside of kubernetes there around networks and you know keeping everything happy there for Cooper Nettie's itself for like building an application that's going to run in kubernetes I think if you spent several days with the concepts and work through some of the examples you would be in a really good place to begin porting you know one of your locations in fact probably within the first day definitely looks look at the different concepts the things you really need I talked about in this talk you need pods you need replication controllers and you need services and and you need to have some process that you've made a docker image of the Alyssa's not a port or something you know that does something outside of the container if you have those things you can probably actually already go and look at some of the examples and make a config that runs them or literally just run the cube control run command that I did that creates the configs for you the the very first command it's and the slides question to Canaries when you do Canaries how do you make sure that the user gets back the same version of the service to the same label of the service right so we're doing so it depends on the load balancer you're using to get into the kubernetes cluster and then kubernetes itself does some some work there so it's a little it's a little bit complex it's not so if you want to make sure let's say how didhow let's say how we do this if you're just running a default setup on container engine on google cloud the google cloud load balancer we use client IP affinity and that will get you to a specific node which then also will kind of use some affinity to get you to the same pod but it depending on the environment you're running in that may not be the case so if you need to make sure for your app that you always hit the version two and you want to do that it may make sense to actually add two services so you add a second service for that and give it a different like subdomain name or what have you and then you can ensure that the traffic hits there well two configuration questions one is is there a declarative way to control complication controllers similar to an Box CFG update Oh the other question which is all the configuration dependent is how does the fixed information about number of replicas in the in the configuration in Yama file relate to D auto-scaling but I think that just entries for auto-scaling yeah kind of the same same thing here so this is one version of the declarative representation of a replication controller and I'll can't really address the board config aspect of that question but this is defining how a replication controller should be when it's pushed up into a Cabrini's cluster as far as how that relates to the actual value of for example the number of replicas especially if you've got if I've changed them later or you're using auto scaling this version you would have checked in somewhere and you push it out when you create the the initial replication controller and then over time the system will adjust you know to match the actual load that's there so if you if you wanted to feed that back somehow into your actual kind of default configs you'd need to add to pull this config back down and and kind of check it back in and if it this is a yeah mel document we're looking at here the actual API speaks Jason you can make Q control spit out the raw Jason or yeah Mel converted version of the document that comes back from the rest API so if you want to pull down what the server knows of the current state and get that out as one of these documents that you could check in you can do that from cube control that's good to know questions around security so one question I have actually is can you Limit visibility of the shared secrets to a single container within a pot or is it obvious a whole pot because otherwise you can use like some evil injection thing to it is actually file sisterly so we're we're working to improve the granularity of security for sure right now the security in kubernetes is very kind of cluster level there's also a concept of game space so like all the stuff I did was in this default namespace you can have different slices of the same cluster that are named spaced apart from each other and they have their own services and replication controllers and pods and all that sort of stuff and there's some security stuff there if you need the very granular security today I definitely recommend taking a look at what it some of the Red Hat Enterprise folks are doing with security they've they've mixed the security in with their existing enterprise you know stuff and and that should be of interest I think okay last question is about at the end you showed how to spin up the cluster for such a setup and the question is all the instances had the same size example you know but I think that might be controllable which are all the instances the same size so yes when you when you create it by default in fact unless you go out of your way to kind of create your own instances manually and join them into a cluster all of the scripts that create you know kubernetes clusters will create homogenous clusters so you've got a couple options we actually think you know almost all the cases that's really what you want but if you have physical machines that are already different and you need to use them differently you can actually label them you can label the actual nodes and you can create constraints on the deployment Docs that you make to say that you know for example I need to be on a node that has a good GPU and so you can label the nodes with GPUs and then you could actually label your your pod templates saying you know requires the GPU it's not strictly answering that question but but basically we normally do homogeneous machines and it helps with scheduling a lot that will if that's really important to you like definitely talk to me afterwards and we'll try to get that into the mix yeah and the question is related to that do you have an example of a declarative deployment script they define kind of which kind of machines you want to run your cluster on ah yeah so we have scripts for creating cluster and there you know are some yeah I should probably chat with that person afterwards like the actual cluster set up it's kind of a different thing for questions oh no this I think you answered all the other ones cool don't forget to rate to talk thank you all very very much I think also questions thank you Brian was really impressive talk and

Channel: GOTO Conferences

Views: 21,102

Keywords: GOTO, GOTOcon, GOTO Conference, GOTO (Software Conference), Brian Dorsey, GOTO Berlin, GOTOber, Videos for Developers, Kubernetes, Computer Science, Programming Languages, Docker, CoreOS, Mesos, Software Development, Software Engineering

Id: DGlQgNmobuc

Length: 50min 40sec (3040 seconds)

Published: Thu Mar 10 2016