Developing Locally with Kubernetes [I] - Ryan Jarvinen, Independent

okay hello welcome come on in so we got a couple people still coming through the door it's just about 2:45 so I'm gonna get move in with my talk about local development using kubernetes yeah feel free there's a good number of seeds down here in the front right feel free to point people this direction if they come in late all right so accelerating local development with kubernetes I am Ryan Jay you can find me online on Twitter github and IRC this is usually what my avatar looks like I am a developer advocate at Red Hat and when I first submitted this talk there were terrible things happening and I believe there still are in the way that we describe and by we I might mean you how we all describe kubernetes especially when we're talking to developers right how many people in this room feel like they are primarily a developer more than an ops person awesome all right you hopefully have and how many of you here are have used kubernetes for less than six months excellent all right well this is a very introductory kind of talk I'm going to try to use just the upstream tooling and show you as much as I can from there and then we'll talk about what additional tooling you might want to layer on as well and so here's my let's go with so why are we so bad at pitching kubernetes I think it's because it's primarily you know it was a tool that was kind of built by and for site reliability engineers or operations folks or really folks that are looking at use cases that are about system reliability high availability you know there's there's a lot in here that people kind of package into their description of what is kubernetes and it rarely has any type of value or any type of resonance with developers and often it's loaded with boatloads of terminology that is just more insult to injury you know there's there's load balancing scaling delivery automation the these are all things that I could care less about when I'm doing local development usually right so this has kind of been a problem for the kubernetes community and they've just been focusing on their core competencies which they should they've been keeping keeping their eye on on making things faster better more performant yet at the same time if you ask kubernetes core maintainer what is an app they will very likely struggle to come up with a clear definition because there isn't a clear upstream definition in the kubernetes community of what an app really is because a lot of it they'll say well it depends are you talking about workloads webpart you know there's there's so much in there I mean people started off with docker was going to be what solved it all we'll containerize all the things and then as soon as people realized we need to scale these decompose these monolithic applications reflect what's happening in the micro services community and allow these pieces to be scaled independently then you see this model start to fall down and people go from docker images to maybe swarm files to kubernetes spec files maybe on two charts eventually there's also a proposal around using label selectors where you could do a kubernetes command line career get all - L - match on label selectors and then look for this label with whatever your app name is and whatever the response from the API is that's that's what your app is right but that's not a very clear standards compliant definition that we can that's developer friendly in any way right especially if they don't already have you know a lot of kubernetes access to play with so there's a proposal I've linked to around label recommendations I think if you're a app developer definitely look into whether this proposal fits the bill for you does it meet your needs does it make sense as far as how we model apps I think there's still a lot more work to do in helping define this term but what you can do in the meantime is start talking about kubernetes in in a way that hopefully makes a little bit more sense to developers and I'll try to show that today so this is where I want to get us I don't know if we can get there today but what I want everyone to confidently say eventually is why kubernetes development velocity that's why right but that's not the agenda of today you know the conference so I'm going to do a quick kind of a case study to help illustrate this kind of conversation of how I would pitch kubernetes locally to my team and it to guide us through this with the kind of a since we're in Austin kind of a music based analogy if you will so this is a theoretical company enterprise records incorporated here's the picture of the band I'll point out a couple folks we have here up at the mic is our architect or perhaps a lead developer but someone who is definitely running the show right he's our frontman just behind him we've got Will Ferrell that's our our front-end web developer right and the product team is over on the far end and I invest in a nice jacket so generally operations teams will come in and and they'll realize that you know the the dev teams been working non-stop they've been accumulating technical debt they want to help increase your overall velocity as a company but they know they need to fix some architectural things in the process right and so you know you may come home from Kubb con and start blathering to everyone in your company about hey look at staple sets and look at this and throwing out term after term and if you're not careful people are gonna be like what is this guy talking about why is he obsessed with things that have nothing to do with what we're responsible for we need to ship we need to ship product we need to ship features and we need to ship them faster than our competitors right we're not shipping kubernetes we're not shipping developer tools go find the best in the industry use those and let's get back to business is generally the conversation and the inertia that you need to overcome they're always going to want more and the web team is going to honestly want to know how do I maintain my velocity while using containers it's a fair question and the kubernetes community needs to have a real answer so I've tried to package so don't screw it up for us Jeanne we need to get there to container town we need to roll out kubernetes this is my approach to breaking through to the rest of your team minimal onboarding I think your number one message should be getting started as easy we'll cover getting started briefly share what you need to know what you do know and model your i/o as application developers one of the most critical things you're gonna have to be aware of when you're developing container based solutions is if you have random writes going to some party that you're caching image uploads or you need to model all IO and mark certain folders as being read rights certain folders as being read-only we use read-only whenever possible and if you expect that data to stay around you're gonna want to use a volume or some kind of persistence mechanism so make sure as developers that is clearly in your scope to know where your data is as you're operating on it and then the third one is really kind of evaluate what's available as far as toolchain goes and and choose what's the what's the best of the day so here's the easy part mini coop start that's it it's one command well there's one other curl you could do a to commands if you want to do a curl and then pipe it into your user local bin you know you could do a one-line install and then a one-line boot up essentially but mini coop start is the easiest way to get going I'm going to open up a terminal and just paste in just so you could see how easy it might be this is using a virtualization mechanism to spin up a VM they also have a new way you could do - - VM provider equals none and then it'll try to skip the virtualization and spin up kubernetes using only containers you need a docker engine or something for that but looks like I'm up and booted I didn't need to sign up I didn't need to wait for the ops team to provision any servers I'm now unblocked right so that's that's the this is the easy part make sure that everyone knows not having a kubernetes environment ready is not an excuse stagings down ops isn't ready no everyone gets a kubernetes you all look under your seats you all get a kubernetes mini Kubb is yeah the best way to get started on that today yeah any coop start I have a link to the official mini coop Doc's if you want setup help I also have a set of slides I put together I do a lot of presentations and so I've put together a series of getting started with kubernetes that's all about half an hour to one hour chunks so if you end up at this bitly k8s mini coop i've got about four different half an hour long chunks of what one is just the set up next one is getting familiar with coop CTL so if you want to learn more I've got a whole series on that share what you know is the next big tip and like I said model your i/o as developers that's going to be a major part of what you're gonna need to be aware of here's one way I like to share what I'm working on so if I'm starting from scratch and all I have is a repo and I've maybe maybe a docker file I'm not gonna start you guys entirely like you don't know what containers are I'm assuming you you're here at a kubernetes conference I'm just gonna assume you have a docker file or a docker image or some kind of container image that you could coop CTL run so if I do this coop CTL run command I can name my application if you will load this image and while I'm at it I can also add a load balancer that is compatible with many Kubb and then add this dry run flag this is kind of a complicated example but dry run allows me to do something particularly special that allows me to share what I know with other folks so normally if I ran this command without the dry run it would immediately go provision the new container using a deployment and it would set up a service or a load balancer so I can contact this this web service if I add the dry run flag what what it'll do instead is prep up all the JSON or gamal depending on my - OH output prep up all this data and write it all to standard out instead of to the API this allows me to quickly generate manifests or spec files that I can share with my team so if you're having trouble getting started or if you want to generate whatever your starting point is and then share that with another web dev try using the dry run flag as a way to generate your specs I'll run an example of this so we could see what the output is oh let's see no outputs oh oh I piped it all to this metrics file here we go metrics review so it looks like the first data element I have is a service and then I have a deployment so this is now something that I can take oops I can take that run Coop's ETL create - F on that file or hand it to someone else and allow them to run Coop's ETL create - F and they have their local development now staged up with whatever I wanted to deliver right they have something that they are now at their I was able to deploy hello world I haven't iterated on hello world yet but I've I've deployed it locally and that's one significant step in your onboarding process for new developers next they're going to want to test that mini koobs service allows you to open up that new deployed resource in your browser it looks like this is still downloading the container but it should pop open in the browser as soon as that downloads completed next up I have example of how I would use this I oh modeling or an aspect of i/o modeling to enable real-time iteration and development against this container I've provisioned so I'm gonna make a local clone of this repo so I have something to work with here and if the y5 holds up let's see what we'll be doing is checking out a copy of this repo well then using the mini cube mount command to mount the repo folder inside the virtual machine right so now it's essentially exposed to the node or the host machine inside the mini coop environment and I'm gonna expose that inside the VM on var dub dub dub HTML looks like I've got a little bit of a weight here see if there's a network cable they told me there would be Ethernet okay here's our deployment has finished so it looks like part of our download happened and hopefully that cleared up bandwidth for the rest of my network connection but this is what I was deploying it's a some kubernetes contributor graphs that I was kind of tweaking on some some data for contributor graphs and this was an example project that I wanted to iterate on so it looks like we've created the new open the browser session we've got our local clone of the repo let me get out of full screen mode okay oops too far back okay here we go dry run modeling your eye oh we did the clone now I can mount this folder inside the VM and I'll need to leave this window open to keep that connection available to mini cube next I'm gonna make a copy of metrics review and I'm gonna modify it this is something that you will have to do by hand unfortunately but now that you have a starting point it's easier to go in and look up what is a pod what is a service and then look up the actual spec find out what fields and what values you might want to tweak and then go based on the spec now that you have a valid starting point so this I actually have a completed copy of all of these modifications inside the repo at this stored in this file metrics dev - yamo but all it does is it takes the earlier file that we use dry run and adds a volume mount into the pod spec so I'll take a quick look at that from the command line it's metrics dev so we could see looks like there's some volume specifications and it's going to do a host path mount from this folder on the node host into this folder inside the container and if you look at the docker file for this project the entire repo is basically sitting inside this folder so I'm going to mount a raw repo inside a running container that's basically serving this repo a static web so it's it's a trivial example but what you ought to see is that I'll have real-time access to editing the HTML and CSS and then I could just reload with my browser and use a full container based tool chain so let's go back now that I have the resulting file I can many Kubb create on our metrics dev example and then if I really wanted to get fancy let's let's take a look at see if that gets us anything let's go into mini cube service metrics dev ooh see what happened not running yet okay I think it's still downloading or something's happened let's see 404 not found you know what I think it's having a problem with the mount and I appear to be having a failed demo but what this should do is basically give you the same view as this page the only difference and I'll fix the demo sorry about this the only difference is I would be able to flip to the command line edit the index file look for my title here contributor metrics and then change it to kubernetes contributor metrics something like this and then as soon as I reload the page on that dev version I should immediately see the changes reflected in the browser that's basically the steps that you need to know sorry I'm having this 404 error on my metrics dev but basically what we've done in the last five or so minutes is set up one deployment to be used for real-time dev and another deployment that I can actually roll deployments to and then I can do real-time development testing and then once I'm happy with those commits and I'm ready to ship that then I can do a docker build still haven't committed yet but do a docker build and then do a I can use this mini Kubb docker in command what that will allow me to do is it'll basically bind to the docker demon inside the VM and then when I run docker build the result of the build goes direct into the VM and I don't have to go out to docker hub and back down to my machine so I could be developing fully containerized fully offline in airplane mode you know while I'm flying over the ocean and potentially have a full you know multi service application deployed developed complete with kubernetes all on my laptop this is going to boost your memory expectations of course but if you're developing with small solutions or scaled-down solutions this gives you a much more production like representation of what you're going to be working on and hopefully less surprises when you do go to promote your code to production any questions about that that's that's the easy part any questions on the first half I'll take a question quick yeah so the question was what are the virtualization options available for many Kubb there's a pretty solid list there's a cait libvirt KVM one I think you need one extra download to get that to work there's also support for using the Apple native virtualization a VMware fusion KVM X hive yeah good solid list yeah and all of that's available on the mini cube Docs yeah Tom oh okay yeah this there are you will see sometimes mixed results if you're using different virtualization providers sometimes mounting external files into the VM works great with one with VirtualBox but not with kbm or vice-versa right so some of these you may have to do some experimentation figure out what's the best fit for your dividing on linux here but if you're on all Apple you know maybe you want to use x5 as long as this mount operation is working appropriately another thing that's going to be a big boost in the future I'm having problems with this external mount thing in the future with with those latest 1.9 release I know there's support for using local disk for persistent volume claims and so hopefully you could just lay out a larger size virtual machine disk and then use that disk for persistent volume claims and other kind of persistence mechanisms that might should really help with some of the modeling you're doing with kubernetes so now we get to the hard part so you know that in some ways you know this is a popular quote in some ways the future is already here but man it's sure isn't evenly distributed you're going to have different teams and different areas of experience different requirements from different teams and different people will be a different states of adoption some of them fully containerized some of them with a lot of legacy workloads that they are stuck with so this is a kind of a typical adoption path that I've seen people generally kind of start with playing with docker eventually they're going to need to start modeling their i/o creating volumes persistence volumes mapping those into the containers hopefully they pick up mini coop at some point and then easy way to model kubernetes abstractions right on their laptop and having that as a just a modeling language on your laptop I find incredibly useful next after that you could definitely if you're looking to share what you know look into charts the the kubernetes charts are the official way of packaging up these solutions they don't currently they're not really labeled as the the app definition but they are the packaging mechanism for kubernetes if I'm not sure you know that's kind of how they're described open ship templates is another option or you can use your web developers I'm sure you know how to do templating you could use Jinja templates you could use really any kind of template Emmure for width and hand roll your own specs once you know what you're dealing with right so maybe start with dry run generate a couple manifests and then figure out which values you need to modify frequently and develop workflows for assisting with that whether it's hand-rolled manifest charts or you know eventually people are going to be looking at you know this new Service Catalog or even leveraging a full past solution and really the more you add the closer you get to pass here so so I have a from here I've got a variety of solutions that I wanted to give the audience an opportunity but my theme for this is keep it simple make it try try to describe these in a way that is easy for developers to understand so from the audience who has heard of draft here Dre okay anyone want to offer a description of it hopefully without using the word container or kubernetes anyone want to offer something no at Tom go for it actually I got a mic as well Tom yeah I'm calling on ya Joe I'm sorry Joe [Laughter] okay so in one sense I guess I would say that draft isn't and this is almost direct quote from the official documentation and event driven scripting system for kubernetes Oh close close that is close anyone else have a guess what I think of draft draft is really kind of you're so event-driven scripting system that's next on my list thank you Joe and so sorry event-driven scripting system is the next one but draft is from the the azure team it's the I think one of the best ways to get yourself a starting point if you are not familiar with the dry run command or you want to get a good starting point use draft in your repo it will attempt to produce a docker file for you and your kubernetes manifests so you can deploy that's my lowest simplest tool make it easy to get started with draft charts is my next one and my simple explanation is to share what you know Hellman tiller have folks heard of helm and tiller more folks than that have heard of charts you use chart Hellman tiller is what deploys your charts this is how you should come on okay all right here we go here we go Joe would you like it up nothing I'll quit picking on Joe Brigade is there event-driven system for producing workflows if you would like to see a demo head on over to the azure booth I'm sure they would be happy to give you a demo it's a new project so I don't have any demos prepared for this but definitely give it a look if you're interested in delivering workflow or event-driven systems I think they've definitely produced a really good collection of developer tools in the past how many folks have heard of telepresence not too many okay this one I think is particularly important if you're using kubernetes on your laptop if you are spinning up large numbers of micro-services and you're on a MacBook Air how far do you think you're gonna get you know you could get a couple containers going but with only eight gigabit memory max on your on your hardware you're gonna run into some limitations real quickly so you can customize many ku-band you could say I want to give it more memory or less memory or you know there's a lot of customization you could do there you'd give it more CPU cores but telepresence really helps pitch in by allowing you to leverage a hosted kubernetes environment and then make those hosted services appear as if they're running locally on your local mini coop environment so it uses proxies and so you could have a large maybe you have a full-scale Oracle big Oracle database somewhere else but you want to be able to leverage hopefully not production data but production quality services during your local development telepresence is a good tool to look into access more with telepresence my last ones many shift and oh see I've got demos of many shift down in the Red Hat booth when I submitted this talk I was in between jobs and so I don't want to put too much ven during in here but man the more of this tooling you layer on top the more complicated you get the more you start to get into that problem of drowning people in terminology and getting distracted from your core mission of delivering product and features and if it all all the talking and all the communication goes to help accelerate your velocity and help the bridge the communication gap between the dev teams and the ops teams then that's exactly what it should be doing make sure you don't overwhelm people with you know a boatload of new terms fresh out of coop con try to give them just what they need give them a solid starting point mini shift and OC will give you a it's basically just like many Kubb except it spins up an open shift environment instead of a kubernetes environment open shift is fully it's a it's a kubernetes environment currently at kubernetes 1.7 if you're using the open shift 3.7 release so these basically give you a more secure multi tenant safe version of kubernetes one of the unique things about it is none of your processes what we won't let you run jobs as route it's not a best practice for folks at Red Hat to run every process at route as route usually not a good practice in production I would argue if you're not doing it later down in your pipeline why do it in local development if you don't need to run things as route in local dev than why do it there either but generally docker run and coops ETL run is generally going to run things as the root user so if you have PCI compliance HIPAA compliance any kind of aspirations towards security at all take a look at open shift and I'd be happy to give you a demo of what we offer down in the Red Hat booth so hopefully I've given you a simple overview just a small couple pieces of what you can learn what you can share here's a collection of other learning opportunities the kubernetes io tutorials are pretty solid there's also a good collection of developer focused education on the catacomb examples I have a lot of developer focused workshops packaged up using these reveal j/s slides and those are easy to fork and share so if you want to fork any of my talks feel free steal them and and let me know if you get some use out of them we also have an interactive learning portal at learn openshift comm and a challenge to developers in the red hat booth if you complete one section of our learning learning we've got a free hoodie for you so definitely stop by the Red Hat booth while we still got them in stock so developers want to get ahead definitely model your i/o and share what you know architects figure out who owns manifest creation right you need to have a solid way to maintain and distribute that that's either gonna be your dev leads or your architects or but it might not be your web developers initially at least QA folks look forward to saying sorry works on my kubernetes operations folks mostly hopefully it's getting pretty boring for operations folks or it should be I know there's still a lot of work to do with kubernetes but primarily you want to be focusing on keeping the environment running upgrading the environments and not getting involved in [Music] too much trash yeah the yeah also you want to loop in security security is a big thing to keep in mind and being able to organize your work in a way where you don't have things like a situation where your front-end web developers are maintaining your docker images and having old versions of libous SL or you know outdated core libraries you want to have a solid mechanism for patching your workloads monitoring your workloads doing a security analysis vulnerability analysis on your workloads so my send-off for you folks definitely join the community on slack in kubernetes users and in cig apps most of the developer focused content happens and sig apps come and join the conversation share what you know share your use cases and experiences and help us develop a range of solutions that expose or hide kubernetes in an appropriate amount of visibility for developers right there's a lot of great videos on the cig apps YouTube channel as well so definitely check that out hopefully that will help you learn to deliver consistent consistently with containers choose the right tool for the job if you are building a pass recognize it and maybe use that tool right if if you don't need a pass hopefully I gave you a good collection of tools to leverage and then hopefully you all get back to making gold records right that's about it for me thank you all for sticking around [Applause]

Channel: CNCF [Cloud Native Computing Foundation]

Views: 18,645

Keywords:

Id: _W6O_pfA00s

Length: 37min 33sec (2253 seconds)

Published: Fri Dec 15 2017