Helm Chart Patterns [I] - Vic Iglesias, Google

howdy folks it's 350 in the afternoon you must be relatively tired let's all stand up for just one moment just really quickly just do some stretching yes that is good that is good now you can all sit down and promptly fall asleep howdy my name is Vic I'm a cloud solutions architect with Google I live out of Santa Barbara California some contact info for me it'll be on the last slide if you need it we're here to talk about you know what we've learned from the helm charts repo and things that you can use in your own home charts to make them more flexible more scalable just better for your users in general so how many people have used helm it's a lot how many people use helm in production how many people have used helm charts from the charts repo wow that's amazing and how many people write their own charts yes sweet okay you guys are committed I like it so I got this like kubernetes problem I'm like into it I get into kubernetes mode and it just makes me go all crazy today we're gonna go a little bit helm 101 for those of you who didn't you know get an opportunity to use home in the past but as you can see we've got a good community of users and it's a useful tool it can be a useful tool for you then we're gonna go through what the charts repo is why we started it and that kind of thing and then talk a little bit about the important pieces of charts and then a couple tips and tricks that we've seen in some of the the charts that we have in the chart repo some of the patterns that we've seen emerge for making them better charts for your users so how am 101 helm is a tool for installing and managing your kubernetes applications so it helps you with the lifecycle management of an application within kubernetes and we'll talk about what that is it has two parts you install a client on your laptop and then inside your kubernetes cluster you have a server called the tiller and that manages kind of receiving the commands and doing the actual work of deploying manifests into kubernetes so how um if you're kind of looking for an analog in the operating system world it would be like apt-get or yum and then you have charts which are more like your packages so your rpms or your debian and chart has metadata about what's in the chart what it is and then a bunch of manifests that are templated so that you can have ease of use and and pass some values through it so what is a kubernetes application this is something that's kind of been tough to figure out over time like what is an application and generally it's gonna be a set of things that you install into kubernetes they may relate to each other you may have a config map that needs to be attached to a pod may have a volume that needs to be attached to a pod and then a pod that needs to be let balanced by a service so these things are kind of all interrelated you can't really update one without thinking about the other at the very least and they're they're all written in llamó huh for most of us how many people do yamo for kubernetes manifests yeah most of us so helm allows you to package these things together so you can template eyes them and use the same values across a bunch of these manifests and then install them all together for your users of these packages or these charts they can see just a simplified config right the the things that are important to the application they don't have to worry about the internals of how your services are mapped to each other or how your deployments work or what M variables you're using you can provide them kind of a higher level abstraction API via the values file or the configuration of your home chart so you deploy your helm charts and then you deploy a new version and that's awesome and you're stoked on your new version and then you hit a problem and you're like wait we need to rollback so in this case home helps you because it can roll back everything as a whole right you don't just roll back the config map you don't just roll back the deployment you roll everything back together and get back to the original state you had before and then you can roll out version 3 and keep going along the way so I've talked about templating it's a templating probably 50 times already in helm you use go templates and those are passed into your your values passed into your templates that you can kind of parse out some some values there so for example here I have this replicas that I've provided to my users where they can change the number of replicas by using the values file you can also use similar names for things so if your service is called you know Vicks or if your application was called Vicks app I can call my service Vicks app and my deployment Vicks app with just these these templated values some of the cool things that you get out of using helm on top of just regular go templating is we have these things called sprig functions and these are little helper functions that make it easier to do some of the more complex things like have a default value or writing a check sum for a particular config file so here are some of the things that sprig allows you to do so on you know go template do we have all kinds of things we can do like loops and stuff like that but sprig allows you to do like math functions or maybe you need to add a date or timestamp somewhere maybe you need to base64 encode a config file for some reason or another so this gives you a lot of flexibility as far as how you're writing your your configs and your services stuff like that so the charts repo kubernetes slash charts on github it's the centralized repository where we're hosting and maintaining the charts that you as a community have provided how many people have interacted with the charts repo in here awesome we love you thank you and it's basically just a place where we can get together and take our knowledge of applications and how you configure them and how you use them and so that everybody has at least a starting point I don't necessarily expect that you're gonna have you know production workload running off of one of our helm charts but maybe it's the first step T for you to get there we provide continuous integration so we have a testing subsystem that will run your chart and make sure that it passes the linter make sure it actually installs properly make sure that pods go running and then we have code review provided by some of our maintainer x' but do we have any maintainer x' in here right now other than my we have one right here that's it he what in the back too so that they caused all the problems and it's the default repository when you install help so the stable repository when you install the helm client or you do a home search those are the charts that we have there so why did we do this I don't know if you guys remember and like a year and a half ago if you did like Postgres kubernetes Google search you'd end up with lots of results lots of different repos that had similar ideas on how to do things and where maybe had you know trade-offs as to what they did and so you'd have to look to do ten results and decide which one was best for you we wanted a place where we could just all work together on the same thing and make it as easy as possible to have a starting point for those common apps so there was a bunch of things then we added one more thing creating a new standard but this time I think it's it's stuck around we have a hundred and thirty charts now a year ago I think we had nine or something like that so it's it's grown pretty pretty fast we've merged 2000 PRS over that period so that's it improvements to charts and new charts and we have over 100 contributors which are you know the folks that you saw raise their hands which is super awesome it's not slowing down in any way shape or form we're still kind of hammering through lots of PRS about six months ago we had like this high watermark of about a hundred PRS being like we need to drop everything and figure out what's going on here now the high watermark is two hundred I expect another three months it's going to be three hundred we'd love to have more maintainer so if you're interested to keep those watermarks down but it's it's growing really fast and it's awesome we have ten thousand unique visitors every week to the charts github repository just a lot of people and the number of Forks we can no longer see our fork graph which is a really great display but we do have the number and this is cool for me not not like it's a vanity metric but this means that people are actually taking the code and doing something with it maybe they're changing it maybe they're contributing back but at least they have that starting point and we're kind of the known place to go for those starting points for these applications go so next we'll move on to kind of the chart structure and things that are important with the files in charts so the chart structure there's a certain set of files that make up a chart one of the most important ones is the chart yamo and this is the metadata about what your application is what thing is this installing so here's the example from the spinnaker chart you have an API version that's the API version of the chart right now we're only at v1 you have the description which is a human readable way to describe what you're actually installing this shows up in the helm search output for example you have the name of your chart then you have two version strings here so version is the version of the chart and app version is what the application version is so for example here I have version 1 1 0 of spinnaker and the chart version the iteration of this deployment mechanism is 0 3.8 this is important we've had some chart maintainer x' try to pin these things and what you find is over time your application doesn't change as much as your chart you're adding things that make your chart more flexible or a bug fix in just your chart and then you're adding you know bumping versions my goodness here was one of our first ones to try that with the Prometheus chart I think he went about three months with that methodology and then we just decided to break from it you have a website here which is just a URL to point people to but I think the more important thing to have in your chart yamo is the sources where can people go to rebuild the pieces of this so for example usually people put the github of the actual project itself a lot of the the charts that we have our open source so where is the github for that thing and then did you build like a custom image we have a custom image point people to that docker file so they can maybe customize it and move on from there you have an icon which is mostly just a visualization tool we have cube apps calm which uses this to show your your chart icon and then we have the maintainer so it the name here is actually the github ID of that user and their email so as maintainer as we look at this and if there's something that's a little bit over something that's not just a trivial bug-fix we try to ping those maintained errs to make sure that this is a good change that they want in their chart now this is the values file the values file is basically the API that you're providing to your users of your chart that's the the way to look at it changes to this that are backwards incompatible you should probably bump the major version of your chart to show people that you are breaking the API here's where you can abstract away some of the complexity of what you've done in the kubernetes deployment methodology and and make it simpler for your users to configure things you can add comments generally people comment in how to do certain things for example here this is again the spinnaker chart I allow people to change the images that are deployed for each component and also for example allow them to configure how mail works it's disabled by default but I have just the the parameters that they need and they don't have to go look in the spinnaker configuration to figure out what things to configure there so with the the values file generally what you'll find is you're on this spectrum or it just charts in general your start out kind of relatively reproducible there's not that much that can change but as people use your chart and your application they'll find that you're going more and more to be more flexible which means that you're opening up some trap doors for people to change things that maybe you can maybe you can't reproduce on your own and that's just kind of a natural progression at this point that we've seen another thing that we have in in a chart is the requirements Yamma and this is what defines what other charts you may want to deploy alongside your chart so in general what we've seen is this is used for data stores you have a Postgres dependency of a Redis dependency or anything like that and the version here can also be a cember so I can say you know give me anything in the 0 for X series of the Redis chart and you'll update to those things let me have your templates folder and this is kind of the core of the chart or the the kind of biggest simplest place you have all of your kubernetes manifests that are templated out or can be templated and this is what will be kind of cute controlled applied to a certain extent into your your kubernetes cluster next you have the notes file and this is one of my favorite kind of experiences in helm or using helm is that the notes you can as a chart maintainer chart creator you can provide to your users a getting started experience via the notes file and the notes file is again a templated file that is template it out and spit out after your chart is installed so when you do how i'm install chart they'll spit out this template a thing that can maybe tell your users what to do next right instead of just saying yes it's installed tell them pay here's how you connect to it here's how you get to the web UI here's if you're going to use it inside of your cluster here's the service name that's currently configured so it's it's just getting people that much closer to being kind of interacting with your application next is a readme and this is generally kind of more in-depth documentation on what people can do with your chart and it may have you know methodologies for configuration or things to watch out for or disclaimers things like that it's just a markdown document and kind of gets into more in-depth information so now we're gonna go look at some tricks that you can use inside of your charts and things that you can look at to kind of make your charts a little bit more flexible or a bit a little bit more useful I think the first thing that most people should be using is when they're first gonna create their charts to use the helm create command how many people have used home create or use it regularly yeah that's good so home create scaffolds out a chart an example chart for you so you can start with this and then tweak it from there the actual application that it deploys is just an engine X web server with no config but it just shows you the things the element that you're gonna have to have and we try to keep that up to date with our best practices so if you're gonna get something into the charts repo it's best to start with this and then you know you've got something that's going to match our expectations another thing you can do is to add starters so let's say in your organization you have certain patterns or certain things that you want to have in each Hjelm chart you can create your own starter templates and then pass them to the helm create command with the dash dash starter flag and have that be the scaffolded thing that gets put out there for users of your charts so super useful one example is you know you have a certain paradigm for how you do stateful sets or for your services you have a certain way that you want to do your deployment rollouts or your deployment strategy right you want to do max unavailable always so just set that for folks so that they have it by default maybe you have a certain way that your values files are laid out this is where you can put all of those semantics in place this is a trick that I really like I've used it in a couple charts and I know that some charts use it as well and it's slightly complicated but it's a very useful tool so what I want to do here is I want to be able to change a config map in my application in my chart and have it automatically update my deployment so what I'm gonna do here is add an annotation to the deployment and I'm gonna use sprig functions to put in a checksum of my configuration so here you can see I have they I'm printing a full templated config file that spinnaker dash config - yamo and then getting the checksum of that that contents when I get that I'm putting that into an annotation so every time the config file changes it's gonna roll out a new deployment because that annotation change so it's kind of a little trick to get config maps to automatically update some applications will just detect that the config map has changed and reload themselves but some aren't that smart so you can do this to kind of hack around those applications that don't know about config maps another super useful tool that I've used in some of the less than cloud native kind of applications that I've had to install is the helm hooks so home let's you do some things as it releases or as it installs or deletes your your application so here I'm doing a post install step so let's say you deploy an application and you just need to like run one command at the end like for example and this one this is again the spinnaker chart and I need to create a bucket in the mini Oh object storage after its installed so helm allows me to add an annotation to a job that job is gonna run until it completes part of the job controller and all I'm really doing in that job is just running the menial client in order to create the bucket so as menials coming up this thing is gonna fail and kubernetes gonna keep restarting it until it just succeeds once and then it's done so now I've gotten those kind of posts in it kind of steps automated through my chart the types of hooks that you have available are pre and post install pre and post delete and pre and post upgrade and pre and post rollback so you have a lot of control over your application lifecycle you can imagine doing migrations and those kinds of things in one of these hooks all kinds of stuff that you might have to do at install or just normally throughout the process of managing your application this one is pretty useful something we saw I think first in the the prometheus chart is to allow people to basically provide you with a volume for data and so you let them provide you with the name of an existing persistent volume claim and if they provide that to you you use that rather than one that you would have originally provisioned in the chart so you might have the the need to decouple your chart lifecycle from your data lifecycle maybe you don't want those ever to you know you don't want something to accidentally delete data or you have a developer who has a PVC that's there like special data set PVC that they want to reuse with your application chart over and over you can give them that ability here another thing that we've seen often is web apps being you know submit it to the charts repo and with those web apps we generally ask that they allow someone to add an ingress really easily like pre-configure this ingress you wire up the service import already for them you can have you know the ability for them to list house names or maybe a TLS secret to include in there but try to have that as part of your API if you are a web app because most things will want to do an ingress another side of configuring an ingress for four people is to have the annotations be available as a thing that you pass in to to those ingress 'as so annotations on ingress --is that are really common are for example like which type of ingress controller to use you might use the nginx ingress controller you might use like the cloud default one but you as a chart maintainer should not dictate that necessarily another one that's seen commonly is like cube Lego they'll get my TLS certs you don't want to necessarily manage that you don't want to over you know abstract that just let them add their own annotations annotations are kind of like Pandora's box anyway all kinds of crazy stuff gets done with them so I tend to just let people add those crazy annotations there this one is near and dear to my heart Michele Nirali wrote this home testing and we like we're like please we need helm tests because as maintain errs we want to have the ability for the chart maintainer to send us a test that says this thing actually works so now you can do home tests from the the home client after a release has been installed and it will run us step and so here I add an annotation saying that I want this test to run and I expect it to succeed here I'm running a pod and it's just gonna run a bash script like like most things the world is tied together via bash scripts so here this is the actual test that's in the Jenkins chart not the craziest thing in the world but it lets us know that like we can at least get a two hundred from the login page right it's something simple like this that as chart maintainer is we can bank on to think that you know your charts actually doing something useful in your in your organization you may have much more in-depth testing and you can run this at any time so maybe you just run this periodically you have some checks that you want to run as part of automation that runs every two hours or something like that and it just spot checks things it's also templated so it's gonna be able to use all those values that you've passed around as well so it's it's a useful hook to have we hope to have more tests in in the chart Street but I think right now out of the 130 less than 10% have tests which is pretty painful that means we have to go and launch it and make sure that it actually does anything useful and that the PR that we're accepting doesn't actually break core functionality for that for that chart this is one that I think is really useful but is also one of these trapdoors that people kind of once they go down this route you kind of have they're on their own some people will want to if you have a config file for your application rather than have you template it out for example for an engine X config or something like that they want to just provide their own so some charts have done you know custom config map parameter that allows you to overwrite or provide your own config map this is like on the opposite side of the reproducibility spectrum I mean you're not really going to be able to reproduce what they did basically ever but it does open up more use cases for your chart which i think is useful another thing we've seen in the engine X controller chart is not just having them overwrite the config map but to provide the name of a config map so can be can be done a few different ways one thing we've done recently is to have all of the helper templates in home charts be named spaced so a helper template if you look at a chart you'll probably have underscore helpers file in there and this is where you can define little chunks of snippets of code that you want to actually inject in different places in your in your chart but you don't want to have a big stack of code sitting there and so these templates can be generally for like names of things in kubernetes you have to have things be a certain length so if people name things in a strange way you want to truncate that to something that's supported within kubernetes so that's what we do with names and full names name spacing it allows us if you have this chart as a sub chart for example you can always reference the Jenkins name by Jenkins name across the board you don't have to think about how it's actually nested or anything like this one is also really useful and so conditional dependencies is a feature of the requirements file a lot of charts we expect charts to actually provide something useful out of the box with without any changes to the values but you may have some data stores for example that we deploy for you as the chart that you already have somewhere else so one example is the spinnaker chart actually does needs object storage so we use Mineo by default because we can prevent a vision that within anybodies cluster and we get out of the box experience that's good within cloud providers though you want to use their object storage so in Google cloud we have Google Cloud storage and in Amazon you might have s3 so you want might want to disable the menial deployment and you can do that with a value called Mineo dot enabled so if you set that to false we will no longer deploy the Mineo chart for you and you are on your own to configure the s3 configuration again on the kind of opposite end of the spectrum these are some where the trap doors that that we've seen I mean chart maintainer is provide to their users is the using the - yeah malfunction in sprig which will basically take a chunk of the values file and just inject it as yamo into some other template that you have so we're this is extremely useful and kind of necessary is with like node selectors and Toleration specifically because these are gonna be practically bespoke cross everybody's deployment you're not gonna know how they label their deployment or their nodes or anything like that so just give them the ability to add node selectors on their own in the values file and then propagate those through to the deployments you have similarly with toleration x' we also have seen this relatively often with resources some people want to have requests and limits some people only want limits you know just give them the ability to set those however they want through the values file and pass that directly into your pod your pod specs so those are some of the tips and tricks we have a hundred and thirty charts out there so if you have an location that you want to deploy go look at the chart sweet bun see if there's something that's similar to it from a kind of deployment mechanism standpoint and try to figure out if you can kind of take some tips from some of the charts that already exist for us in the charts repo the next thing we're gonna try to do is give the chart maintainer x' more power over their actual chart and merging their own PRS that's gonna help us with our velocity so we don't hit that 300 PR mark that is almost inevitable at this point and also to delegate the responsibility to maybe even a different get repo so if you want to host your your git repo or your chart somewhere else but still get it into the stable chart repository we want to be able to facilitate that as well now that we're at a hundred and thirty charts it's time to look at how many of those are actually updated or useful and are still maintained I think we'll get at least a fifteen percent attrition rate on those once we really start looking at it and just remove those from the tree or put them in a folder somewhere like deprecated or something like that and if someone wants to bring them back to life that's great maybe we'll call it zombie and and then you know we can we can go forward with it the other thing is more functional tests for charts there's an idea being thrown around that will require a functional test or else we won't merge anymore PRS in that I don't know if that makes you scared or not but it should be scary for any maintainer x' but it the the functional test is really important to us it's something that gives us more comfort in merging things and will give us better velocity as charming tenors so that's all I've got thank you folks and I'll do questions out in the back [Applause]

Channel: CNCF [Cloud Native Computing Foundation]

Views: 31,544

Keywords:

Id: WugC_mbbiWU

Length: 28min 32sec (1712 seconds)

Published: Fri Dec 15 2017