What The Heck Are Kubernetes Resources, CRs, CRDs, Operators, etc.?

today we are going to do something completely different yet somehow the same I want us to explore kubernetes but not in the way I usually do today I want to demystify how we should Define stuff and why some of the commonly used tools might not be the best choice I want to talk about resources custom resources custom resource definitions operators and a few other similar less known things [Music] custom resource definitions or crds kubernetes is first and foremost an extensible API let that sink in for a second that means that kubernetes was not designed to be a thingy that runs containers at least not exclusively or to be more precise that's true since version 1.7 of kubernetes when custom resource definitions or crds were introduced since then if not before we can say that kubernetes is designed to manage whatever and however we want to manage so what are kubernetes crds or custom resource definitions a simple version is that kubernetes crds are a way to extend kubernetes API now you might be asking why would we want to extend kubernetes API well and this might come as a shocker kubernetes itself the one that comes out of the box is very very very basic it does not do much now you might say hey that's not true it does a lot of things and you would be right you can say that we can use it to Define deployments which are going to create replica sets which are going to create pods and our containers are running in those pods and that alone is a lot and it is you can say that we can use it to Define services and ingresses that allow us to access those containers those pods or you can say hey kubernetes is awesome because it allows us to attach external storage into our applications package that's containers or container images to be more precise you would be right on all those statements partially at least to begin with all those things that I named then others that come out of the box in kubernetes are still very basic networking for example the one that we get through Services is not doing much it's not really Advanced it's very very very basic moreover there is no storage baked into kubernetes there is Storage interface that can be extended with implementations but that might not be obvious if you're running manage kubernetes cluster what does matter is that storage is not there either we need to add it to kubernetes unless we use a third-party provider or service provider that did that for us so even storage is not there what I'm really trying to say is that yes kubernetes is doing a lot of things especially if you compare it to what we had in the past yet we can do so much more and to do that so much more we need to extend kubernetes we'll make a very quick break to thank cast AI for sponsoring this video castaya is likely one of the best and fastest Auto scalers on the market it works with kubernetes and apart from normal servers it also supports GPU nodes so those working with AI and ml are covered very well covered and if you want to save on your bills it works with spot instances as well it's a wide reaching automation platform for kubernetes that automates cost optimization and removes the need for you to manually fiddle with resources try it out for free now let's get back to the video anyways we got crds long long time ago that allow us just that they allow us to extend kubernetes with something for example there is no such a thing as application entity in a kubernetes cluster there is a collection of building blocks like deployments ingresses services but there is no such a thing as application we can use those building blocks but if you want to have something that resembles application at least from the end user perspective we need to edit that ourselves or as an alternative we can install a third-party solution that will bring us something that resembles an application similarly there is no such a thing as a database in kubernetes if you would like to run databases in kubernetes one option is to work with those building blocks which are not really great or specially complex we can Define crds ourselves or we can install third-party Solutions like for example Cloud native postgres or CMP now before you say that's okay I can live with those building blocks let me ask you a question related to let's say databases build those building blocks give you a possibility and feature an option to Define when your database will be backed up when it will be restored and how it will be restored how to replicate data between replicas of the database and so on and so forth now there is no such a thing you cannot do it at least not in a kubernetes native way without extending it through custom resource definitions we can say the same thing for quite a few other solutions for almost all kubernetes Native applications that would be cross-plane cubevela kubernetes operator for Prometheus Cube verd and quite a few others if you're already using kubernetes you already extended it probably extended it whether you know it or no it's enough to install almost any third-party solution for kubernetes and you are extending your cluster with custom resource definitions and those custom resource definitions Define how we can do something how we can create Argo CD application how we can create a database using crossbank how we can create virtual machines so the short version is that customers definitions enable us to Define what something is in kubernetes now with that out of the way the important thing to note is that custom resource definitions or even resource definitions that are baked in kubernetes are not doing much by themselves they're actually doing nothing they're just the way to extend kubernetes API without performing any actual work hence crds are useless by themselves what they need are controllers when we instruct kubernetes API to create a resource or custom resource all that we're doing is publishing an event saying hey somebody created this resource this thingy whatever that is the event essentially says hey I have this resource that's it that's all it does actually that is not all it's doing but for the sake of this story you can consider either that being the only thing that we get when we create a resource so message is like hey I got this new resource or this resource was modified or this resource was deleted are just events and something needs to listen for those events and do something about it and that is where controllers come in swooping controllers are essentially processes that listen for events related to specific resources or custom resources and then do something when those events are published so if kubernetes API creates an event saying hey somebody wants to create or manage a database there is one or more controllers that kick in and say hey that's me that's me I would like to do something with the database if that would be Cloud native postgres or cmpg it would catch that event and create whatever needs to be created for that database to be running and run successfully and be managed and do whatever needs to be done for our databases inside kubernetes clusters on the other hand if that would be cross-plane with a controller created through crossbank compositions it would do something similar meaning it would create whatever needs to be created to manage databases but in that specific case in a hyperscaler like in Google cloud or AWS or Azure heck you could have multiple controllers and you often do listening for the same events and then doing one thing or the other for those things those controllers do can be anything it really depends on controllers and that means that kubernetes can be extended to manage anything or everything or nothing it could be managing virtual machines it could be managing Cloud resources in AWS or it could be ordering pizzas actually there is a controller for ordering pizzas now please note that the story is the same no matter whether we are talking about controllers related to custom resource definitions or crds or controllers that are already baked inside kubernetes itself controllers listen for events and then do something actually some resource definitions that are baked into kubernetes like networking do not even come with controllers we need to pick which controllers will fulfill networking tasks so when we create a service in a kubernetes cluster that comes out to the box there is no networking but if we install cilium that understands events related to Services then something is happening then we have networking between our applications so long story is short controllers are processes that do the actual work related or initiated through events and those events are coming from us creating resources or custom resources that are fulfilling the schema based on custom resource definitions or resource definitions now there are also operators but operators it's just a pattern that defines how controllers should be created not every controller is an operator because operators need to follow specific patterns but every operator is a controller so operators are just a pattern how to create controllers and probably the most important part of that pattern is that they need to implement control loops still for this story all that should not matter you can think of operators and controllers as the same thing they manage life cycles of things defined as custom resources and now comes the interesting part or to be more precise the answer to a question that I get quite frequently it goes something like this Victor you talk often about creating operator same controllers and share these and all this stuff and we can be writing it ourselves so we can get them to third parties applications or services and I would like to know whether I should do that or I should keep using Helm or customize or jsonnet or Vitality or whatever I'm using to Define what something is so share this and controllers from one hand or help and similar tools on the other which ones should we choose the short answer is that one does not exclude the other kubernetes expects us to communicate with it by sending requests to its API and those requests must be in form of yaml or jsonet there is no way around it kubernetes API only understands yaml and Json didn't say jsonnet no Json so Jamil and Json we came right to the Tamil either directly or we can use some other tools like Helm to to help us write the te amo which sounds strange right now the important thing is that it is always yaml or Json always it must be yaml or Json we cannot send Helm chart to kubernetes we cannot tell kubernetes here is a Helm chart do something with it no we need to communicate with it in form of yaml or Json so what is happening when we execute commands like Helm install converting templates Helm templates into yaml and that yaml is sent to kubernetes API it's always yamo now whether that yaml represents a definition of a resource or definition of a custom resource does not matter it's all the same to kubernetes if that's something if that resource is acceptable or to be more precise if there is a schema for that resource if it complies to a specific schema it's good to go and that schema is either what is baked into kubernetes or what we extend it with and that's here it is so share this and operators and controllers are a way to extend capabilities of kubernetes and Helm or customize or jsonate or Vitality or any other tool that you use to define what you want to send code to kubernetes are just shortcuts they are just templates or overlays that get converted into the Tamil one does not exclude the other however using CR this might result in less of a need to have something like hell so you can use Helm but you might not want to let me explain we use templating Solutions like how to define all the variations of all the resources that constitute something let's say an application and those variations are many we might have to change every once in a while the tug of an image or a host or number of replicas and so on and so forth now when we use kubernetes building blocks like services and ingresses and deployments and stable sets and so on and so forth we have a lot of those resources to manage hence we need something to help us with that otherwise we would end up with thousands of lines of yaml and that's why we have templating through Helm and similar Solutions it now comes the important part we are generating all those resources all those building blocks on the client side Helm is a client-side tool when we execute film installed or from a terminal let's say and I could see the terminal like a client then we are generating the end results the final manifests that are combination of all those resources and then sending it to cube API so most of the action is happening on a terminal or in a pipeline or anywhere outside before kubernetes API however if we create our own crds and our own controller so if we install third-party Solutions like but there are many k native for example then most of the action is not happening before something reaches kubernetes API but inside the cluster controllers in those cases do most of the heavy lifting they're responsible for creating all those building blocks all those pods and deployments and services and increases inside kubernetes clusters essentially what Helm is doing on client side before something reaches kubernetes API controllers are doing on server side inside clusters so you can think of a controller that defines how a database should be managed is equivalent to hand templates except that it's doing the work inside the cluster and that means that we humans do not need to Define all those building blocks we do not need to deal with all those details with all those low level resources ourselves because clusters are doing the work for us and that makes things much simpler we can Define 10 20 30 lines of yamo that Define some custom resource based on custom resource definition instead of hundreds or thousands of lines of yaml that Define all the low level details of that something what we end up defining is much simpler shorter yaml less yaml less of everything on our side on client side and that means that there is less need for templating if it's 10 lines then a simple yaml file should be enough there is no need for help if it's more complicated slightly more complicated or slightly more complex then we might be still better off with customize because hey it's not 20 lines of yamla but it's 50 but do we need templating for that probably not in other words if complexity is moved away from Terminals and pipelines and wherever we are doing things into the Clusters then the complexity of what we need to Define before we reach kubernetes API is much lower hence lower need for templating complicated thing becomes a simple yaml file so let me go back to the initial question should you be creating crds and controllers and so on and so forth and the simple answer is yes you probably should sear these are a way to extend kubernetes API with new schemas and through those new schemas allow controllers to do the work that we normally do before we get to kubernetes controllers are about delegating work from humans to machines more importantly crds and controllers are kubernetes native way to do things Helm is not kubernetes native way to do things controllers and crds are that does not mean that we do not need help we still need to generate that yaml somehow but we do not necessarily need to have it as complex as we do if you have a backend application Define a crd and a controller that will manage and create and delete and do whatever needs to be done with a backend application if you have a front-end application well again crd controllers do the work inside the Clusters you if you need a database well again the same thing the only question you need to ask yourself is whether you should create those definitions and controllers or you should use some third-party solution for that like K native cubic Cloud native postgres and so on and so forth and if you do that many of the things will be so much easier for end users it will be easier to define something like hey my application how do I Define my application well those 10 lines of yaml that contain whatever is the host or the type of the application whatever you need to Define without going into low level details observability will become easier blogging monitoring everything will become so much easier if you follow what kubernetes prescribes you to follow and what kubernetes wants you to do is to extend it but there's a word of warning I must say in advance no at the end of this video do not write yourself controllers in most cases there are tools that will enable you to do that that is Cube well that is cross plain project I'm very invested in there are many many others I will not suggest which one to use but remember there are easy ways to create custom resource definitions and controllers that will do what you want them to do that will extend your kubernetes clusters to a reasonable level to a useful level to a usable level all in all know this custom resource definitions and controllers had a Bedrock of kubernetes they're probably the most important feature of kubernetes kubernetes is an extensible API designed to be whatever you want it to be embrace it use it as such and you will see how things change drastically do not be afraid to do what kubernetes eventually suggests you to do do not be stuck in a past do not try to apply the logic you were doing or using or applying before kubernetes with governeth thank you so much for watching see you next time cheers thank you

Channel: DevOps Toolkit

Views: 11,132

Keywords: Kubernetes architecture, Kubernetes operators, container orchestration, containerization, custom resource definitions (CRD), custom resources (CR), kubernetes clusters, kubernetes resources, kubernetes, k8s, viktor farcic, devops toolkit, crd, cr, kr, krm, helm, kustomize, jsonnet, ytt, carvel ytt

Id: aM2Y9m2Kazk

Length: 21min 7sec (1267 seconds)

Published: Mon Oct 02 2023