How To Shift Left Infrastructure Management Using Crossplane Composites

if you're a software engineer you're either sitting on the left side or on the right side being on the left side means that you're in a team that is in charge of an application or a service that serves and users your team should be autonomous you should be in full control of the life cycle of your application you and your team should write code run tests build stuff deploy stuff manage infrastructure and so on and so forth what you don't want is to start opening jira tickets whenever you need something to be done and wait until some other teams do things for you like hey i need this to be deployed let me know when it's done or hey i need a cluster can you please create it for me i can wait your team wants to be autonomous and your team wants to be self-sufficient on the other hand if you are on the right side you are specialist in something that could be infrastructure cicd security whatsoever it could be many different things what matters is that you're specialized in something you're an expert in some area that is not directly related with development of an application and you do not want others to open jira tickets you do not want to spend your time deploying applications for others creating clusters for others and so on and so forth if you are let's say an expert in infrastructure what you do want is to create services that can be consumed by those teams on the left you want to create a system a platform of sorts that will enable people on the left those teams in charge of developing applications to self-serve themselves you and everybody else wants to shift left we want to enable people working on those applications to be self-sufficient and the key word there is enable we cannot expect everybody to know everything there is a reason why somebody specialized in let's say infrastructure it takes time to master everything there is to master about a specific area or specific subject and we want to leverage that expertise but not in a way that experts do stuff for others but in a way that experts create systems that can be consumed by others and since we understand that not everybody can understand everything and be expert in everything those systems those platforms need to be designed in a way that they fulfill all the needs we have and yet be very easy to be consumed by those who are not experts in that area we want to create opinionated tools that are easy to consume and those opinionated tools cannot be bought cannot be purchased by third-party vendors simply because they do not know what your opinions are you're an expert it is your job to generate those opinions to define those opinions and do it in a way that it is easy to consume software vendors should give you tools to create opinionated frameworks and then give them to the teams on the left and that's what we are going to do today today we are going to explore crossplane composites now i already talked about crossplane i will not repeat what crossplane does if you're curious if you haven't used it yet check the video above my head it introduces you to cross plane in general but today we are not going to talk about cross plane in general we are going to focus on composites because they will allow us to do something amazing a quick note i joined upbound upbound is a company behind crosslane and that might make you think that i'm subjective and i might be subjective about crossplane or you might say victor joined upbound because he believes that crossplane is awesome it's up to you to decide whether i'm subjective and try to sell you something or i'm working for outbound because i believe that crossplane is awesome so here's what we are going to do we are going to create a way for developers to create clusters by themselves we are going to make it extremely easy for them and yet those clusters are going to have everything that they need to have and we are going to make them provide diagnostics so that those clusters can be created in aws in azure in google or wherever else in other words we are going to create complex things that are very easy to consume we are going to make something that everybody could use and yet that something is not going to be trivial for the sake of conversation i'm going to call people on the left devs and people on the right ops i know that that's oversimplification i know that we shouldn't call those groups of people or experts like that but for the sake of simplicity if you're in charge of an application if you're developing an application for an end user i'm going to call you dev and if you are in charge of infrastructure if you're the guru of infrastructure i'm going to call you ops so let's explore how ops can enable devs to manage infrastructure before i started recording a session i already created a kubernetes cluster and installed crossplane inside that cluster i'm running a local cluster using k3d normally you would use a real cluster for that if it would be production usage but for this demo a local cluster should be fine besides installing crossplane itself i deployed also providers for aws google and azure you do not need all three of those you can use only one depending on how many providers you want to use and i created secrets with authentication to those providers so that crosslink can do whatever needs to be done the repository with sample code and all the commands that i executed before this session and that i'm about to execute now are in a gist and the link to the gist is in the description of this video i will be using aws today but the gist contains instructions how to do the same in azure or google cloud if those are your preference and if you want to use some other provider just let me know and i will extend the gist with your favorite provider so i am an ops person right now and i can say hey dear developer how about you taking a look at this file cluster.yaml that file contains everything that you need to know or do to create and manage your clusters and this is how it looks like right i removed all the complexities i will allow you as a deaf person to focus on what matters to you and all the complexities that you do not necessarily care about are handled by me this is all you need to know to create a cluster so what i did as an ops guy i created the new custom resource definition that is called composite kubernetes cluster i will show how i did what i did later for now and focus on what a developer needs to know you their developer you can change the name we can keep it in a you can change or read labels nobody cares about that for now but what does matter is that here you can choose the provider you can call it cluster google or cluster azure or cluster aws depending on the name of the provider you will get the cluster in that specific provider so i use it there can say oh that's great look at that i'm going to actually run my cluster in aws so i'm going to change this to cluster aws brilliant now there are a couple of parameters that i can tweak to get the cluster i need the cluster i want and those parameters are only those that i care about i can change the size of the notes to be small medium or large i do not need to know how those sizes translate into specific sizes for specific providers is it t2 something or t3 something or is it this or is it that no for me the only thing that matters is that i can have small or medium or large nodes and what needs to happen in azure or google or aws is not my problem i can specify the version of kubernetes i want to use or leave it empty and in that case it will be whatever the ops guy decided should be the most appropriate kubernetes version that i should run and i can specify the minimum number of nodes that i want to have in that cluster that cluster will allow the scale it will not auto scale to infinity it will have to scale to whatever ops decided it should auto scale what should be the maximum but i as dev can control what is the minimum number of nodes and finally i can choose what is the namespace where the secret with the important information will be stored that secret might for example contain the connection string that i can use cube config to connect to that cluster and what matters for me as a developer and also for somebody else's ops is that this definition was not created by some third-party vendor this is not what somebody else somebody outside my company thinks that matters this is a composite a crd that was created by other people in my company people who are considered experts in managing infrastructure for my organization also one more thing you can see here that those two parameters are commented that means that i will use whatever are the default values or to be more precise whatever somebody else whomever is an expert in infrastructure thought should be the default values so in this scenario i just showed that i want to use aws and i kept everything else as is later on i might change some parameters but for now i do not really care i just care that i want to create a cluster in aws so let's do that let me save these changes and i will do cube cattle apply dark dash file name cluster dot yaml and now the magic is happening there is actually one thing that i did not do in this demo and that's the final thing that anybody should be executing cube cattle this or that it would be even better if i as a developer could just push this manifest to git repository and not even deal with the cube cattle or whatsoever i should just store what i want in a way that is easy for me to understand and manage in a git repository and let some other tools whatever tools there are maybe argo cd or flux or whatsoever manage all that so we'll skip the github story from this demo and maybe explore it in a different video you just remember that everything i'm doing with cubecuttle you should be doing vidargo cd or flux and just by pushing changes to git repository now while everything is being created let me change the role and show you how all that works if you're an ops guy how did i get to the point that i can offer something like that to others what was the process behind it so i'm an ops guy now i'm an expert in infrastructure and this is what i did i created first the file called definition yamo and that file defines what will be the interface that everybody will use to create infrastructure this is not related to any specific provider or anything like that this is just the definition of the definitions of the manifest that others will be able to create and what matters here is really this section which says hey i want to create the definition of what the cluster is and that definition should have three properties it should have a version of the kubernetes cluster somebody should create it should have the size of the nodes and it should have the minimum number of nodes the minimum number of nodes defaults to one and only the size of the nodes is required so whomever is consuming this has to specify only the node size and nothing else everything else can be left as default values which i will show you later how that really works and i also define some properties for the status which is cluster name control plane status and node pull status you will see the statuses later finally i define additional printer columns which are more or less the same cluster name control plane and node pool and you will see later what those are for now what matters is that i create a couple of parameters and i wrapped it all up as where is it composite resource definition so this is a special type of definition i can define in cross plane and it is called composite resource definition it will be a composition of quite a few different resources and it defines what others can define when working with this composite which is a few properties few statuses and a few columns that should be printed somewhere on the screen now as you already saw this composite can be used in azure or aws or google so let me show you how i define providers and those are all providers that can be used with this composite resource so let me show you those providers the provider for azure which is the simplest quantity i believe is stored in here and this is what it defines it says i want to create a composition the composition is kind of a combination of one or more resources done in a certain way and i have full control of how that composition should be created and how it works with the definition that we explored a few moments ago so what do we need when creating a cluster in azure i need to have a resource group that's azure specific and i need to create an akf cluster and here i'm defining what those things are i will not go deep into how this really works if you want to have a deep dive into crossplan compositions let me know in the comments i will do that for now i just want you to have a feeling of what is possible so let's find one interesting example that could be yeah here note size right so that parameter that you saw earlier that allows us to specify the size of the nodes which with values that are small medium or large this is where we map those values into something that is understood by azure so i'm saying here hey whenever somebody says small in azure that means standard d2v2 and whenever somebody says medium that means this and whenever somebody says large that means that now this creates a resource group and a kubernetes cluster in azure in this case and we could do so much more we could install in that cluster whatever our end users need it could be ingress it could be steel if developers would always need a database which could enable them to have a database whatever is the database service in azure and so on and so forth we could make this as small or as big depending on what we really need what do developers need when they need a cluster it could be many other things outside of that cluster but for now i'm keeping it simple similarly i have a file gcp.yaml which is basically doing the same thing but for google it is an implementation of the definition that we saw earlier and in case of google i'm translating that simple cluster definition into whatever google understands and saying hey in google we need some resources which is gk cluster control plane for gk and we will need the node pool and that node pool should be running in those locations and this is how we translate those sizes into sizes of nodes understood by google and so on and support and finally i created the same thing for aws now aws is always complicated there are usually many things that need to be done to do something successful in aws and in case of pks we need a route table and we need subnets one subnet second subnet third subnet and we need a security group for node pool and we need a vpc for not pulling and we need some raw policy attachments and we need what else do we need uh some services and we need some rules and we need the what else node groups and the eks cluster itself now this can be complicated but there is no way around it i'm an expert in aws in this situation it's my job to understand all those things but others do not need to suffer through it as much as i'm suffering so i'm taking all this complexity i'm defining it here and developers should still be able to say hey i just need a cluster in aws i do not care about those things i care about other things i care about the size of the notes and a few other things now if somebody doesn't have that simple file that i showed you initially uh the one that i used when i was pretending to be a developer we could also do cube cattle uh explain and then the name that i defined which is composite kubernetes cluster and i want to do it recursively recursive there we are now if anybody is interested hey what are the parameters i can use to create those fancy clusters that you simplified for me hey parameters are all here look at this section i can just go and say explain to me how that composite works and what i should do and i have all the parameters and all the other details just by saying cubecut will explain i could just as well export this as a documentation now let me change my role again and become a developer the one that actually created the cluster in the first place and let's see what's going on with my fancy clustering aws so cube cattle get composite kubernetes cluster i should have made it a shorter name here we can see all the clusters that were created as composites right with all the complexity hidden and we can see that in this specific case the control plane is already active and node pool is being created those fields over here are actually fields that i defined as an op sky those are the columns that i said hey you should print those columns and the columns are the name of the cluster the status of the control plane and the status of the node pool and that status is coming from aws itself or if i created the same cluster or similar cluster in azure it will be from azure and so on and so forth now let me show you what's really going on what is behind it simple definition which was like 20 lines of yaml you know the one that i used as a developer this is actually what was created i will execute cube cattle get clusters and node groups and im roles and policy attachments and vpcs and security groups and subnets and internet gateways and route tables all those aws resources are being created by crossplane and i can query the status of all those resources by executing simple kubernetes commands and we can see here hey those are all the things this is all that is happening as a result of me applying those few lines of yaml i mean me as a developer and you can see the statuses of all those they're already and all synced except the node group node group is still being created it takes a while in aws until everything is up and running probably like 10 minutes or 15 or 20 you never know so i will wait until my composite cluster is fully operational and i will do that by outputting the composite clusters actually now it's finished we're almost finished node pool was not active a few minutes ago and now it is active so my cluster should be fully operational it says ready for so probably some final touches final actions are being executed but my cluster should be fully up and running in a few moments so let's see if i can connect to this cluster but here actually there is a problem how can i connect to the cluster what is the cube config where is the cube config if you don't know where cube config is that's probably because i was too fast when explaining things it could take a while until i explain everything about composites but what matters is that cubeconfig is stored as a secret in the namespace that i chose to use which is team a so what i can do to generate a cube config that i can use to connect to that newly created cluster is something like this i can say hey within the namespace team a get me the secret called cluster output cube config that secret might contain other useful information but i'm interested in cube config only i need to decode it and i will store that output in cube config yammer and here it is this is cube config that i can use to connect to the cluster and if you don't believe me let's do this export cube config equals whatever is the current directory in cubeconfig.yamo and now i will do something like you couple get nodes there we are this is aws node because i specified that the minimum number of nodes should be one and when i say i i mean i is obs specified that is the default value and i is dev did not change the default values of something else and just to be on the safe side let's say cubecut will get namespaces and we can see that hey this is my cluster now i can use this cluster and all i did was change one or two lines in a jumble that is 20 lines or 15 lines altogether and the next question is hey how would i as a developer manage this cluster myself do i need to call the ops guys or should i do it myself can i do it myself and the answer is definitely yes so let's change something about this cluster and see how maintenance works i will unset the config just go back to the control cluster cluster where crosspen is running and i will edit the same file that i used before which is cluster.yaml so what should i do yeah let's say that one node as a minimum is not enough i want to change my cluster to have a minimum of three nodes how would they do that i would just uncomment this parameter this is one out of three parameters in total that's all i needed as a developer and i would change the value from two to three so let's say that i want three nodes what they do next as i said before if i would be using argo cd or flux which i'm not in this demo but if i would i would just push those changes to git repository and argo cd would do the trick it would do whatever needs to be done by the way if you want me to show you how to do that there is a video i created the video how to control cross plane with argo cd the link is above my head i think anyways there is no argo cd so i will do it manually cube cattle apply the file name cluster dot cluster.jumble and there we go i just made simple modification to the yaml file and applied this manifest that's all i did as a developer it cannot get easier than this i'm having full control over the things that matter to me but somebody else ops defined everything else all the complex things that i do not want to care about as a developer or to be more precise things that i do not know how to care about hey how many developers understand vpcs and subnets and this and that all the things in aws or whatever they are running i mean not many because it takes time to understand those things and not everybody can spend year or years figuring it out that's why we have people that manage infrastructure but in this case those people do not manage infrastructure in this case people who previously manage the infrastructure are now managing services are managing definitions of what infrastructure is and i as a developer can take full control of it actually not full control but control over the things that matter to me now let me go back to the cluster that i just modified and let's do cube couple get nodes did they get the second node or the term node yes almost you can see that the second and the third node were created six and nine seconds ago and they're not yet ready if i repeat this command they should become ready in a few seconds so let's uh let's be patient and let's wait until this is fully operational there we go the second note became ready and now the third one should be ready there we go my changes were applied i as them just change the minimum number of nodes the last thing i want to show you is hey what if i want to delete my cluster what if i'm finished with the project or maybe i just wanted to play around doesn't matter the reason what matters is that if i'm allowed to create a cluster and i'm allowed to manage my cluster i might be allowed or not depending on the rules that we set but in this case i'm allowed to delete the cluster as well so let me delete the cluster so let me unset cube config that points to the new cluster and let's do cube catal delete and what are we going to delete well whatever is defined in cluster dot yaml the same simply yaml that i used to create a cluster and update the cluster i'm going to use that file to delete the resource from my control plane type of kubernetes the one where crossplane is running and then let crossplane figure out how to delete all the stuff so here we are is deleted and now if i do cube cattle get composite kubernetes clusters it's already gone but is everything gone let's say cubecut will get clusters my cluster is still there that means that not all the resources from aws were removed it takes time until you delete things in aws what matters is that the fields ready and synced are now set to false it is not ready it is being deleted actually let's take a look at all the aws resources and here's what's happening right if you remember from before the last time i executed this command i had more resources these are the leftovers these are the resources that were not yet deleted composites might be the best feature of crossplay i already explored in previous videos check them in the description the importance of crossplane it provides a universal control plane that allows us to manage everything with a single api we can use kubernetes definitions kubernetes api to manage everything and everything can be things within a kubernetes cluster it could be clusters themselves or it could be any other type of resources networking storage whatever we might need to manage can be managed with a single api and that api is kubernetes api powered by crossplane crossplane is effectively a universal api for everything one api to rule them all and on top of that we can use composites to combine different resources and enable everybody to use our services in a simple way composites enable shift left ops people or devops people or sres or c sub means or whatever we call the people on the right can define complex operations in a way that is very easy to consume by everybody in your organization if you look at aws as an example creating a kubernetes cluster involves a bunch of resources is a complicated thing to do and i use the simplest possible example if you would add databases and storage and what's or not those hundreds of lines of definition can easily become thousands but we can combine all those resources we can combine all those definitions into composite definitions that are easy to consume everybody no matter their experience with infrastructure or anything else can understand a few lines of yaml if we simplify it and what is really amazing is that that simplification does not come from a software vendor because if software vendor decides hey this is how you deploy applications this is how you do this this is how you do that our freedom disappears to define things the way we want those things to be defined so we are having three actors here three personas we have crossplane as an unopinionated way to create opinions we have ops that deal with complexities and define the opinions themselves and then we have devs or whatever we call people on the left side that consume those opinionated frameworks you are in charge of defining opinions you're in charge of deciding what is what and what is not yet devs have freedom within the things that interest them so devs might not care about subnets but they might care about let's say the size of the nodes or the number of the nodes and so on and so forth you

Channel: DevOps Toolkit by Viktor Farcic

Views: 2,538

Rating: 5 out of 5

Keywords: crossplane composite, crossplane resource model, xrm, composite resource, xr, composite resource definition, xrd, crossplane, upbound, infrastructure, kubernetes, control plane, universal control plane, aws, azure, google cloud, gcp, devops, devops toolkit, review, tutorial, viktor farcic, infrastructure as a service in cloud computing, infrastructure projects, crossplane kubernetes, crossplane upbound

Id: AtbS1u2j7po

Length: 28min 9sec (1689 seconds)

Published: Thu Jul 08 2021