Full GitOps Tutorial: Getting started with Flux CD

and we're going to go to parts in our stable system name space and here are all our different parts now these parts these vulnerability report parts they just they have one-time parts they're jobs okay they're basically kubernetes drops that run one time as you can see they're proof gun now now last thing i want to talk about is would i keep using flags and i defined four key measurements here first one is the community then design ease of installation and documentation and i want to talk a little about that just what i would say like it's just there just works like as simple as that which is i know it's like such a low bar it just works but it's amazing when tools just work so [Music] hi there and welcome back to my youtube channel now for those who are new here my name is anees and usually i talk about kubernetes on the cloud native ecosystem now in this video it might come as a betrayal to everybody who's so fond of ico cd and i thought i'm going to be forever using igd as my choice for githubs and continuous delivery now in this video i want to introduce you to flux now up to now for the past one and a half years i've kind of avoided using flux for several reasons now the first one is that when i tried flux once right at the beginning when i got started with get ups i was completely confused i had no idea what was happening i had no idea what i was doing i had no idea how to use it now that experience was such a deterrent for me to try out flux any further so it just stuck with argo cd also there were some other factors such as the company at the time that i was working at was integrating heavily with aggressive d and igor cd has this really fancy ui so you can click buttons instead of nice commands in your terminal however that is not the git ops way now if you're completely new to getups let's quickly revise what is skit ups actually why do we use this when do you use it and how do we use it now let's assume this is our git repository okay get rapport and we have here our kubernetes cluster and we want to get the code from our git repository the plot to our kubernetes cluster now in our git repository we have the code itself but we also have our cumulus manifests for example those could be helm charts there should be it could be customized files it could be anything else and we want to get this manifest deployed to our kubernetes cluster now in our cuban use cluster we have several namespaces let's just draw them and several different tools several different cloud native tools installed now what we can do and what you usually do at the beginning when you get started with kubernetes is that you just do cubecal and apply right apply and then you specify the file or the resources that you want to apply and this is called the imperative way to deploy our kubernetes resources to our kubernetes cluster so we basically sell kubernetes through the cube cuddle cli to take our kubernetes manifests and apply them to our kubernetes cluster now the thing is if anything changes with an occupant this manifests our class i won't know about it and we will have to do cube color apply let's assume you're working within a team right you have several people who all do changes several different teams maybe even and they all do changes to your git repository on different branches or even to the cubane's manifests now they don't want to go ahead and maybe merge things right so one team is maybe deploying one change and then they want to do a cube cuddle apply so they go ahead and do that and then the other team wants to do the exact same thing right with their resources and in the end nobody really knows what's actually running within our humanities cluster nobody knows who deployed what who deployed what what is deployed when did it get deployed or anything in between so you don't have any answers to these questions or they are really annoyingly difficult to find out now the problem is at 3 am you want to know what had happened if either one of the features that got deployed at 3 pm broke or if the team australia deployed something while you were sleeping and that broke things in your cubanian's cluster right you want to know what's happening so you can fix it as quickly as possible you want to have all the insights possible into your kubernetes cluster and it's only possible if you have the answers to those questions now what do you do with github's tools such as argo cd such as flux you deploy an agent into your kubernetes cluster now with flux i think they are called controllers so you have different controllers running inside of your kubernetes cluster that all help you to manage the resource in your kubernetes cluster and what you do instead of this instead of let's cross this out so you don't do this what the workflow looks like instead is that people make changes to the git repository okay they make changes just to the code then you have a ci cd pipeline that is running within your repository so for example github actions right it's running and let's say things get deployed to the main branch they pass all the tracks everything's good they then get pushed to the main branch and then they update the cuban needs manifest for example to version 2.0 of your container image or whatever it is right so you have that and um once a change occurs there then the agent living inside of your cluster is responsible to pull that change let's do this pull this change right let's move this a further up that we don't do want this we don't want this anymore so the agent pulls this change into your kubernetes cluster and deploys it in for example the app name space and how does the agent know about your cubing manifests well the administrator of your kubernetes cluster is the only person who has access to your kubernetes cluster okay and this person tells your cube i need this question about this where the cuban needs manifest live okay this is the only person who's allowed to tell to point your cubans manifest to the agent okay this is how the agent knows about those kubernetes manifests only the cluster admin can do this this way anybody else all of the engineers here all of the engineering teams don't need access to your kubernetes cluster they should not have access to make changes with new companies cluster they could maybe have access to specific namespaces to test new versions new features or something if you don't want to have a separate cluster for those for the staging or test environments um but nobody else should have access to kubernetes cluster and the agent is responsible to detail who deployed what so who made the changes to the code which is then reflected in the manifest and got deployed in your kubernetes cluster what is deployed region 2.0 okay it's clearly supposed to tell you what is the plot in your kubernetes cluster and when did it get deployed now there are two main tools that help us do this the first one is argo cd and the second one is flux okay i'm not going to go into the politics of these uh flex is an incubating project maybe our city is about to graduate maybe it's also an incubating project i have to cross-check that but ultimately these are the two tools that you can use now the thing is argo cd has a ui and i showed that in several different videos where it says flux is cli based now the thing is they both have different design choices okay so with aguzidim you have different options of telling your kubernetes cluster about your resources with flex it's really just that path and i'm going to go into the details further and what i think about it towards the end of the video so if you want to know my feedback my insights go to the end of the video okay now in this video we're going to look at flux previous videos checked out iqcd so what is flux flux is the git ups family of projects so you have basically the githubs toolkit that's part of flux now i just got started with flag so i have this one tutorial and to a tutorial in this video that i'm going to show you it's based on the documentation so we're going to dive right in i'm going to show you how you can set up flux install flux in your cubenese cluster you can use any kubernetes cluster it could be a micro case mini cube docker desktop kubernetes class it could also be a professional managed amazon aws managed cuban needle cluster it could be any kubernetes class it doesn't matter okay we're going to install flux on the kubernetes cluster we're going to tell it about a helm chart specifically our starboard helm chart now what is starboard now what is starboard starboard is within the aqua security open source project is one of our open source projects starboard is specifically used to scan your kubernetes cluster for vulnerabilities and misconfigurations so if starboard is installed within you can use cluster you can scan for vulnerabilities configuration audits uh cs benchmark reports pen tests all of this can be done through starboard it just basically lives inside of hypnosis cluster and whenever you install a new application for example it checks that that deployment for vulnerabilities inside of your kubernetes cluster so it comes as a kubernetes operator that we can install through a helm chart for example there are installation options and it comes through a cli command so that you can use for example cacd pipelines now we're gonna use the helm chart and it's gonna be managed through flux then once we have the hamstride installed we want to deploy an application now we're gonna deploy the application through kubernetes manifest we're gonna tell uh flux about us kubernetes manifests i'm going to show you in both cases how you can install those resources either through the flex cli command or how you can deploy them through kubernetes money fast so you can define everything declaratively within kubernetes manifests now lastly we're going to set up reporting to our slack channel through flex it's pretty straightforward and then lastly i'm going to show you how you can also set up monitoring for your flux resources through prometheus and grafana so this is the entire tutorial where we that we're going to cover in this video now like always if you prefer the written version of these tutorials i have a blog post linked below that guides you through everything that i'm detailing in this video all the different commands with the output everything that i'm covering in this tutorial this is the written version that's linked below now i'm gonna cover everything here in this video you can check out all the commands everything that i'm doing right through the blog post if that's your preferred way of consuming content also please please if you like these kind of videos and you want to make sure that other people see these videos too and you want to signal me how much you like these videos and that i should make more of this kind of content then make sure to give the video a like thumbs up also hit the subscribe button to stay up to date with new videos on my channel and hit the bell notification icon so you know whenever i have a new live stream coming up or new videos now let's dive right into the tutorial now as you can see i was connected to my kind tracy kubernetes cluster but i wanted to create a completely fresh cuban in this cluster so you can follow along now i call this flux cd and now i'm connected to my kind flexi d cube anita's cluster so if i say cube cuddle get notes i can see my kind notes hopefully here we go now this is a completely empty kubernetes cluster there is all the default stuff installed but nothing else so we want to go ahead and we want to use flux now in this case you can go ahead and you can install flex for example through homebrew so you can say brew install flax cd now i already have flex installed i already have access to flux this will become important throughout this tutorial you can also then add flux to your to your profile path of your terminal now i'm not going to do that right now next you want to go ahead to specify your github token now where can you find your github token i'm going to show you so where can you find your access token your personal access token to your account you can go into settings and then tokens on your github account and you can generate a new token now this token is going to be for flux so we're going to put that as like that and i think it needs access to the repository and then access to [Music] my user what else are we going to give it access to i think that's pretty much it it doesn't need access to anything else not for now at least and now we're going to generate the token and this will display my token however then you go and you say export github token and you paste your token right here which i'm also not going to show you now lastly we will need our github username so you specify your github username this is my github username and then once we have everything defined we can check whether or accumulate cluster is actually suitable for flags so we just run a few checks before we get started so checking private seats cuban eaters frozen blob pro-req receipts checked awesome so now we can use our flux bootstrap command so we have here our communities cluster and we have here our git repository and here in the cli we're gonna run the flux bootstrap flux bootstrap command now what it does it creates a git repository and in that git repository they are the cubanetes manifests the same time installed into a flux system namespace inside of your kubernetes cluster okay so it both creates this repository and the flag system name space and if any changes occur within a get repository it's gonna again then deploy those changes to your kubernetes cluster okay that's ultimately what the flux bootstrap command does oh here we go so we can go ahead and we can check out our kubernetes cluster and specifically we're going to check out the namespace flag system so as you could already see there are a few controller parts being installed so our notification controller our source controller our customized controller our home controller now when you hear customize you might think about customize as in the tool customize which is a bit confusing because at the beginning i thought that flux is related or implementing or building with customize for some reason but then i really couldn't find a direct connection between flux and customize and how customize is shown within flux so after all of this long story short i think it's just using the name for this resource customize okay so it's just a resource type customize and the resource type is basically um any specific resource could be like a home chart resource it could be a git repository resource i'm going to show you some of the resources in a second through the cli that you have do you have the helm controller as well so these different controllers that's what you have inside of your kubernetes cluster now lastly i have here the repository that got generated through the bootstrap command so as you can see it just has its resource clusters my cluster flux system um resource so basically in the flux system namespace with my cluster that's where these resources got installed that's what i think um that's what i think happened um now uh this basically used to install flux through flex which is an interesting thing i think that's what's happening here um now let's go ahead and tell flags about our starboard ham shirt now how do we know about the starboard helm chart so within starboard we have here our operator the operator can be installed for helm now usually you would specify first of all the repository where you can find the home chart then you would update all of your repositories you have really delayed this changes of all your home charts and then you would say helm install style board operator aqua operator in the namespace tablet system okay this is the usual process of how you install a hand trade now what we're going to do instead is we're going to first add a source type to flux so this is our source type flux create source helm starboard operator based on this url in namespace tablet system now we don't have the namespace system yet so let's create a cube cuddle create namespace starboard system this is our namespace and here we're gonna install the flux source we're gonna create that okay so it's of type home repository so as you can see you're creating custom resources through flex okay so we can look now form home repositories helm repositories isn't that how it's called helm repository let's do this again namespace and then we're going to go to starbucks system and we're going to look for helm repositories and here is our starboard operator so as you can see it's ready and it has the latest revision of starboard operator which is amazing so we have access to our starboard chart um starboard helm chart now we want to create through flex a helm release for our operator now this is how you do it you say flux create home release starboard operator chart starboard operator and then the source is from the helm repository the starboard operators we called the helm repository object that we created starboard operator that's what we're referencing that's the source that's used here in a namespace tablet system this is what we're doing okay this is all we're doing so we're waiting for this resource to be reconciled meaning that basically flux takes the helm operator let's go ahead and draw this as well so if here's our chart registry and i hope you can see this let me just cross check that you can see it okay this is our chart registry okay in our child registry we have our different home charts so this is the aqua chart registry that are all available and within here within the virtustream we have our sabor chart now we told flux please pull this and deploy it to our starboard system namespace starboard system namespace that's ultimately what flux does flux not the bootstrap command okay this is ultimately what we told flux to do now we told it to do it through the cli and usually when we used aggro cd and we told it to do things through the cli it wouldn't automatically make updates unless we specifically tell it through the cli to do it now flux is a bit different and i'm going to show you that in a second and that it does make automatic updates so now we have the release so we're going to check our cube we need this cluster again and we're going to go to parts in our staff assistant namespace and here all our different parts now these parts these vulnerability report parts they just they have one-time parts they're jobs okay they're basically kubernetes jobs that run one time as you can see they are proof gun now okay so what have they done they have created vulnerability reports now we have to go to all of our resources and now we can check for vulnerability reports because it's vulnerability reports are in the flag system namespace because they belong to flats so we've created vulnerability reports on flux basically on the flex resources on the flex parts these are the vulnerability reports that have been created as you can see here are the different vulnerabilities in flux you know in the in the blog post link below i compared it to the vulnerabilities that starboard found within argo cd resources now just tldr flex vulnerabilities are a bit lower um for security newbies this is kind of one point of indication of like okay which resource should i use but it should it gives you higher confidence and a tool right if there are lower vulnerabilities found in that tool so these are the vulnerabilities that i found if there are online vulnerabilities it might be that those one abilities that there might not be a fix available right or that it just can't really be classified as like high medium low critical um yeah so as you can see this kind of was done through 3v 3v is used to scan vulnerabilities you can use it to scan vulnerabilities in container images file systems your infrastructure's code git repositories so you can also use it for your darker files for your terraform files to scan for vulnerabilities now through starboard tv is used to scan for vulnerabilities inside of your kubernetes cluster to make it super easy for you to know how many vulnerabilities you have right so we have managed it through flex okay we have installed starboard full flex which is amazing so now we want to go ahead and we want to also deploy an application now the thing is that you can do the same thing that i just showed you but also through kubernetes manifests okay so we could deploy our application through this kubernetes manifest where we tell flux about our git repository to create a resource git repository where we link the git repository that we want to install and then upon linking it we create the customization resource okay so this is kind of the source the git repository and we want to exclude we want only want to include the manifest files only them and ignore everything else and then we can create a customization resource that installs the manifests from the git repository and then it's it's basically installing it into the app namespace so we can use this resource instead now here for instance i've created a resource where you can install the helm chart the home repository in the helm release the way that i used flux cli to do that right i use the flux cli to do this but you can also use this kubernetes resource and now with the alerts and notification provider we're going to look at that in a second so i want to go ahead and install the application yaml so i'm going to say um cube color create namespace app that's the first thing i could also specify the app namespace as part of my manifest but i didn't do that so now we're going to say go ahead and say cube color apply file application yaml and this created our good repository visas and our customization resource okay so we can go back to our cluster and look at our cluster and check for our git repositories and as you can see here is our react app in addition to our flux system when we install flux we have also our react app and then we have our customization resource okay now the customization resource here has been deployed so we should now see in our app namespace we should see two parts running okay and these two parts have a container image running right they might not contain image and we can see them in our vulnerability reports we can see that the tag is 9.0.0 these are vulnerabilities there are seven high vulnerabilities and two unknown vulnerabilities within that container image now what i want to do instead is i want to downgrade that image right so we're going to go to our git repository and we're going to change that container image okay this is like this update would be made like mentioned up here would be made through cicd pipeline for example you will make the change or you would make the change within your git repository to this manifest for example and then the agent or the controller in the case of flux within your this cluster say controller is responsible to make that change okay here we go okay so we're gonna go to our deployment and we're gonna make the change to our container image and we're gonna commit it change container image and now that this change happened we have to wait until flux sees that update right and is spinning up the new parts so we're gonna go back to parts and this might take a while because in our resource we said that we want to check our resources every five minutes so this might take up to five minutes before flux is checking our resources again and it's gonna see if the resources within our git repository are actually the same as within our kubernetes cluster so for example before we had within our cubing needs cluster we have the app 9.0.0 and here we said that we want to actually run the app 8.0.0 right so what flux is going to do it's going to compare both of these it's going to compare them and i'm going to see oh the actual version inside of our cube needs cluster is not the desired version that's defined within git so git is always the desired state the state that we want and our git repository is always the actual state okay this is how git ops works kind of so let's see okay see it's just terminating our part here from our old version and spin up the two new parts of our of our new old of our other container image so if i go now to vulnerability reports vulnerability report okay you kind of have to be in the right namespace ability report so as we can see there's a new vulnerability report in addition to old vulnerability report and the vulnerability for port for tech 8.0.0 as you can see it has a lot more vulnerabilities than what we had deployed before which is not good so we want to make go back to our updated version of 9.0.0 now as you can see starwar makes it super easy to compare different versions and then see how your vulnerabilities hopefully decrease over time as you modify your container images right now we have both running right we have it's nice we have both starboard running as well as our application so we lastly want to create an alert for flux so what we're going to do to create the alert is we have to create a secret right we want to get alerts inside of our slack um channel about any updates that happen within our kubernetes cluster so you need the slack api web hook so you literally have to go to your slack account to the apis to the api section and enable the web hooks and then create a url it's pretty straightforward but you have to go to the api dot slack section not just select settings now api dot slack settings and then you can find your web hook so i'm going to replace this with my web hook now to create a cuban needs soup grid for this web hook next we need our notification provider so this is our notification provided using the flux api and it's for slack and we are specifying our secret that we just created and we want to deploy that within our kubernetes cluster our notification provider now once we have the notification provider we then want to create an alert and the alert basically tells us if anything changes in our git repository our customization resources of our home release because these are the resources that we really care about with starboard and our application that we just deployed right so we're gonna deploy our alert.yaml file as well alert and once we have that once we have our alerts let's just double check that we have everything within our communities cluster alert we have our flux system on call web app uh it's ready good we can use it i guess um now we want to change our app back to what it was before where was version 9.0.0 since that had fewer vulnerabilities and once you make this change we hopefully get notified in slack that this change happened right so if somebody has access to my git repository makes a change that results in changes inside of my cube needs cluster i should then be notified about it in slack right so we're gonna go to namespaces we're gonna go to our app namespace as you can see the new ports have been spun up the old one is gonna get terminated and then we can go to all of our resources um and we can go to our vulnerability reports and then hear all the vulnerability reports again so they are basically unchanged because they've already been scanned like this container image has already been scanned so tv is not going to rescan it or start but it's not going to be scanning with tv now let's check slack right so as you can see here's the flex channel and there are lots of messages because i deployed it or tested in one cuban needless cluster and then i deleted the kubernetes cluster and flux started to complain about it for some reason that it's basically the app can't reach it anymore and here is our new revision deployment app react configured so it will basically tell me if there are any changes that occurred right within my resources that have been specified in the alert now lastly flux is providing really nice guides on how you can set up for example the prometheus the cube prometheus stack through flux the stack that we use the helm chart basically that we used in the previous videos to use for me fusion grafana so you can set that up you can set up the gridfinder dashboard and then you have this amazing flags dashboard where you can monitor everything that's happening inside of your kubernetes cluster now this is a really like straightforward guide so i'm just going to link it below but also in the in the blog post that's linked below and you can also then fill for specific metrics and everything that's going on through flex now last thing i want to talk about is would i keep using flags and i defined four key measurements here first one is the community then design ease of installation and documentation and i want to talk a little about that so first of all the community i got stuck installing the flex so first of all the community i got stuck installing the starboard helm chart through flex because in the installation command through helm you specify the helm registry differently to how i had to specified through flags so that was a bit confusing then i posted on twitter on my twitter if anybody can help me and somebody told me hey just comment in the slack channel for flags so this in the cncf slack channel there's a specific channel called flux where you can jump in and ask questions somebody told me hey just go there and ask a question so i posted my issue there i created an issue on the flux github repository outlining everything that i had done everything that wasn't working so people had the information of what i was working on why it didn't work what i tried already and so on like it's really important people have that information right so somebody was able to help me like within minutes just by posting that in the slack channel which was amazing it's really amazing to be so supported within the community if you get stuck right because getting stuck is really frustrating and then you might want to just don't want to keep using a tool if you get stuck right now i'm going to outline why i got stuck in a second then the next part is the design so from my experience a lot of tools what they do is they have this vision right this amazing version of like this is going to be our design this is what we want to do and it's basically that they have a set of features and then you're supposed to use those features right and why users use those features and while maintainers and the people creating the tool the project managers and so on build and use these features they realize oh there are gaps within our design they're things that are not working how we envision them to work or we forgot about things things are not working or maybe our design has been a bit off or you know anything along those lines and what they then try to do is fill back the gaps they try to fill the gaps of like okay this didn't work let's patch it with this or like okay we tried this it didn't work or that's missing let's try to fill this in right so you have those really incomplete tools or like tools that make design wise not a lot of sense or that are just unclear over time of like what is it trying to go for or like what is the best use of it because some tools have like so many different use cases or ways of using it that you can easily get confused or that the tool basically has best practices you know when somebody tells you this is the best practice of using this tool it's likely that the tool itself is unclear of how its intended use is supposed to be right so there are already issues in that right i don't have to write a blog post about how something is the best use case if there's only one use case for it like if i know that flex can only be used in this certain way and this is what it does if i do this in this command then that's what's gonna happen and i don't have to write a blog post but this is the ideal use case for it right so flux in its design i have been amazed by it because i feel like that the people really thought about what they wanted to create they thought about it and they started from from the pure minimum let's say and then build upon it right they said okay this is our design this is what we envision we're gonna build this and then we can put additional features where it really makes sense on top what if additional features are wanted and needed we can put them on top but this is our core design so i think there's very little ambiguity ambiguity um on how you can actually use flux of what its intended uses now so community and design is amazing now ease of installation it took me a while it took me a while to get started because the documentation offers several different ways of installing it i not even really separate different ways of installing it the problem is really that you don't have a clear installation of like this command installs flux and then you use this command to install an application and here's how you link to the documentation right there's no clear path from installing flux to installing your resources there's a getting started guide but it kind of conflicts with the other guides so it's a bit irritating of where you're supposed to get started and what commands you're supposed to use for what that's why the ease of insulation it feels like the documentation to be honest is written by an engineer it feels like there are little details between installing it like little steps that i have to think about that the documentation doesn't tell me if i have to think about those different steps you give me room to make arrows you give me room to screw up don't give me room to screw up please because i'm going to screw up and i'm going to get mad and i don't want to get mad so please like fill those little gaps right like observe what i would recommend them is observe how people get started with flux what do they do what do they struggle with what paths do you say as jumping between so different different files within the documentation back and forth to figure out like what is the ideal path because it wasn't clear to me that's like the main criticism i would give to flags like once you figured it out it's a breeze to use it's very straightforward in using it um that's just what i would say like it's just there just works like as simple as that which is i know it's like such a low bar it just works but it's amazing when tools just work so yeah that's it um those are really my two cents let me know in the comments below if you used flux and agua cd before what is your preferred tool what tool do you prefer and why which tool do you want to see more tutorials on should i make more tutorials on argo cd should i make more tutorials on flex i will probably use flux more in the future so you might see more tutorials but also do let me know what type of tutorials you would like to see on flux i really hope this was useful if it was again please do hit the like buttons this tutorial takes me a long time to make also please subscribe to my channel for upcoming videos i hope you have a really amazing day and we'll see you in one of my next videos bye you

Channel: Anais Urlichs

Views: 36,745

Keywords: DevSecOps, Kubernetes, cluster, docker, container, Kubernetes Security, DevOps, Cloud, CloudEngineer, container security, vulnerability scans, Cloud Security, Kubernetes for beginners, Kubernetes tutorial, Custom Resource Definitions, security reports, monitoring, observability, security by example, devsecops tutorial, kubernetes tutorial, helm, kubernetes helm, helm charts, GitOps, GitOps best practices, GitOps deployment, ArgoCD, FluxCD, ArgoCD UI, Helm Charts, GitOps workflow, Flux

Id: 5u45lXmhgxA

Length: 42min 11sec (2531 seconds)

Published: Thu Apr 14 2022