How to Build and Deploy an app Helm Chart on Kubernetes Cluster with GitLab CI/CD

[Music] hello everyone my name is Malik and welcome to my YouTube channel today I want to demonstrate how to set up another gitlab cicd pipeline to deploy Helm charts on cernes class in this pipeline we have a design web page and we will see how to deploy it as microservice containers on cernes cluster by gitlab cic [Music] CD as you know in this days most of the application are built based on microservices not monolitic so each part of application are done by a separate team here we assume we were asked to design a login part of application this one and we need to test it on cernes cluster as you see it is on my local computer also here is the HTML source code of this application we need to test it on kues cluster before deliver it and see if it properly works and at the end of this presentation with entering the IP address of cernes node and access to this login page page so as a designer I design this HTML page and it is ready to test it on kubernetes cluster so the source code here is on my computer so I need to build a container image from the data that I have and push it on container registry then on kubernetes cluster by running proper manifest pull image from the container registry and bring up the container here so we want to do all these steps by using gitlab CI CD as you see in this diagram I'm going to use three repositories for this lab but you can use only one the first one is using to make a connection between gitlab server and kues cluster here I have created a repository with the name k8s Das connection and also this one act like firewall and we can Define on it which repository can have access to cetes Cluster the second one is for recording application data and also act as image Repository here I have created a repository with name k8s Das data for this one and the third one we are going to use for Helm chart repository so let's continue for running this presentation we need to have a gitl account it doesn't matter if you are using gitlab.com or your local gitlab server also we need access to a kubernetes cluster if you have a cloud account on one of the public cloud provider like AWS Google cloud oci and so on you can bring up a cluster and use that and maybe these public clothes charge you also there are another choice to bring up a kubernetes cluster on your local machine if you are using Windows or Mac machine you can install and configure Rancher desktop on your computer really I have recorded a video about how to install and configure runer desktop on Windows machine and I pinned its link about this video and by click on it you can access to it and configure runch your desktop on your computer if you are using Linux obono distribution or Windows sub Linux you can use multipass package to bring up quickly a cetes cluster on your local machine so these two steps are mandatory to run the laps of this presentation in the first step of this lab we have to make a connection between gitlab server and cetes cluster So to that end we have to install a gitlab agent on kues cluster I search for inte R kubernetes cluster with gitlab from the first result you see a document that is written connecting a kubernetes cluster with gitlab if you read it here to connect a cetes cluster to gitlab you must first install an agent in your cluster so click here installing the agent for kubernetes prerequisite before you can install the agent in your cluster you need an existing kubernetes cluster as I told you in prerequisite some minutes ago access to kubernetes Cluster is mandatory also I mention if you don't have access to any public cloth provider you can and bring up a local one on your personal computer or your laptop here is my personal computer and I'm using Windows so I uninstalled WSL on my computer and also uninstalled Rancher desktop so if you install Rancher desktop on your computer automatically you have a kubernetes cluster for testing just type Cube CTL get notes and you see the kubernetes cluster with one node is up and ready to use so back to document on gitlab so and existing kubernetes cluster is ready so going down in installation steps to install the agent in your cluster create an agent configuration file in the first step we have to create an agent configuration to create an agent configuration file choose a name for your agent the name must be unique in the project contains at least 63 characters and so on okay let's see in continue in the Repository in the default Branch create an agent configuration file at the root let's copy this one back to gitlab in the cetes connection repository that we have created for connection between gitlab and kubernetes here it's the rout and just click here new file and put it here here for agent name we have to select a name so let's select k8s connection and at the moment no need to write inside this file and just commit change you see we have a config yaml file and at the moment we don't have any configuration inside this here it's like a firewall and in this file we Define which group which username which repository can access to kubernetes Cluster so in next some minutes I will completely explain what configuration we should Define here let's back to Roots you see here we have created a file with this path here back to documentation here it is mentioned you can leave the file blank for no and configure it later the next step is to register the agent with gitlab we have to register that in gitlab at first how we do that you must register an agent before you can install the agent in your cluster to register an agent we have go to operate kubernetes cluster let's back to gitlab account from the right side going down to operate kubernetes cluster you see it's empty and we don't have any connection to Cluster connect a cluster just click here open and you see the name we have created some seconds ago click on it and register here give you configuration and it is mentioned you have to keep this configuration in a safe plate because if you close this page you will lost this configuration agent access token you can copy this token and use in kubernetes cluster to make a connection with your gitlab or you can use Helm so I'm using this section but let's copy this one and keep it in a safe place let's back to terminal in kubernetes cluster and run this configuration copy the first one and in terminal also here when you bring up a cluster by runch your desktop Helm package is also installed so check Helm it is installed so let's yeah it's already exist here Helm repo update okay update is complete and copy the last one and enter you see thank you for installing your release is name k connection let's back to our repository on gitlab let's close this one we don't need it anymore let's refresh see it is connected so the first step successfully done and the tunnel between gitlab and cernes cluster is up okay now let's test the connection between gitlab server and kubernetes cluster just in Google type using gitlab cicd with a kubernetes cluster and from the result click the first one and here you see the official document from gitlab just go down and you see this one update your. gitlab-ci yaml to run Cube CTL command okay good let's copy this one and going to gitlab server here is my repository that's working as a agent between gitlab server and kubernetes is cluster and this is the file that some minutes ago we have created so for testing just we need to create a new file with name gitlab-ci yaml because we want to test a cic CD pipeline So based on this document let's grab this one copy it and paste it here we are using this just for testing so it is going to use Cube CTL image from bitnami because this image have Cube CTL in install on it so here we have to put our agent name our agent name was K8 s- connection and also we have to find this agent located there just leave it as is and we will see what we have to write here just commit change and see as soon as we save it it will go and automatically run a pipeline so from the left side going to build an pipeline here you see it is failed let's going inside that and see what is the reason look here error no context because we didn't Define the P to agent project if you look here when we run CU Cal config get context it will show us the correct path so let's grab this one copy and going to Pipeline and replace to this one so from the left build an pipeline editor remove this one and replace it here this is Agent name and this is the path to the project this is the unique ID of our username and this is the project name k8s connection so let's commit change and see if it can connect to kubernetes Cluster and get the list of parts click here it is on running status here you see job succeeded and it could connect to kubernetes Cluster and get a list of pod that are running on cernes cluster on default name space let's check it manually on kubernetes cluster in terminal type Cube CTL get pots look here only we have one PO is running on our kubernetes cluster in default name space if I run Cube CTL delet pod and the name of the Pod it will go and delete the Pod here manually you see nothing found in default name space let's back to gitlab server from the left side Pipeline and run the pipeline again to be sure everything is correct click here it is connected to gitlab Runner and you see no resources found in default namespace so far so good let's going to the next step in the next step let's copy our project data to second repository so let's back to other repository as you remember its name was K8 s- data so at the moment it's empty let's clone it back in terminal and just type git clone and the address LS let's switch inside that you see it's empty get status and you see it's up to date what data we need to push on repository at first I need to project resource that I design by HTML and it is located here in SRC LS SRC you see CSS file HTML and so on so to use this project as a container we need to build an image as you know for build an image we need a Docker file here I created a Docker file just cat Docker file let's clean and open it again you see inside Docker file I'm using Eng genix as a base image to show you quickly let's bring up an engine server by Docker Docker run- D D- name web 001 dasp connect Port 80 82 from my computer to Port 80 in the docker and the image Eng genix let's browse it by my browser local host Port 8082 you see welcome to engine so I going to use Eng genix as a base image to build an image for my project let's back to terminal and let's stop this container on and let's remove it let's clear the screen so in my Docker file as you see here I'm using engine X in the second line I wrote hey Docker please copy my resources from the SRC folder and paste them in Eng genix in this path SL user SL share slinix SL HTML Let's test it manually and everything going well we will use this instruction in gitlab cicd to automatically build an image for us let's continue how to build an image manually just type Docker build dasht it means add a name and tag to New Image that you are going to build so I going going to name it sample and tag it with the version one and at the end where is my Docker fire is located just type dot it means is located in the root so just enter you see this image with this tag is build just type Docker image LS grab sample you see in the first line sample version one 40 seconds ago was built so far so good let's use this image and see if everything is correct so Docker run in a state of engine let's using sample version one do ER container is built so let's browse it here if I refresh you see it's my project so at the moment we see how to build an image manually and how it works let's back to terminal and stop this one and remove one nice the next step is how to push this image to a repository that we can use in kubernetes cluster normally there is Docker Hub and we can use dockerhub to push or image there but we don't want to use it because normally every company has its private container registry so here we going to use gitlab container registry as a private registry for our project let's back to gitlab going to left side and in deploy section you see container register at the moment it's empty and we have to push or build image here at first I show you how do it manually and in next minutes you will see how do it automatically in cic CD if I going down you see at first we need to log in gitlab registry so let's copy this one and back to terminal already my credential are exist on my computer and successfully loging to gitlab but if you are first time logging like that it will ask you username and password so in first step we are connected in registry gitlab.com through CLI let's see next step in Next Step It show us how to build an image back to terminal and let's add sample we1 image built just Docker images you see here it is built let's going here and show us how to push it to repository copy and just sample version one you see here it's going to push the image to the gitlab registry there you are it is pushed to gitlab container registry let's check it and refresh this page you see it is here so how we do this steps automatically by cic CD we don't want this image anymore because we need to push it here automatically let's delet it delete done let's back to terminal and copy the project data to the repository you see I did copy them in the repository let's get status get add get commit minus am at project committed and get push they are pushed let's going and check repository quickly you see this folder and Docker file copied here for cicd we need create a file called dot gitlab-ci.yml here we need to Define job and steps now how we configure this file let's back to Mr Google and in Google just type gitlab container registry cicd and from the result click here and see the proper document for this step here build and push container images to The Container registry here this step that we did manually so we need this section configure your. gitlab-ci yaml file let's going down there is some example just we need this example container registry example with gitlab cicd just copy this one and paste it in the file that we have created on gitlab let's back paste it here let's make it more cleaner Define stages we Define which stage we need to use so we name it build stage you can name it everything you want but for user friendly I use build here is the name of or job change it to image build image which image we going to use to run on gitlab Runner we're going to use the docker image which stage we are using this stage build which Services Docker let's remove this one Docker in Docker Services what it means gitlab Runner is Docker container and we going to run another container image inside that container so it's called Docker in Docker container so in escript section there are three commands you remember them the first one is for loging to gitlab registry the second one for build image and the third one for push image to gitlab registry so for the first one Docker login dashu username there are some builtin variables inside gitlab let's make it more clear let's copy this one which username we going to use to connect gitlab repository my username here my username is and which password I going to type my password and also the gitlab registry address in a state of writing all the credential all in text here it is better to using builtin variables from gitlab so let's remove this one for the username there is a variable CI registry user automatically gitlab determine this one because you are logging in your repository for the password also gitlab has your password and no need to Define here just using this variable and which server if using this variable CI registry automatically gitlab will determine it means registry. gitlab.com it's done this the first line done the second one Docker build CI registry when using CI registry as you see also it here it means registry. gitlab.com let's back to terminal and show you something remember this command registry. gitlab.com here it's related to my account in gitlab the name of the repository and the name of image with going to make it so let's copy this one just let's copy it here the first section will return this one registry. gitlab.com so let's remove this one the other section should be copy and paste it here so when this line run it will create an image with this name for us and in the third line it will push that image let's copy this one where to gitlab registry there also we can add an echo message image built let's save it as soon as we save it it will go and run and it will build an image and push it to gitlab repository let's add a message by cicd and commit change this file has been successfully created let's back to section pipelines and here it show you it is running let's click it here build image you see here it's using Docker executor with image Docker you see image build let's check it what it happened run Docker file create an image and then and push that image to gitlab repository and job sued let's go to left side and check deploy container registry and see here the image added here if I click it here you see 1 minute ago it was pushed here so we could build an image and push it to gitlab container registry by cic CD okay let's start with create a chart for or login application and see how it works before that let's see how we can create a chart for create a chart just type Helm create chart name for example hel create give it a name for example I select this name test- login d a just click enter and it will create a fold there with the selected chart name let's go to terminal type Helm create test- login Dasa enter the chart created so LS you see here a folder with this name created if I get a tree of this test application it will show you there are some f file also has been created inside this folder let's see what are this file if I back to my presentation I added a little info about them in the first line you see the chart name that we have selected in create command also in chart folder you see a file with name chart. yaml this file G give us information and description about chart back terminal let's see the content of this file in this file you see API version the API version that this file is using to communicate with kubernetes cluster it's version two in the second line you see name this is the name of our chart and in the third line you see description about this chart you can add everything in this description there is another item called type there are two type of charts one is application and another one is Library here most of the helm charts are application and the default type also is application the version it's about the version of the chart itself but the application version It's the version of your application okay so we have created our chart CD test login application let's going inside templates here are the default manifes that are located here let's remove all the content here and replace them with or manifest so let's remove all of them we don't have anything here let's back from here CD this folder my project manifest are here let's copy all of them to test login application and in temp templates let's install this chart on my kubernetes cluster how we can install Helm chart just type Helm install give it a name here it's called release name and then chart name for example Helm install login application and the chart name test login application let's buy back to terminal exit from here clean screen type Helm in install login application this is selected name and chart name dot SL chart name test Dash login application enter you see this is your release name last deployed is today in which name space default name space because in our manifest we didn't Define any name space and it show us the status is deployed with the revision one so let's check if the application deployed on kubernetes cluster with Cube CTL get pods here you are this is the parts that are deployed and just get deployed deployment this is the deployment name and get SVC the service it uses not Port if I type Cube CTL get node this is the IP address of the node copy this address put it here double points what is the port three 1,000 here you are the application deployed so for remove the application just type Helm uninstall release name done if I go and get Cube C get Services everything down after we prepare Helm chart we have to push it in a Helm repository or in Reverse if we want to install a chart on kubernetes cluster we have to to add the repository name on Helm then Helm can access to that specific repository and pull the chart then install in kues cluster so I need to push it on a repository here I want to push it on gitlab repository let's going to internet in Google if I type push Helm chart to gitlab from the result click on the first one here we'll show you how to push your chart to the gitlab registry as you see here publish Helm packages in your project package registry then install the package whenever you need to use use them as a dependency so for publish or Helm chart here there is a requirement create your own Helm chart so we have it we prepare this Helm chart and use it also in kubernetes so package a Helm chart into a chart archive I will show you how to do it so we can click on this one there is instruction here and I will speak about it in next some minutes let's back also for authenticate to the helm repository you need a token okay so before pushing our chart to gitlab we need to package it look here we have to put our chart in a box and seal it then push it Repository how we can do it just with this command hel package back here hel package test- L-A I want to package this chart just enter successfully package chart the chart package and seal it in a box and also save it here let's ls- l test- L-A with this version 0.1.0 do you know these numbers come from where if I go to the chart and open chart yaml look here version 010 so let's clean the screen exit from this folder and this is the pack package we prepared to push it on gitlab repository let's back to documentation on gitlab with this command kl- request post this is an API command we want to post this package to where to our gitlab repository this is my gitlab account I wanted to use this repository and publish the helm chart here let's have a look to the command again here we have to add our chart name here it's our username and our token and also this is the path for our project for our repository just here we have to add project ID and also select a channel name for the project ID we can grab it from here and also we add our user name and token here so let's clean the screen this is the command that I replaced my user name and token and also added project name and channel name here and just enter you see this message 20001 created let's back to gitlab repository and from the left deploy package registry look here test login application Helm manually published so far so good okay for run Helm steps from build package publish it to helm reposit and deploy it to cernes Cluster I uploaded chart to help project repository if you see the other files here just I wanted to keep my files save here so just attention to the Chart folder also you see two python files here that I will describe you soon for run cicd pipeline I am going to create a gitlab-ci file here just click here new file. gitlab-ci do Yano so let's start to config this file at first I did Define a variable to save Cube context here in the next step I put my stages here in this file I'm going to use five stages I will show you one by one and describe them so let's start with the first one pre-build in pre-build I going to extract data what it means let's going to chart folder and you see a file called chart yaml and I did describe you before in this file you see the chart name and also the version of the chart as you remember and we did manually when we package a chart to publish it in Helm repository the package will use this version in the file name and also we need the chart name for other stuff so at first I need to extract this data from this file so I wrote this to python file and we extract this data from the chart. yamel let's look inside them this is just a simple python file it will go and open the chart. yaml and it will read line by line and we extract the name and version and finally in output it show us the version and the chart name for example here it show us the chart name and the second one print out the package name let's go into CI file so as you see here a stage is pre-build I'm using the python image to run on gitlab Runner and also in script section as you see here I use two environment variable and it will go run both of the file and they will be recorded as a key value data in build. EnV as you know when each stage is finished all the data on that stage will gone unless you save them as artifacts that can usable in another stages normally we are using file in artifacts but if we want to pass variables from one stage to another stage there is a concept in gitlab called EnV if we search on internet for this subject exporting environment variables from one stage to the next in gitlab CI you will find some articles about it so let's going down and here it is mentioned artifacts. reports do. EnV and also mentioned pass and environment Vari to another job so if I click here it will go to the gitlab document pass an environment variable to another job here give you an example so I'm using this example as you see they save it in this file and uploaded this file as a environment variable in artifact so it is usable in another stages but one thing is very important look here each line must be formatted as variable name any value here in each line just we should have one key value or otherwise you will get wrong answer so let's back to gitlab so at the moment you understand what I want to do with pre-build est stage so the result that include package version and package name will save in this file and will be uploaded in artifacts let's go to the next next stage this stage will create a package from Helm chart that can push on package registry I called this stage as build here I'm using Helm image to run on gitlab Runner here there are some Echo messages just I wanted to show you and see the content from the last stage you see here package version and package name they comes from the pre-build stage also the main line is here Helm package and the path to the package package is located there it's located here test login application and we extract this this name from the chart. yml in the pre-build stage so let's back here so it will prepare the package and the next line will push or prepared package to the gitlab helm repository you saw them that we did manually and the last one just it's a message that show us chart package published to gitlab package registry here you can save this file and will automatically run and you can see the result before going to add another stages here let's save it and run add cicd file and commit changes here you see this gitlab C configuration is valid let's going to the left side and and check build pipeline let's click here extract data finish let's see here here you see python file did run and the result save and uploaded in artifact and the next stage it's running to publish Helm package to gitlab repository was run successfully look here message 2001 created so it means package publish to gitlab package registry let's go to helm repository and see what will happen there just here deploy and package registry you see the package published here if I click here you see the package with this name and this version published and uploaded here let's back to CI file at next stages here it's a deploy so at the moment we did publish the chart on gitlab repository and now it's time to deploy this chart on kubernetes cluster so here I'm using this image why I'm using this image because this image included Helm and Cube CTL in the last stage you see just I use this hel because I don't wanted to use Cube CTL and this image not included Cube CTL so I using this image but you can pull this image on your computer and install Cube CTL inside it and push it to Docker hop or another repository and use it as your specific image but here I'm using this one I found this image on internet I prefer to use it so here as you see deployment started in a state of this one I can use this variable that I have created at the top of this CI file just I add this two line to show you so this line is not needed anymore and in the next line you see Helm repo add also I did show you before how to add a repository in kubernetes cluster so with this line we will add repository and in the next line we update repository and just I use this one to see in the outputs the list of repo also this is not mandatory you can you can uh comment this one also in the next line in a state of using Helm inall release name and the chart I'm using Helm upgrade D- install what is different between Helm upgrade Das Das install and help install is if for the first time it go and run Helm upgrade normally it could find that so it will go andinstall it in the next time if you run this CI file so it will check and if exist just it will upgrade it if there is any change inside the chart files so finally this command will install the chart on on kubernetes cluster and the last two commands will show us in output a status of the parts and SVC that has created by Helm also there is another stage for destroy the application that we going to build so let's put it here but this one not run automatically so when you want to run this stage you can run it manually and you will see how to the how to do that so here we have four stages so it seems this one is extra let's remove this one here you see pipeline syntax is correct and it is ready to commit just add a commit message here deploy Helm chart on on kubernetes cluster commit changes let's click on this one see pre-build stage build deploy and Destroy and each one is running after by another one the pre-build done build done deploy also is done let's click here look here Helm upgrade dash-in install look here release test- login Das application has been upgraded or installed deployed look here with Cube CTL get pods it show us they install and running and in the services you see test- login Das SVC the name of the service so if I going to terminal here just type Cube CTL get node-id here the IP address of the node grab this one going to browser and also let's check the service port this is the note port and let's back here and add it at the end of address and enter look here so you see how we did deploy or chart from es scratch to deploy it on cernes cluster let's back to gitlab and check pipeline look here a still one stage is remaining to run if I click here at the moment if I want to uninstall the chart and remove the application from the kubernetes just click this one so it will go and remove and remove the application from kubernetes cluster by using Helm look here it's done if I go inside look here they are terminating and also the service is gone if I back here refresh page you see it's gone so we arrived to the end of this tutorial I hope you liked it if you like it please comment your feedback in the section below see you in another videos

Channel: be-devops

Views: 5,013

Keywords: Kubernetes, HelmCharts, GitLab, CICD, DevOps, ContainerOrchestration, Automation, DeploymentWorkflow, GitLabCI, K8s, Containerization, DevOpsTutorial, TechTutorial, ContinuousIntegration, InfrastructureAsCode

Id: qSbE7NOXm8U

Length: 61min 30sec (3690 seconds)

Published: Sun Dec 10 2023