ULTIMATE CI PIPELINE ON GITLAB | LIVE DEMO|GITLAB END TO END PROJECT |#devops #cicd #devopsproject

hello everyone my name is Abhishek and welcome back to my channel so I hope you are all having a great CI CD week we already did two videos one is with Jenkins shared libraries and the other one was with GitHub actions where we used a self-hosted ec2 instance as a runner for GitHub actions but this video unlike the other two this is going to be a end-to and CI setup so this is not end to end CI CD because last time I did with Jenkins and it took almost 90 minutes and many people felt it very lengthy so that's why I'm divided dividing it into two parts this video will cover gitlab CI and the next video we will do the CD with Argo CD which is more or less similar with what we did with uh Jenkins so this this part is very important rather than the other one because deployment you already watched uh in one of my previous videos that is Ultimate CI CD pipeline using Jenkins but watch this video till the end so that you understand how to use gitlab as your source code repository along with creating the CI with gitlab itself so if you look at the diagram here I I am going to explain you all of these things today and I'll try to keep it as informative and as simple as possible firstly I'll show you how to create an account with gitlab I think that most of you already know but still we will try to create account and we will import a project into gitlab so you can use my project whichever I'm going to show and after that I am going to explain what is the Java project that we are using and what exactly it does so it's going to be a spring boot micro service because everybody is asking a project on microservice so I decided that I am going to explain a spring boot micro service Java project then I am going to show you high level understanding of what the spring boot application will not take more than two minutes where I'll explain you what happens if you execute this project after that we will do SCI explanation so I'm going to show you how to do and how to create a CI file in the gitlab but imported things are after that where how do you create a runner for gitlab see you are writing a ci5 for gitlab but how do you execute this CI in Jenkins for example if you people are familiar with Jenkins what you usually do is you create a worker note for Jenkins right whereas gitlab offers you two things similar to GitHub actions one is you can use uh gitlab using the self-hosted runner which we are going to do today and the other thing is gitlab also offers you the runners so these Runners are hosted by gitlab and they own the runners so I always tell you it depends upon your project where you want to choose the gitlab hosted runners or you want to use the self hosted Runners but most of the organizers organizations use the self-hosted runners so that's why today's CI demo will be on the self-hosted runners we will create an ec2 instance and we will add the tc2 instance as a runner so I'll show you the configuration for adding the runner as well but not just creating the ec2 instance we have to do a lot more guys here because we need to install Docker we need to install sonar Cube and configure the sonar Cube we need to provide the configuration for the gitlab runner and finally we also have to configure the traffic rules so I'll show you all of these things how to achieve and then like I told you we will use the docker executor so what is Docker executor on top of this ec2 instance which you have configured as a runner for gitlab similar to Jenkins where you create an ec2 instance as a worker mode but depending upon your requirement you can use a Docker agent or you can use a virtual machine agent similarly because using Docker agents or Docker executors is the most efficient way because they are very light in weight in nature and you can configure your Docker executor for different projects so that's why I'm also going to show you how to create this Docker executor on the ec2 instance that you have created finally I'll explain you the CI file and also show you how to run the CI file so the part 2 of this video will cover this part where we will see how to upload this Docker image and once the docker image is uploaded we will use Argo CD to deploy that onto the kubernetes cluster so this part I have already explained the exact same part in Jenkins and 90 percentage of it will remain the same for gitlab as well so that's why this part I'll take it in a different video where this part needs special attention so that's where I'm going to talk today again so this is the end to end cicd pipeline that we are going to cover where the CI part will be till here what we will cover is how to use the gitlab as an account I mean as a source code repository and how to put your code you can use any of your Java application but I would suggest to use my Java application that I am going to show you here because it's a spring boot Java application if you already have a springboard Java application which is in a good State then I don't mind you can also use that then I'll show you how to write a poem.xml for it and how to build that project using the pom.xml where we will use the maven I think of course uh most of you are waiting for this to show I mean to me to uh sorry for me to explain uh how to use Maven as a build language in gitlab so I'm going to show you that what are the main event targets that are required how to use the gitlab CI file for using this Maven targets and finally we will use sonar Cube I will create my own sonar Cube server on the ec2 instance that we have created and we will do the code scanning for this Java application that we have written so this is what we are going to do as part of CI and in the cd part we will cover the docker image how to update the docker image and how to deploy it onto a kubernetes cluster using Argo CD if you want you can add more stages here you can also do codes code security here using trivi or you if you have any sash Dash those are all you know once you understand this entire process adding more steps is just like adding one more item to it but nothing more than that so it is very important to understand this part today and probably once you understand as an assignment you can take it and add one more stage for trivi or probably if you want to use a sneak you can use sneak for Docker image scanning after this part whatever you would like to so because we have to cover all of these things uh let's quickly go on to our gitlab so if you do not have a gitlab instance okay or if you don't have see gitlab can be self-hosted or you can also use the open source version of gitlab.com so what I will do is firstly show you how to create an account with gitlab.com again this this is not a rocket science all that you need to do is go here subscribe for the get great uh get free time and once you do that it will ask you for simple information let's say you don't want to provide all of these things and take a lot of time you can also do use this one called GitHub so you can sign in to your gitlab using your GitHub authentication okay this is a very simple thing that gitlab offers you and I'm also using this one where I have signed into gitlab using GitHub or you can also use your Google accounts perfect once you do that you will get a gitlab UI something similar to this where I have logged into my gitlab using the Abhishek veeramalla I am viramala which is my GitHub account okay so I have logged into gitlab using the GitHub Authentication so all of these things there is no extra effort required all that you need to do is click the buttons and go there but after this comes the interesting part where do you get the Java project from there are multiple resources on the internet for Java projects but most of them what I've heard from you people where you have uh try to use some other Java projects they don't get executed or you run into some issues which you have to fix it by yourself instead use this Java project that I am going to show you here which is a spring boot Java application I've written this Java application which is a very very simple one and I use the same Java project Even in our ultimate CI CD example as well so if you are new to our Channel let's say and if you are not aware of the ultimate CI CD pipeline that we have done on Jenkins which contains the CI CD everything right from installation of Jenkins to to deploying the application onto the kubernetes so you can follow the ultimate CI CD pipeline go to my channel go to the video section and this is this is the one here within one month there are 53k people who have viewed this and trust me hundreds of people have implemented this pipeline you can also check my LinkedIn where people have continuously posted that Abhishek thanks for this I have implemented this one using the example that you have shared on the YouTube so you can also try this if you are aspiring a devops engineer and looking for Jenkins examples you can take this as reference don't worry I'll put the link in description as well now I am trying to use the same example same Java project so what is this Java project on a high level this is a very simple spring boot application even if you see here so what does a spring boot micro service do a micro service is a very minimalistic or lightweight or independently managed application which can be deployed created destroyed without the dependency of the other micro services for example you can consider if amazon.com is the entire website you can take login as a micro service log out as a micro service where if tomorrow a developer at amazon.com wants to make a change to the login application he can do it without disturbing or without making any changes to the logout application and your Ci or your CD that you have built CI CD solution will redeploy with the new version of the login application that the developer has changed right so this is how a micro service and the CI CD of a micro service is managed so in this case what is this this is a very simple test application I have also provided the steps of how to run this application locally let's say before executing the CI CD pipeline you want to try it locally again I have shared this entire thing in the Jenkins ultimate cicd pipeline I have showed you by running this application so that's why I'm not taking extra effort to show you show you this example one more time because that will be duplicated for the subscribers who already know this okay so the requirement is you need to have Java 11 on your local machine if you want to try it if you don't want to try it and if you want to proceed with the CI CD example then there is no problem so this one here there is SRC folder and I have created a simple package called Abhishek and inside this there is a very simple start application.java file and what it does is it will render or it will get few values from the template and it will print when you execute this one there will be a very simple HTML page it's a static HTML page which will render this information what is the title what is the message and it will update you the user who is trying to execute on the port 8081 it will say I have successfully deployed the end-to-end application using Maven and I have deployed the application onto kubernetes using arcosity even if you run this on your local you will get the same information because I have hard coded this and rendering few values like this is a value that controller is passing to the view so this is the control and if you go here in the templates this is the view where I am receiving these values right so this is our spring boot application works let's not go into the details of it because it's irrelevant to the video and it might disturb the viewing experience of people who want to learn cicd but if you still want to understand this you can go to my Jenkins ultimate cicd pipeline video and you can understand this flow as well so once you get this repository uh you can simply Fork this repository you can start this repository uh once you create your gitlab cicd account just Fork the repository you can use this Fork button and you will get this entire project onto your gitlab organization or onto your gitlab namespace so like I told you this is what we are going to do today uh this step is done how to create a gitlab account how to import the project and I have explained you the Java project as well it's a spring boot Java project then comes adding Runner and creating the CI file so I'll take the add Runner part first so that I have to do a lot of configuration here right why are we adding a runner again to take the example if you want to compare it with Jenkins in Jenkins what you usually do is you create your Jenkins instance and if you want to use the master for running the application you can do that but most of the cases what you do is you add a worker node for your Jenkins and you execute your Jenkins pipelines The Groovy scripting Jenkins pipelines on this worker node sorry sorry again so you will execute this Jenkins Pipeline on the worker nodes but coming to the hosted Solutions like gitlab or coming to the solutions like GitHub actions you have two options one is you can create your own Runner which is equivalent to worker node in Jenkins or equivalent to agent in Jenkins or you can use the hosted Runners that are provided by gitlab or GitHub which is an added Advantage right you don't get that Advantage with Jenkins but gitlab or GitHub is offering you that Advantage okay so how do you take that Advantage it's very simple uh if you go to the settings option here okay then go to the CI CD option here I hope the screen is visible I'll just try to increase the font then you have an option here called runners okay so expand this section for runners what you will see is two options do you want to configure your own project runners or do you want to use the shared runners okay so if you want to use the shared Runners you can use it if you want to use the project Runners you can use it as well so it's up to you whatever you would like to use either use the project runners or use the shared Runners depending upon your project uh for if I have to put that in a very easy way probably if you're using an open source project or if your code is public then you can simply go for the shared Runners because there is no problem with using shared runners in that case because your code is already open and you know what are the security concerns that will Rays if you are using the shade Runners whereas if you are an MNC if you are an organization you should not be using the shared Runners because shared Runners are hosted by gitlab and you don't get the complete information of it you don't know where are the shade Runners you don't know what exactly is happening to your code with the shade Runners so it is always good to have your project run as configured and this is a registration token which is secret but don't worry once this demo is done I'll reset the registration token so there is no problem for me to share the registration token done so you understood the concept of shared Runners and project Runners right now let's use the project Runners because it requires additional configuration it requires you have to do a lot I mean you have to do a lot of things with shared Runners sorry project Runners so that's why I'm going to show you the project runner's example so firstly go to your AWS account and create an ec2 instance and keep in mind that this ec2 instance should not be a free time ec2 instance because it cannot run with all of these things like you know you need to configure Docker executor you need to have a sonar Cube that is running on it so one CPU and one ram memory is not sufficient so what I'll do is whenever I'm launching instance to understand or to demonstrate the gitlab CI gitlab CI CD I will use the T2 medium which is next one uh do not even go for T2 large because this is a demo videos and you are trying to practice you are trying to simulate real-time environments so there is no requirement of you using T2 lats you don't create a lot of dashboards in sonar so that's fine so I'll call this one as gitlab the ec2 instance select Ubuntu and here choose the instance type not as T2 micro but choose the instance type as sorry yep T2 medium select an existing key value pair I don't mind creating one more key value pair uh but that's fine I can use GitHub actions demo uh cicd pipeline that we have demonstrated in the previous class so I'll use the same GitHub GitHub actions demo key value pair but if you want to create one more you can do that as well launch the instance after that comes few important things that is what are the ports or what is the traffic that you need to open so I already know what traffic to open I am going to explain you but if it is your first time you can understand this uh traffic rules once you configure all of these tools but I'm going to explain you right now that if you are using Sonar Cube sonar Cube runs on Port 9000 right so you need to open the port 9000 and moreover there is a communication between your ec2 instance and gitlab right so if this is your AWS and this is your gitlab there is some sort of communication that needs to happen probably if gitlab has to uh I mean your project has to run on ec2 and then the runner the ec2 instance Runner using Docker executor it has to send some information back to gitlab saying that oh okay I executed and most of the things have passed or few things have failed so this information ec2 instance has to send back or the runner has to send back to gitlab so for that reason you need to open port 9000 for sonar and also you need to open few ports that is HTTP and https traffic on the AWS instance for inbound and outbound traffic rules so this is exactly what I'm going to show you right now so go to your instance then in your instance there is option for security groups go to the security groups here instance ID after that what you'll do is go with the security tab and inside the security tab you have inbound and outbound traffic rules right so firstly let's start with inbound traffic rules like I told you in the inbound traffic rules uh where is this one security groups yeah so in the inbound traffic rules I'll open port 80 Port 443 and also I'll open the port 9000. so for the purpose of demo what you can also do is edit the inbound traffic rules and open all the traffic because it will take a little amount of time so I just want to save that time and I have configured all traffic but do not do this one in your demo part or in your project or the GitHub uh sorry the readme file that you prepare or LinkedIn summary that you are going to post do not say that open all traffic but say the exact ports that are required to be opened so save rules I have opened all the traffic for incoming uh ec2 instance traffic and then I'll do the same thing with the outbound so that way you can also allow the traffic that is Flowing from the ec2 instance to gitlab edit outbound traffic rules add rule just say uh all traffic uh coming from any IP address and save the configuration there was oh okay uh there is no problem uh already you have the similar traffic rule that is configured so now every incoming traffic and the outgoing traffic is configured for my ec2 instance now go to the ec2 instance click on the instances section right now my ec2 instance is up and running and the security groups are also configured so let's proceed with the installation pattern now why do we have to install all of these things I hope you already understood because we are going to perform the CI stage where we need maven we need sonar Q for code analysis and how do you get this Maven so we are going to use the docker executor what is Docker executor I have made one full video of using Docker executors or Docker agents as your uh you know worker nodes in Jenkins or if you're talking about gitlab as your executors for running the CI process in gitlab or GitHub the reason for that is if you are using this Docker executors or if you are using this Docker agents against the traditional virtual machine agents you can get a lot of Advantage One is security second is configuration you don't have to configure a lot of things again and again in multiple VMS and because these are lightweight in nature they can run multiple pipelines parallely right if you have one worker node let's say uh that you have configured for Jenkins you can run only two pipelines or you can run only two jobs similarly if you have only one virtual machine that you are using for gitlab as a runner the problem would be if one Java application is running on it then you cannot run the other Java application because of some uh probably these two Java applications have different versions or these two Java application have different dependency packages so they'll mess up with one another if you are using a Docker agent or Docker executor the advantage is that you can use the same single virtual machine that I've created and run hundreds of pipelines or run hundreds of jobs so that's why always go with the docker executors or always go with the docker agents irrespective of the CI solution that you are using whether it's Jenkins GitHub actions or gitlab anything so I'm not bothering about installing Maven or configuring Maven because the docker image that I am going to use as executor will already have Maven I'm going to show you how how is that as well I've already created that Docker Maven image and I've also used that in the Jenkins example and I'm also going to use today that is a Public Image so you can also use it there is no problem so let's proceed with the first steps of installation here where is this one perfect uh what are we going to install first the ec2 instance is configured now let's proceed with the installing Docker why we need to install Docker because we are going to use Docker executor on this uh specific ec2 instance so to create a Docker instance you would need Docker to be running on that ec2 instance so let's get the terminal if you already know uh some of these steps then you can forward the video uh just that you don't get uh you know bored with the video or just that you don't uh yeah just forward the video okay so what is the instance configuration here uh the IP address let's get the IP address perfect now let me use the same uh key value pair that I already have and log into the instance perfect so I have logged into the instance it will just ask me for the fingerprint authentication I'll say yes and now I am into the ec2 instance so let's proceed with the creation of Docker of course first of all you need to update the packages before you create any uh I mean you create any package or you install any package you have to make sure that your app purpose or your relevant package manager reports are updated it will not take much time so the app proposal updated after that sudo apt install Docker dot IO okay now I have installed Docker while this gets installed let's go and see what will be our next step in the figma so our next step would be to install and configure sonar Cube many people have this problem of okay now how do I install configure sonar Cube I don't know the steps for it so I have a very minimalistic steps that will install and run sonar Cube on that ec2 instance and I will make sure that the docker executor that we are using for running the pipeline talks to the sonar queue so I'll show you how that also happens you need some authentication uh you need the sonar Cube API key and you can store that as a secret as a secret not as a variable I'll also show you how to secure the sensitive information in gitlab so I'll store that API token in a secret and I'll talk to sonar Cube and Maven will push the generated dashboards onto the sonar Cube so we'll see all of these things but how do you do that so I have detailed steps here if you go to uh the repository and if you go to the readme file this is a repository go to the readme file and scroll down whatever I'm explaining here everything is part of this readme document let's say you are not able to follow this video then you can go back and reference the readme document as well but coming to the sonar queue part I have very clear instructions on how to install sonar Cube these are very minimalistic I don't have any database that is attached to sonar Cube and this is a very easy way of installing and configuring sonar Cube let's go back perfect Docker is installed now firstly I need to install unzip package so let me go to the root user so that I don't have to use sudo all the time first is app install unzip so I've installed Diamond zip then create a user for sonar Cube let me do that uh select a password done then so now there is a user sonar so let me go to that user okay I want to run sonar cube with that user so I'll go to the user called sonar Cube perfect then wget get the required package for sonar Cube this package has everything to run the sonar Cube server this is a ZIP file so I have to extract the zip file so I can use the unzip command that I've installed before perfect now once this is done what would be my next step is to Grant permissions to this folder because I have I'm going to execute a few things in this folder okay now I moved to the package where the sonar start server script is present so this is a script for starting the sonar server all that I need to do is use dot slash sorry uh I can just say dot slash sonar dot sh followed by start the same instruction is available in the document as well now here you should see that sonar Cube starting and failed to start sonar Cube okay no problem let's see why uh sonar Cube has failed okay I understood the reason why sonar Cube has failed because sonar Cube has a prerequisite for Java and we did not install Java okay so for that reason what you need to do is go back I have mentioned that in the document but I have missed while explaining so you can install Java as a prerequisite which I forgot no problem let me go back to the root user install Java it will not take much time Java is installed okay once Java is installed you can verify the Java installation and go back and start the sonar server okay many people are asking me I'm not able to start the sonar server this can be possibly one of the reasons uh unfortunately I also ran into the same error fortunately or unfortunately in the demo and I realized that is because I did not install Java Okay so go back to the sonar Cube user sudo Su hyphen sonar Cube so I'm back to the sonar Cube user now what I'll do is let's go to the folder where the installation script is present bin I am on the Linux so I'll use Linux folder start sonar dot sh sorry sonar.sh star now it said sonar has started you can verify the status of sonar using the status option it said sonar cube is running now sonar Cube by default runs on the port 9000 that's the reason why we expose the port 9000 all that you need to do to verify is go back to your ec2 instance what is the IP of your ACP instance followed by so use that http IP of my ec2 instance colon 9000. so this way I have logged into the sonar Cube I have created the sonar Cube now I'll quickly show you how to get the API token and use that in gitlab and within next five minutes our CI part will be done I am going to show you the maven part explain the former.xm sorry explain the pom dot exam XML and also the maven targets that we are using finally how to push this onto the sonar Cube the dashboard the default password is admin admin but once you log in it will ask you to uh configure a new password for sonar Cube just for the security purpose choose anything that you would like to the old password has to be admin and then you can enter the new password Here uh I think I mistyped it perfect I have updated the sonar password and right now there are no dashboards in it if you see here in projects there are no projects right now that are available and I am going to create a project for the Java application and there will be a dashboard which says how many number of errors are available in the project go to my account here go to security enter name of a token that you would like to why I am creating this token again it is very important for you people to understand why are we doing these steps so why are we doing this because in the CI step what we are doing is we are asking sonar Cube to perform code scanning or we are asking sonar Cube to perform analysis on our code if sonar Cube has to do this then it needs to have authentication to the source code that is present in gitlab right because gitlab has to push the configuration uh to sonar Cube so that's fine what I'll do is I will create an authentication token API token in sonar Cube and Grant that permission to gitlab okay so gitlab generate token this is a token that got generated don't worry I'll delete this on our Cube instance so there is no problem sharing this token with you all people just copy this store this somewhere perfect now I've stored the token the next thing that you will do is go ahead and secure this password as a sensitive information in your gitlab but before I explain that many people might not understand the CI process itself so I'll take some time in explaining the CI process then I'll show you how to secure sensitive information that is in your CI so unlike uh Jenkins in Jenkins what we will do is inside the root of the project we'll create a file called Jenkins file whereas in gitlab we'll create a file called dot gitlab ci.aml in GitHub we'll create a folder called workflows and inside that we'll put your workflows any number of workflows that are present so depending upon the CI tool that we are using there are different processes in gitlab like I told you you will create a file called dot gitlab hyphen ci.aml and this is the CI file that I've written I have commented the cd part intentionally because this will be part of the next demo if you want uh if you people already can follow the Jenkins Pipeline and if you think that CD part is not required then put that in the comment section so that I will not waste your time and I will not reiterate I will not repeat the steps for the CD process one more time but the CI part is divided into four stages so what I have done is I said I want to install code quality okay let's come from the top so this is the image that I want to use as a runner okay so what I want to do here this is the image that I want to use as a Docker executor now don't worry uh we haven't created the docker executor on the ec2 instance there is a reason for that firstly I want to show you with the shared runners or okay that's that's also okay I can after explaining the CI file I can show you how to add the executor onto the ec2 instance but this is the image that I'm saying that I want to use as a Docker executor the reason for that is I have created a custom image which already has Maven installed so that I don't have to install Maven again and again or people who are following the demo they can use this same thing the reason is even in your project you might have hundreds of java applications and everything is used in Maven why do you want to reinstall Maven again and again so that's why create a Docker image instead of using base images like Ubuntu latest or something create your own Docker image which you to use as a Docker executor or you can use my image as well just re-tag it and you can use it and then in the variables you can provide the information whatever you would like to in our case I want to just provide the information of sonar variables like what is the sonar host what is the authentication token and if you see here I have provided a variable here and where does this variable get referenced from this gate reference from your CI CDs secrets in gitlab this is one additional feature that gitlabs offers you that you can put your sensitive information inside the environment variable sorry inside the CI CD variables all that you need to do is go to the settings tab click on cicd take it in a new tab here if you scroll down there is a option for variables just expand and you can save your variables here in my case I have stored the sonar API token I was doing a demo for CD as well so I have a Docker username Docker password every sensitive information that's part of your CI file you have to create a section called variables and you have to store this section I mean store all the environment variables under this section perfect after this if you have anything in the before script what is before script before executing your CI okay if you want to perform any step you can put that as part of your before scripts in my in my case the source code is not directly available in the root folder it is available inside a folder called Java Maven etc etc followed by Spring boot app so that's why I switch to the directory where the source code is present so that every time inside all the stages I don't have to repeat now finally the important part what are these stages right so this is the key part of your CI CD pipeline stages are nothing stages are basically you are trying to build your cicd pipeline right so entire cicd pipeline will not be built in one stage so you will divide it into multiple pieces first piece will be check out the code second piece will be build the code what is build use the maven Target to create your Java artifact the third step will be to deployed somewhere or run the static code analysis or run linting or run you know some additional uh probably dashed sash settings anything that we would like to you will divide into multiple stages for basic understanding let's say you are a beginner I'll take one simple example uh let's say there is a login application for amazon.com and you are writing the entire cicd pipeline for login application what happens sorry what happens is for this login application you will try to build multiple stages to verify that you are seeing that your build process is right and more than that the changes that developer has made are valid for example developer has made one new validation to the login page you have to verify that existing tests are passing only then your CI stage will pass you have to verify that there are no security implications that this code is bringing into bringing into your existing application or you have to verify that the build is successful and you can create a jar file War file or any particular file out of it so this is your CI CD multiple steps in my case I have a build stage exactly using Maven I'll build a jar file then I have a test stage where I'll test the application to verify that the code change does not disturb any existing unit tests then sonar Cube stage where I'll pass the dash I'll pass the application to sonar I'll execute some static code analysis steps and I'll push the dashboard to the sonar Cube that I have just created dockerization I don't want right now because I don't need to uh I don't want to demonstrate CD as part of it so just let me comment the cd part very quickly so just remove this dockerization part I think that's much easier so right now there are only three stages uh let me commit this one perfect now it is committed the only step that is required before executing the CI is I have to use Docker executor on this ec2 instance if you go back to figma we have created Docker we have created sonar Cube we have opened the traffic rules only thing that is left is the gitlab runner now how do you do that go to your gitlab again and go to the CI CD section inside settings here you have something called as Runners as I explained and there are detailed steps that are available just click on showrunner installation instructions our runner is a Linux Runner so let's get the terminal back let's go to the root user sudo Sue hyphen we are to the root user now execute the steps as it is you don't have to change anything here and your Runner will be configured just grant the permissions for your Runner binary then add some uh you know comments what is this Runner what are you exactly doing then install the runner service and finally we need to start the runner service okay so to start the runner service what you will do is you will execute this command and register the runner with your gitlab okay these steps are just copy paste you don't have to worry about it only thing is just replace the registration token with the registration token that you see on your screen here now the gitlab runner is also configured it will ask you for just some information which you want to overwrite what should be the name of your Runner do you want to make any changes just provide the runner as a Docker Runner that's the only thing and press on the enter the default Docker image sorry this is again something that you have to do the default Docker image should be the image that you are using as part of your job okay because this is your Docker executor image right provide that information and now it said that everything is configured right now you can start using the docker executor let's see if the pipeline gets succeeded now to succeed the pipeline what are we doing firstly we are using the mvn clean package to build the package after that we are using mvn test the maven Target to test the package and finally if you notice here this is the sonar Target which is using the sonar host URL which have which we have configured in the variables as sensitive information and this is the password that we have got from the authentication token what will be the URL ec2 instance colon Port 9000 that's it now go to the uh project and click on cicd pipelines and just try to run the pipelines or make any code change the pipelines will get executed automatically anything that you would like to so already it is running because we made a simple change right we commented the dockerization part but that's okay I don't want to show you with that example I can run the pipeline one more time so that you get clear understanding of what is happening under the hood foreign okay I don't want to pass any variables just click on run pipeline now the pipeline will run and you will notice that there are three stages that it is trying to build the build build States test stage and sonar Cube stage okay so this is currently in pause pending State because there is another pipeline that is getting run uh that was because of our previous change that I've just explained I commented the dockerization part right so because of which the pipeline is running so go to the pipelines and you can kill the previous pipeline or wait till the previous pipeline execution is done go to this one here and you can stop it from here as well just click on Cancel running so that you don't have to wait stop pipeline so as soon as this pipeline gets stopped the other pipeline starts running because the executor is free right just give it a second I think it's still taking time to realize that the previous one uh but I can rerun as well but that's fine see now it started running it says in progress or running so while this gets run once the first step is successful only then it will move to the next step because that's a default behavior of the cicd pipelines and we will see that build step will be done test step will be done and also the sonar cube step will also be done but if you want to see the what is happening in the build job just open this in a new tab and you can see what is happening here so Maven was trying to get all the required information to build and finally it said that the job is succeeded it provided everything that is part of the pom.xml to the uh spring boot web application and it has built the package now if you go back you will notice where is it here sorry if you click on this one one more time you'll notice that the next step should also be done by now yeah unit step unit test is also done again when has executed the mbn test and finally once this is done I'll show you a new dashboard on our sonar Cube meanwhile we can see what is getting executed here so this is the command that is getting executed what is this command this is the same command that we have passed in the cicd pipeline so see here that GitHub is not showing the sensitive information because we have secured the sensitive information if you did not secure the sensitive information anybody who is running this pipeline they can see what is your sonar host URL and what is your sonar Cube API token which is a security thread okay it says the sonar Cube server was not able to reach perfect I know the reason why because I did not update the pipeline uh with the latest sonar Cube URL I just used the previous one when I was trying the demo by myself so all that I need to do is go back to the cicd environment variables here one more time and update the password sorry update the URL variables go to uh the sonar login one here reveal variables that's fine and edit so update the API token with the new API token first one what you will do is go here my account I think I have already shown but I think I forgot to update no problem I can try it one more time API token generate copy this API token and replace that in the cicd environment variables I think I explained but I forgot to update update variable awesome and inside your CI CD file as well you need to update your sonar host so this one here I need to update the sonar host URL which I forgot right so you can open edit in ide get your sonar Cube URL the one here go back to your easy uh this one IDE gitlab ci.aml and update the one and just commit the changes I am fine to commit directly to the main branch because this is just a demo and now the CI pipeline will automatically run I don't have to run the CI pipeline because I've made a change right so the CI pipeline will automatically run while this is running and I'll come back and show you that the ca has successfully completed and I'll show you the new dashboard in the sonar but before that let me just take a minute to explain the pom.xml so usually pom.xml is not written by the devops engineers just to be clear but in some organization devops Engineers write the pom.xml but they can take the help of developers okay what's inside the pom.xml is the dependencies that are required to build your application the reason why devops Engineers don't write this form.xml is because devops engineer might not have complete picture of what the dependencies are required okay so that's why it is essential for you to collaborate with the engineers the developers and understand what goes into this CI pipeline okay or what goes into the pom.xml but this information you can write by yourself what is the artifact name what is the artifact ID what is a version that you would like to this one here so that this gets saved the artifact the jar file War file that you have created get saved with the same representation com.abhishek dot uh springbootdemo colon 1.0 in your artifactory whether it's Nexus artifactory whether it's j for jfrog artifactory in my case I don't have the artifactory I did not create one because my end goal is to create a Docker image but not create the jar file War file I want to take this jar file profile and put that in the docker image so I don't have to use Nexus or any other artifactory so now let's go back and see if the build job is successful or not so go to cicd pipelines and let's see the latest pipeline if it is successful we should see a dashboard in sonar Cube awesome it said passed click on the pass and see if each and every job is successful obviously it will be so build job is successful unit test job is successful code quality job is successful go to the code quality job and you can get the dashboard URL instead of directly searching that on Sonar Cube so it says this is the dashboard URL just open that and you will notice that there is a new dashboard which says the code gate status the quality Gates have passed and there are no bugs there are no vulnerabilities with my code of course because I've written a very simple application and I don't want to embarrass myself in front of all of you just kidding yeah so that's why there are no bugs but in your real time you will notice some bugs you'll notice some vulnerabilities and this dashboards can be used to fix the vulnerabilities and moreover if your new change is bringing in some vulnerabilities and bugs you can also inform your developers or fail the ca pipeline saying that there are new bugs with the existing CI pipeline with the existing code commit that you have made so this is the demo which we have covered with all the CI aspects right from installation configuration I hope you enjoyed the demo and uh put that in the comment section if you want to take a look at CD or you can watch my Jenkins ultimate cicd Pipeline and you can do Siri by yourself anything I'll take your input which is very valuable to me and I'll decide if I have to do that video or not but thank you so much for watching the video really appreciate your time and the cicd week was very special for me as well because I got to just revise all the cicd tools again Jenkins GitHub actions and gitlab for myself and I've also explained to you people so that everybody is benefited that's why I keep saying keep learning keep sharing thank you so much if you haven't subscribed to my Channel please subscribe to my channel Abhishek let's grow together and let's build a wonderful devops Community thank you so much I'll see you all in the next video take care bye

Channel: Abhishek.Veeramalla

Views: 22,999

Keywords: devops interview questions, devops mock interview, devops discussion, devops, devops round table, aws, azure, k8s, tanzu, vmware, openshift, redhat, gcp, openstack, aws complete devops project, devops proxy interviews, #bts, BTS, bts, Ansible, Ansible FAQ, Ansible Q&A, Devops interview questions with answers, terraform, terraform interview questions, terraform Q&A, git, github, azure git, bitbucket, stash, gitlab, azure devops interview questions, devops interview questions for experienced

Id: 8NYAHkTuQdc

Length: 47min 39sec (2859 seconds)

Published: Sun May 07 2023