Kubernetes CI/CD Pipeline Using Jenkins | DevOps Tutorial/Project - 2023

hello welcome to today's tutorial today we're going to cover a complete devops pipeline this is a full Hands-On tutorial everything will be live it's designed for beginners so we'll walk through all the stages that are required to build a complete uh devops pipeline this includes building the artifacts creating a container pushing the container doing some Security checks and uh with that said let's actually walk through what it is we'll be doing so the pipeline that we're building today is once a developer pushes the code into an SCM in our case we're going to be using GitHub we'll push the code into an application Repository Jenkins will then be notified that there's been a new application update Jenkins will then build and test the application in our case we're using a Java application so we'll use Maven to build the application to perform some tests once the application has completed building we'll analyze that application using a static code analysis tool called sonar Cube this will help us identify any potential code code smells security issues and help the developers remediate any potential problems before code is pushed into production if everything is successful we'll build and push a Docker image into our Docker Hub registry at the same time we'll also scan the docker image with trivi using a SCA or a static analysis this will identify any additional vulnerabilities that may reside within the container so this is another Security check before the image gets pushed into our production kubernetes cluster we'll then update a set of manifest files these manifest files will reside in a separate Repository in our case it's going to be a devops repository it'll contain a deployment manifest as well as a service manifest and we'll update these with each successful pipeline build to have the latest version of the tag once the tag has been updated Argo CD which is a continuous delivery or continuous deployment application will be monitoring that repository for any changes as soon as a new tag is detected it'll deploy the latest version of the tag into our kubernetes cluster we'll then send a slack notification if the deployment and build is successful we'll also send a notification if there are any problems during the build steps along the way we'll do this using a declarative pipeline in Jenkins so we'll walk through each of the stages and I'll explain in as much detail as possible to keep you informed so please fuel up this is a live real-time tutorial which means that it is going to take some time I do encourage you to watch the entire video because we'll get into some of the specifics we'll address some questions that may potentially come up on interviews for folks that are potentially looking to get into a devops role so with that said let's jump right into the Hands-On portion of this okay so the first thing that we need to do is we need to set up Jenkins in my case I'm using individual virtual machines but you can install this all on a single node or a single virtual machine but to simulate as close to production as possible I'm going to go ahead and use individual virtual machines I've also gone ahead and created DNS entries for each one of the IP addresses so that we can use reverse proxies as well as TLS certificates again to simulate as close to production as possible so the first thing that we need to do is we need to sign into our first virtual machine where the Jenkins UI will reside and then become root so in my case that is running on this particular IP address and then we're going to become root the next thing that we need to do is we need to make sure that we have updated our packages and then install Java and we can just copy and paste these commands directly from the documentation and I'll leave the documentation uh Link in the video description below so let's update our packages let's go ahead and install Java again you here you can use openjdk I've chosen to use adopt-ins Tamarin version 17. um and you can copy and paste these commands or please feel free to use open jdk if that's something that you prefer so let's copy that command paste that here and then we'll go ahead and update and install Java so as you can see I've already installed this in the past so it doesn't need to upgrade since it's already running the latest version next what we need to do is we need to install Jenkins so to do this we're going to use the official repository from Jenkins directly and we'll install the latest LTS of the long-term support version of Jenkins so to do that let's go ahead and configure our Repository so we can copy these commands here and then we'll want to add our actual packages here so let's go ahead and copy this okay once we've done that we'll want to update just to make sure that we can fetch the latest versions and then we can go ahead and install Jenkins now this may take a a moment or two um but I'm going to leave this recording going because as I promised it's going to be a live unedited version of a tutorial just to give you a real world sense of how the insulation goes as well as a complete devops pipeline uh if you're enjoying this video um I'll take this opportunity to ask if you could please give it a thumbs up a subscription is greatly appreciated um it also helps the YouTube algorithm so it looks like Jenkins is almost completed here and if we go back to the documentation and we look at the next steps we'll need to start Jenkins and then we'll need to go ahead and do some configuration so let's see if Jenkins has now finished installing which it has so let's start it and we can take a look at the startup to see if it's uh finished starting up by looking at the status so let's look at the status of Jenkins and we can indeed see here that it is active and running so now that's uh completed we can access our Jenkins user interface using the IP address by default Jenkins runs on Port 8080. now keep in mind we haven't done any reverse proxy configuration yet so we do need to access this via our IP address so if we were to go to http 192 68 and it's running on 10 port 8080. we'll see that we can now see Jenkins now the initial password is stored on the file system and we could we can retrieve it by taking a look at this file here so let's copy this and cap that file so this will be our initial password so let's add our initial password and then we're just going to go ahead and install the suggested plugins this may take a few moments to get installed so what we can do in the meantime is we can proceed with the documentation to start installing nginx and setting up our reverse proxy so we can access Jenkins through a domain name so if we go back over to our documentation we can see here that we have a a guide that's specifically designed to help install the reverse proxy so here what we need to do is we need to install nginx so let's go back to our terminal window and install nginx once nginx is installed we'll be able to access our uh our server on the IP address and we can see that it is running so here we actually accessed our Jenkins application directly on port 8080 but now what we can do is we can actually just hit the IP address and we'll see that we get the welcome to nginx so next we need to go ahead and configure our nginx to point to Jenkins through a reverse proxy so to do that we need to create a configuration file so let's copy this let's create the configuration file and I've already gone ahead and created a a sample config file just remember to change your domain name in my case I'm using my domain name that I've already created a DNS record for so let's copy that and paste that in here we'll then want to test to make sure that the configuration is correct but before we do that we need to enable it and we can do that by creating a symbolic link for the configuration file into our sites enabled so let's first do that and then let's test to make sure that there's no typos in the configuration and everything looks good so the next thing we need to do is we need to restart Jenkins perfect so what this now means is if we go back into our browser window we should be able to access the server using the domain name so if we go to jenkins.dev.dman.cloud and by default it does try to redirect it to https so let's make sure that we we change that to http and we can see that it is indeed bringing us to our Jenkins installation as expected so we're going to ignore this just for a second because we're we want to complete our setup so if we head back over to the window where we had originally kicked off our plug-in installation we can see the plugins have finished installing and it's now asking us to create our first admin user so let's go ahead and create an admin user and be sure to set a secure password because we are going to be exposing this to the Internet so we want to make sure that the password is not easy to guess and then we'll give it a email address and then we can save and continue so since we have already created a domain name that's mapped to our Jenkins we can change this by default it's going to use what we had access this with originally but in our case we know that we're going to be using a domain name and we're also going to be using the certificate so let's just get a head start on that and we can configure this to use https and we know that it's going to be using the domain name of Jenkins in our case jenkins.dev.dman a cloud remember to include the trailing slash so let's save and finish so we can now hit start using Jenkins but before we move any further let's go ahead and add a certificate to secure our installation so what I'll do is I'll close out these two windows and then we'll head back into the documentation and there's a tutorial here which uh shows you how to configure Jenkins with TLS now a lot of this work we've done by installing nginx uh previously so we can skip down to the section here where we need to install certbot so let's copy that let's install startbot and then we can skip the rest of this section because we've just confirmed that nginx has been set up since we just did it a moment ago and then we just need to go ahead and create a certificate so you can use this command again just make sure you use the domain name that you're using in your setup so let's copy that and then we will need to provide our email address and then accept the terms of services we can say no to sharing our information and then this will now go ahead and request and install our certificate into nginx this usually only takes about 10 to 15 seconds depends on how busy the uh the servers are so we can see that it's already facial certificate it's deployed it and everything is now running so if we go back to our browser we can now or we should be able to hit our Jenkins installation on our secure URL and we can indeed so you can see that we've now got the the security lock or the padlock here and we have a valid certificate so let's sign in and the next thing that we need to do is we need to configure a agent for us to execute our jobs on it's best practice to not run any jobs on the machine that's running the user interface now there's lots of different ways that you can do this you can have a dedicated virtual machine that runs jobs you can use Docker this can also be installed inside of kubernetes for the sake of this tutorial we're going to actually use a separate virtual machine that'll act as the agent node and we'll configure that using SSH so if we head back into our documentation we have a tutorial here that shows us how to add an SSH based agent into Jenkins so what we need to do is we need to sign into our agent machine and run these steps so let's go back to our terminal we will exit out of the Jenkins UI and then we will now sign into a another virtual machine that I've created that's going to act as our agent so we can see that we are now signed into this second machine so let's go ahead and become root and just like before we want to make sure that we've got the latest versions of the packages that are installed so we can run a quick update followed by an upgrade and then we want to run our agent as a specific user we don't want to run it as root so let's create a user in our case we'll use the user Jenkins so let's add a user Jenkins set a secure password now remember this password because you will need to use this in the future and we'll call this the Jenkins agent and then we'll want to make sure that we Grant pseudo privileges to the user that we've just created and we can do that by copying this command the next thing that we need to do is we need to install Java on this machine as well so just like before I've got the installation instructions here this tutorial talks about Java 11 but we'll change that to use Java 17 but the steps are the same so let's just copy the adoptium repository and add that and again we are missing wget here so let's install oops wget on this machine and then now we can add the repository and in this case what we're going to do is we'll update but we will install version 17. and once that's installed uh since we're going to be building and pushing Docker containers we'll want to make sure that we have Docker installed on this machine as well so I do have instructions here uh these instructions are actually for Ubuntu Linux so what we'll do is let's just head over to docker's official webpage and copy the commands to install Docker on Debian 11. so let's just search for this and we get presented with the steps that we need to run here so let's go through and copy these blocks of code foreign I'm not getting into the specifics of the installation of the tools here because we want to focus on building that pipeline and then let's update and then finally we can install docker now once this is installed we do want to make sure that we give privileges to the Jenkins user and the way to do that is to make sure that we add the Jenkins user to the docker group if we scroll down to the bottom of this page there's some post installation steps so here what we can do is create a a group called Docker and in our case this has already been created so if I run this it'll say already exists and then we want to add the Jenkins user here into the the docker group and then we can run a new group Docker as the user Jenkins so right now we're we're signed in as root so let's go ahead and become Jenkins and then we can run Docker run hello world just to make sure that we're able to successfully run Docker and we can perfect so now that we have Docker installed let's head back to our documentation we've already added the the non-root user here so the next thing to do is we need to connect our two virtual machines to communicate via a remote SSH ssh agent so here what we need to do is from the Jenkins UI we need to sign in to our Jenkins agent so again the UI is the first in the first virtual machine that we set up that's the one that has the user interface and the agent is where we'll be executing our jobs so let's clean up our screens here a little bit just to make it easier and I'm going to open up two terminals so on the left hand side I am going to sign into our Jenkins UI and then from here we want to connect as the user Jenkins to our agent host and this is also that we can get the SSH Keys set up correctly there's lots of different ways that this can be accomplished this is just the easiest way that I found uh that works so let's sign into our machine here and we've signed in as Jenkins so the next thing we need to do is we need to create some SSH key pairs so let's copy this command and let's get those key pairs set up we're not going to use a passphrase and we're going to change the permissions just to make them a little bit more secure okay and it looks like that we may have a typo here but that is okay that's not a uh not a big deal we just missed the the dot Pub here so we can foreign not quite sure what happened there anyway so the next thing that we need to do is we need to take a copy of our private key so let's cut the private key I will need to keep keep note of this now the other thing that I want to do is just to make sure that the SSH doesn't keep prompting us for uh assign it on the first time let's go into our Jenkins agent which is the one that will be connecting to the UI for it to run its jobs so I know we're already signed in here so what we can do is from this machine Let's SSH as the user Jenkins but this time we're going to go back into the user interface virtual machine so let's go ahead and sign it in here so this just ensures that we've gone ahead and created the note host and I didn't set the uh a password yet so we can go ahead and sign into this machine and let's create a password for Jenkins and we need to do that as root so now that we've done that we can log into that machine okay perfect okay so the next thing that we need to do is let's head back over into our Jenkins UI and we're going to manage Jenkins and we're going to manage nodes and clouds and the first thing that we'll do is we'll disable any executors from running on our user interface so let's click on configure under this guy and make it zero and then if we head back up to nodes add a new node and then let's give it a name so let's call this Jenkins agent we'll make this a permanent agent and then the description is optional will allow two concurrent executors to run at the same time and here we need to specify the remote root directory so we want this to be home Jenkins we'll assign this a label of Jenkins agent and again keep in mind what you use here because we'll use this in our pipeline uh upcoming here in just a little bit we're going to use this node as much as possible and we're going to actually connect via SSH so here we need to specify the agent's IP address so again pick the IP address that matches what you have set up in our case it is running on Dot 11. and we don't have any credentials so here's where we need to add that private key so let's click on ADD Jenkins and then we're going to choose SSH username with private key and then we're going to call this our Jenkins agent key we'll give it the same description the username is Jenkins and then we're going to enter that private key directly so what we need to do is go back over and we need to copy this entire block that we had catted earlier so let's copy this and let's paste that in here there's no passphrase so we can add now it's important to make sure you go back and actually select the key here and everything else we can leave as default so let's go ahead and hit save and now this agent should be connected so let's go into the agent and look at the log and it looks like no known host was found okay so this is what what we were describing earlier so we can see here that on our Jenkins UI there's no known host file so if we copy this and this is good this is why I wanted to do it live so we can troubleshoot some of these issues so let's copy that let's go into our Jenkins UI and change into that directory so we can see here if we look at the logs it's looking for a DOT SSH known hosts and it doesn't exist and the easiest way to create this is just to Simply SSH from here as the user Jenkins into our agent node so let's do that and this will create the the known host file for us so now we can exit back out to our UI and if we take it out take a look at the listings we now see a DOT SSH and we can see this known hosts has been created so if we go back into our UI what we can do is we can just hit configure and then hit save and relaunch agent so this should now allow us to connect okay let's see it looks like there's another failure so the SSH key does not match the known host the SSH key exchange was not finished now let's troubleshoot why this might be happening so just give me a moment here while I think why this might be happening so let's go back over here again let's make sure that that known host is in here let's go into our agent machine and actually before we do that let's just make sure we didn't miss a step in our documentation so uh so what we did we forgot this step this is a very important step so we need to make sure that we add the public SSH key to the list of authorized hosts on that agent machine so what we need to do is we need to cat the the public key and add it to our authorized hosts so let's copy this and let's go back into our agent host which is here on the right hand side then we can head into our SSH directory and then here we need to run this command so now we can go back take a look at our nodes let's go into our agent and launch that agent again okay so now we can see that we've successfully connected and and come online again these are the types of things that will come up uh nothing will be perfect as you go through your devops journey so it's good to be able to do some troubleshooting so we've already identified one area where you could potentially run into issues which is um making sure that the agent and the user interface for Jenkins can communicate okay so now that we've got that out of the way and we know that we can um connect our agent let's do a very quick test let's just make sure that we can execute a job so let's just call this test we'll create a pipeline job and what we'll do here is we'll just use a a hello world that's built into Jenkins here and since we've disabled any executors on the UI and we've got this set to agent any this should now run on our agent node so let's just go ahead and Save build now and we can see that it has successfully built perfect so we now know that we can run our jobs on our agent node so we can go ahead and clean this now up so let's delete this okay so the next thing that we need to do is we need to make sure that we configure our Pipeline with a complete devops pipeline so let's go ahead and do that so what I've done is I've created an application that you can use I'll make the the repos public after this demonstration but just to uh to show you how to use the the credentials against private repos it's private for right now and the application is just a simple Java application that uh launches a a very basic web page so if we go back into our terminal we can clean up these windows here and then let's just go back to our our local machine and I've got a project created here and you can clone this so it's called complete end-to-end pipeline production this is on uh my git repo here and I'll make these links all available so I've already gone ahead and cloned this uh there's no Jenkins uh or or pipelines that have been configured but just to show you what this does let's very quickly just run a a build of this job so if we do a a clean install and then we'll run the application from our local machine which of course is not our ultimate goal our ultimate goal is to create this um into a Docker container and publish it into kubernetes but let's take a look at what it is we're doing here so we've built it and then by default uh spring boot will launch its application on localhost port 8080 so let's go back and take a look at what the application is so here we can see this is just a an each other page that talks about the end-to-end devops pipeline perfect so we know the application itself works so now let's head back into our local machine and here uh what we can do is we can start building out that Jenkins file so let's go ahead and create a Jenkins file I'm going to be using visual studio code uh it's my editor of choice again you can use anything that you're comfortable with that could be Sublime maybe you're used to just using VI in my case I'm used to using visual studio so let's open up this project from its current working directory and then what we'll do is we'll create a new file at the root and we'll call it Jenkins file so let's create a new file we'll call it Jenkins file and then here what we need to do is we need to first set up our initial pipeline so if we start with typing pipeline I have a plugin installed that goes ahead and just creates a skeleton but we'll walk through each one of these so we'll remove some of these sections because we'll build them out as we go through the project so we're going to eliminate this this post section to start with and then we'll remove some of the post sections that exist in the first stage here so let's go ahead and remove that and then what we need to now do is first set our agent so if you remember I told you to remember the label that was assigned to that agent in our case we had called it Jenkins agent this needs to match the name of the label that you had created when we had set up Jenkins earlier so the first thing that we need to do is we actually need to install a few tools since our application is using um is rather based on Java and we're going to build it using Maven we need to install the Jabba and maven tools for it to to be able to build so let's save this file for a second and head back over into our Jenkins UI so from here what we need to do is we need to install a couple of plugins so let's head into manage Jenkins under manage plugins we want to search for maven and then here you can see that we're provided with a list of options the two that we're going to use are miven integration as well as the pipeline Maven integration and then the next uh plugin that we need to search for is Java so if we search for Java we can see might be a better one to search for actually instead of searching for Java let's search for the and then here we can see the eclipse Tamarin installer so let's go ahead and download and install after restart so again this may take a few moments we can see that it is installing and then what we'll do is we will restart when Jenkins is complete so while that installs and restarts we can head back into our our Pipeline and start configuring it here so we'll we'll talk about what these uh these labels are in a second when we when we set it up but in preparation let's go ahead and add the tools that we're going to use so to do that we need to create a a tools section and what we'll do here is we're going to create um two different tools one for Java and one from even so the first is the jdk that we're going to use and we're going to use Java 17. now this Java 17 is a label that will specify uh within our UI here in a second the next thing is the miven version so let's do Maven and we're going to call that one maven 3. so let's save that and head back into our UI so see if Jenkins it's still still pending so let's continue with our pipeline build so the first thing that we need to do is we need to understand how the pipeline works the pipeline is broken down into stages and each set of stages has a stage within it we're just going to use a single uh stage at this point and we're going to use multiple stages within a single set of stages so it's not a multi-stage pipeline I know that's a little bit confusing but uh in some Advanced videos we'll get into multi-stage pipelines as well so the first thing that we need to do is we should always start with a clean workspace so let's go ahead and clean up our workspace so we'll call this cleanup workspace and then each stage contains a step and then within a step is what it is that you want to do so here what we're going to do is we're going to actually use a built-in function that comes with Jenkins called clean WS and clean WS will clean our our workspace and once we've done that uh the next thing that we would want to do is we would want to check out our code from our source source code repository in order to do that we'll need some credentials so since our repository is private we'll need to create a uh a username and password now remember you can't use your password directly you need to go ahead and create a personal access token so that's outside of the scope of this but it's quite straightforward if you do a quick search for Google GitHub personal access token it'll show you how to very quickly create one of those so what we'll do is we'll build the stage out first and then we'll go back and create that token so once we've cleaned up let's take a copy of the stage and then we'll add another stage directly below it but this time what we're going to do is we're going to check out our code so let's check out from SCM and for our steps what we're going to do here is we're going to use the main branch so to do this we need to use get branch and then we need to select the branch that we're going to use which is mean and then we need to use a credential helper so to do that you do credential ID and then what we need to do is specify the ID that we're going to create so in our case it's going to be uh let's call it GitHub just so we can remember what that is and then we'll need to specify the URL of the project so it's probably easier if we just go back in here and copy that and then that's it so what this does is it tells us that we're going to check out the main branch using a credential that we'll create here in a cycle in a second called GitHub and we're gonna check out this repository that we've specified here so let's save that and let's see if our Jenkins is back [Music] and let's give this a refresh and it looks like Jenkins is back so let's sign back in again now the first thing that we need to do is we need to configure the plugins that we just installed so let's go back into manage Jenkins we'll head into Global configuration tools and then let's search for maven so we actually want the the second one here and we need to add maven we'll give it a name so this is the name that needs to match what we created in our pipeline here so Maven three and then what we're going to do is we're going to install this from Apache and you can select a specific version we'll just go with the latest version of 3.9.1 and then we'll hit apply and let's save that now if we head back again into Global configuration tools we'll need to also configure Java so let's go here and search for jdk and under jdk we want to do the same thing so let's click on ADD jdk and again we want to make sure that we use the same name so let's copy what we'd set up in our pipeline in our case Java 17 and then we can leave the Java home blank and what we want to do is we want to install automatically and we're going to use the adoptium.net and we're going to select version 17. and then we can hit apply and Save so now we've configured the tools that we're going to use now there are other workarounds Some people prefer to install Java and Maven directly onto their agents this is just another option I prefer to use this approach but there is a lot of flexibility with uh with how you can do this so before we run this pipeline what we need to do is we need to make sure that uh we set up this GitHub uh credential so let's go back into Jenkins let's go into manage credentials and then we're going to add a new credential and this will be a SSH with username and password but remember here you cannot just use your password you have to use the personal access token that was created in in GitHub I've already gone ahead and done that this is just a temporary token I'll delete this after this video is complete so I can share that on the screen and then we're going to call this GitHub which is what we referenced in our in our pipeline so let's make sure we get the spelling correct here and we'll match that description so again very quickly make sure that you use a personal access token here and not the password otherwise it will not work so let's go ahead and create that so we can see that that has now been created so the next thing that we need to do is we need to set up an actual pipeline so let's create a new Pipeline and what I like to do is I like to just name the pipeline the same as the uh the application again this can be anything that you would like it to be but in our case I'm going to just use the application name or the GitHub repo in this case and it's going to be a pipeline we hit OK there are a couple of things that we want to set up here just to make sure that we clean up after ourselves so let's discard our old builds and we only want to keep two of them again in production you may have some requirements that say that you need to keep a certain amount of builds but since this is a tutorial we're just going to create uh or rather keep two days worth of builds and then what we're going to do here is for the pipeline we're actually gonna use our pipeline from our Repository and then here we can select git and then now we now need to specify our repo URL or repository URL so let's copy that [Music] paste that in here and you can see that it's going to uh complain that it's not authenticated it doesn't support uh you know password as the authentication but we've already created uh credentials using a personal access token so let's go in here and select GitHub and once we do that we can see that everything is okay also make sure that you match the correct branch in our case it is being pushed into the main branch and then that's all you need to specify we're going to call our file Jenkins files we created in our repository so let's save and apply that now we need to commit that into our repository so let's head back in let's add and then what we need to do is commit so we added our Jenkins file again best practices to make sure that you use messages that you can refer to in the future it may make sense today but if you come back and look at this code or you need to revert something in the future it's always helpful for you to create messages that that make sense so now that we've done that let's go ahead and push this up and that should just take a moment and once it's complete we should be able to execute that pipeline so let's head back into our Jenkins and let's build now we can see that the bill just started and let's look at the console output so we can follow along okay so it looks like we've got some typos in here so this is great so Let's uh let's troubleshoot what's actually happening here so it looks like we've got multiple instances of stages okay so it looks like we've got some some error in our pipeline let's go back and take a look and indeed we can see what's happened here right so what we've done is when I copied uh the previous um stage for the the checkout of the sem I copied the the full block so what we want to actually do is we need to Nest our stages inside of stage we need to Nest our stage inside of stages so in order to fix that let's go ahead and let's remove this and then we're going to remove this guy as well so now we can see that this stage completes here and this one matches correctly and then this one matches so we should now be good so the first thing we'll do is we'll clean up our workspace and then our second stage will be to check out our Branch so let's make sure that this works let's save let's add commit again helpful message let's push okay so let's see what happens this time so let's build it now so let's take a look at our console output this time and the first time it may take just an extra minute or two because it is going to go ahead and fetch the version of jdk and Maven that we'd specified and you can see that that's already happened we're now doing a checkout and this should finish in a second again I'm leaving this unedited I'd like to you know go through all the troubleshooting steps nothing is ever Flawless so there are a lot of YouTube videos out there where everything just seems to work uh but we know in reality that's not the case so it's good to understand how how to troubleshoot some of these issues so let's see here um we're almost done okay so it's cloning the Repository and then we've got a finished success so if we go back and look here we can now see that our stages executed as expected great so now if we go back and take a look at our pipeline that we're building what we've done so far is we've pushed our code to the application repository we've started building our uh Jenkins user interface and the agent so the next thing that we need to do is we now need to build the application itself and execute some tests so to do that we're going to need to create a uh two more stages a stage for build and a stage four test so let's go back into our editor and this time let's make sure that we copy the right stage and what we're going to do is we're going to add two more stages so here what we're going to do is we're going to build application and we're going to test application now since this is a Java application the first thing that we'll need to do to build the application is we'll need to run a shell script that is going to do an mvn clean package and then to test what we need to do is we need to run a shell script and here we will do nvn test so as you can see we've just copied the previous stage twice we're gonna now clean the package build it and then we're going to run a test so let's save let's commit that code first of all we need to add the changes so we added our build and test stage foreign and then let's push that back up again okay so now we should be able to run this pipeline Again by hitting build now we can see that the job has started let's follow along inside of the console again it's completely normal for there to be problems uh arise as you're going through building out your your pipeline the important thing is to learn from your troubleshooting steps so the next time that you do it you have a sense of of what you did wrong the first time and you'll probably make the same mistake over and over again but it'll cut down on the amount of time that it takes for you to uh to troubleshoot so here you can see that it is now running its tests and it has finished so if we go back and take a look we can see the two new stages that were introduced so we built the application and we've tested the application great so now we're making some progress and what is the next thing that we need to do so the next thing that we need to do in our pipeline is we've now successfully built and tested we want to do some static code analysis and we're going to use a tool called sonar Cube to do this so let's go ahead and set up sonar Cube I actually have a tutorial here that walks us through how to set up sonar Cube [Music] okay so in the tutorial we can see here there's an installer Cube I'm going to be installing this on its own virtual machine uh you may have already done this so feel free to go ahead and Skip these steps but I'm going to keep mentioning I want to go through a complete Hands-On tutorial so we'll do this from scratch so in my case I have a third virtual machine that I have created so let's sign into that um that virtual machine and it is running on ip12. and then we can very quickly just copy and paste these commands so let's update our virtual machine and we're going to run sonar key with postgres there's a couple of different options I've chosen to use postgres this is not a tutorial specifically about sonar Cube I do have some separate videos if you're interested in learning more about it so let's go ahead and add in the postgres Repository and just like before we do not have wget installed so let's go ahead and install that first okay so now that that's installed we should be able to run through these commands so we're going to go ahead and update and install postgres and we'll enable it okay so while that's installing let's look at the next step so the next thing that we need to do is we need to set up a database for sonar Cube to talk to so to do that we're going to need to set the password for the postgres user which we just created in our prior step so let's go ahead and set a password remember this password because you will need it in the future the next thing we need to do is we need to become the postgres user and then now we're going to set up the the actual database itself so first let's create the user and then we're going to sign into the database and then we're going to execute these commands so again all this is doing is it is granting privileges to that user sonar to this database sonar Cube once we've done that we can exit out of the database and then we can also exit out of the postgres user so now we need to install Java again so we're already root so let's go ahead and just like we did before we'll use the Tamarind version of Java and we're going to use jdk17 sonar cube is particular so please uh pay attention to the version of java that you're installing here so let's go ahead and install this version and while that installs let's go ahead and take a look at some additional configuration that needs to happen so there is some kernel tuning that needs to happen first sonar Cube to perform uh optimally and I've gone ahead and just uh created uh some documentation here that talks about the limits um that we need to change for sonar Cube to run so we'll just go ahead and execute these commands we're not going to dive into each one of these specifics um just know that you do need to do this in order for sonar Cube to run so the first thing that we're going to need to do is we're going to need to make some limits changes so let's edit the file and we can just go all the way to the bottom and then we'll paste in these values the next thing that we need to do is we need to increase our mapped memory region so we can do that by editing this particular file and again just go all the way to the bottom and then we need to paste in this value and then we can save that and then let's go ahead and reboot our virtual machine just so that those take effect permanently so that'll just take a moment to reboot and then let's take a look at what the next steps are going to be so what we'll do is we'll download and extract uh sonar Cube I've already got the latest version here so at the time of this recording the latest version is 9.9 uh we'll create a user we'll set some permissions and then we'll update the sonar Cube properties to make sure that it can talk to the database okay so let's go back our virtual machine should have rebooted by now which it has let's go ahead and become root and then here what we need to do is just um copy these commands so this is going to download the version of sonar Cube extract it it'll move it into the opt directory structure and then it'll set the um we'll then need to set the appropriate permissions after we create a a user for sonar so let's go ahead and copy this command so now if we go into opt and we take a look we can see that the sonar Cube application has been extracted and the permissions have been set correctly so we now need to make sure that sonar Cube uses the appropriate database that we'd set up so let's go ahead and edit the properties file so again I'm moving through this a little bit quickly because I do have dedicated videos on this but I did want to include it in this video because it is an important part of the process so here we're going to copy the the settings if you recall earlier we just created a user called sonar and we set the password to be sonar we're using postgres which is also running on this virtual machine and by default postgres runs on Port 5432 and the database that we're using is sonar Cube so let's copy that we want to find the section here where it says database and then we can either uncomment and fill in the username and password or you can just copy and paste the um the configuration from the documentation so let's do that the next thing that we need to do is we need to create a service so that sonar Cube starts and stops with our virtual machine so let's create a service called sonar and again I've already gone ahead and created the uh the values that you need to use here so let's copy this let's paste those values and then we just need to Simply start sonar we'll enable it and then we'll check on the status so let's go ahead and do that so you can see that it is started we've enabled it and we can see that it's on our Cube service is now actively running now just because it's actively running doesn't mean that it's fully online yet we can take a look at that by watching the log files so the log files are written into the opt sonar Cube logs directory so let's take a look at the the logs here and we can tail along and we can see that it looks like it is still coming up so sonar Cube by default runs on Port 9000 so what we can do is we can now go over to our browser and go to the IP address of the virtual machine and access port 9000 and you can see that sonar cube is up in all systems are operational so let's go ahead and click on let's change that and that should go back in again this time it'll prompt us for our username and password so by default the username and password is admin and password uh sorry admin admin and then it'll ask us to change the password so the old password is admin let's set a secure password so again we are going to expose this to the internet in my case most organizations may not choose to do so um you may still want to configure a domain name with an SSL certificate but then have some firewall rules that only allow you to access it internally but that's outside of the scope of this so in my case it'll just be directly exposed and I'm gonna tear this down as soon as this tutorial is complete anyway so let's go ahead and uh complete our password and we can see that we now have sonar Cube up and running so great so now that we have sonar Cube um there are a couple of things that we need to set up for Jenkins to be able to send the applications code to sonar Cube for analysis so the first thing that we need to do is we need to set up an API key so to do that we go into the top right corner under administrator we can go into my account we go into security and then generate a token so create a token that's um that's friendly here uh but before we do that let's go ahead and actually set up a reverse proxy so that we can access this um through a domain name uh through a secure URL and if we head back into the documentation uh you can see I've got that laid out here it's an optional step um but you will want to do this especially if you want to use a web hook in the future where it doesn't really like you to use the same machine for the web hook again that's outside of the scope of this but let's just go ahead and very quickly run through this it will just take a few seconds so we're going to install nginx we're going to create a config file and I've already gone ahead and created the config file that's necessary so this is really just proxying all requests that come in to localhost Port 9000 and it's going to listen on this domain so please make sure you update the domain name whoops let's write and save that we want to then create a symbolic link to make sure that it's enabled we'll test the configuration and if all looks good we can restart nginx so it looks like everything is good let's restart nginx and now we should be able to hit our sonar Cube insulation on http without needing to go to a port and we can see that we need to do that but it's not secure so let's go ahead and quickly install a certificate to do that we need to install certbot again so we can copy the command and then we just need to create our certificate again for the domain make sure that you choose a domain that matches one that you have configured within your environment so let's do that we will need to enter our email address again accept the services we can say no to sending our data and then in just a moment you will see that it will return the certificate install it into nginx and then we should have a secure access into our sonar Cube setup so we can see that it's cleaning up the challenges deployed the certificate and everything now looks good so if we head back and then this time what we'll do is let's close this tab and let's go to https and we can see that sonar cube is now running on a secure Port so let's sign in with our new password that we had set and then let's get back to creating that API key so as I was explaining earlier to do that we go into the administrators account so in the top right my account security generate a token give it a friendly name so that you can recognize it uh we'll call this the name of our Jenkins server so uh you can select the token type in our case we'll get we'll go ahead and just use a global analysis token and we'll set this to no expiration and generate that token now it's important this token is only shown once so if you leave this page this token will disappear so let's go ahead and copy that token we'll store that token in a notepad for now and then we'll create a a secret so let's head back into Jenkins let's go into dashboard manage Jenkins we want to manage credentials and then let's add a credential and this time we're going to create a secret text so here we need to paste the contents of our token so let's copy that and paste that here and then here you need to create a an ID that you will reference in your pipeline so let's go ahead and just call this Jenkins sonar Cube token and then we'll use the same for the description as well great so now we have the credential uh set up we've got sonar Cube and the there is a couple of plugins that we need to install so that the application can send its uh its code over for analysis so if we head back into our Jenkins we want to go under manage Jenkins we want to go back to manage plugins under available plugins let's search for sonar Cube and what we're going to install is we're going to install the sonar Cube scanner um and then if you actually search for quality Gates we want to install the sonar quality Gates will install quality Gates and what this will allow us to do is it'll allow us to pause or block the build if it doesn't pass its analysis so let's go ahead and install after restart so this should just take another few moments so while that's happening let's uh go ahead and start setting up our pipeline so we can see that everything is downloaded let's restart Jenkins and then go back into our Pipeline and add an additional stage for the sonar Cube analysis so once we've tested the application the next thing that we need to do is create a new stage so we can just simply copy this stage block let's paste that here and then we can call this something that's recognizable so in our case we'll just say sonar Cube analysis and then in order to do this we need to make sure that we use a sonar cube with its credentials and then there's a maven command that allows us to run the tests so the command to run is very simple it's just nvn sonar colon sonar but right now it's not passing any credentials so this will fail so what we need to do is we need to make sure that we do this with credentials so let's go ahead and create a another line here that reads with sonar Cube EnV and this reminds me we do need to set up sonar Cube so we'll do that here in a second and then we need to do credentials ID and then we need to specify the token which we called Jenkins sonar Cube token okay so that looks good and then we do need to add an additional curly bracket here which we need to close here so let's indent this to get the the spacing correct so and I always like to make sure that we've got the the appropriate amount of open and close brackets so if we if we click here in Visual Studio code you can see that this closes the entire stage this wraps around the steps and this wraps around the environment variable so let's take another quick look here just to make sure I don't have any uh spelling mistakes so it's with sonar Cube EnV credentials ID and that is the correct token that we are using okay perfect so let's save that and Jenkins has restarted so let's sign back in so we installed the plugin but we didn't configure it so let's go back into manage Jenkins let's go into configure system and I always forget it may be under Global configuration tool but let's take a look let's search for oh it is under configure system so that's great okay so under sonar Cube servers it's very important that we check environment variables and then we need to add a scanner so let's go in here and give this a name so let's call this sonar Cube scanner so this will download the actual scanner binary and then we need to specify the URL for our sonar Cube system so in our case since we use the reverse proxy we can copy this URL now in your case you may have chosen not to do that so in that case make sure that you specify either the IP address or localhost followed by the port now we had already set up this authentication token so if we select the drop down we can use the authentication token and we can hit apply okay now there is one more so that I I misspoke that was setting up the actual scanner so if we now go back into manage Jenkins and we go into Global configuration tool let's also search for sonar here so here we do need to install the scanner so before we were just setting up the the location of sonar so here what we need to do is we need to add sonar Cube we can call this so in our Cube scanner and we'll just call it latest we're going to install it from Maven since we're going to use the um the MPN command to execute this and then we can apply save now these are just warnings um there's nothing that we can do about them it does tell us that some credentials may be transmitted in plain text if you don't like seeing these we can remove them since this is a complete devops pipeline I may as well go ahead and show you how to do that so here you can go ahead and just click on configure which warnings are shown it'll redirect you to this page and then if we scroll down just a little bit here um it'll tell It'll ask us under hidden security warnings which warnings we want to be notified about so I'm okay to accept these again this is personal preference I just don't like to see the big red on the screen uh but certainly before you do that I would encourage you to read the security advisories to understand what it is that's happening here but let's save and apply and that should take care of the the warning messages for us so if we take a look at that pipeline one more time we're now going to tell it to [Music] um do the analysis we've got the right credentials hopefully so let's go ahead and Commit This and give it a test so what we need to do is we need to go back to our machine where our source code is which is here so let's add let's commit and then we can push this and if all is well we should be able to now run this pipeline and monitor the console output and it looks like we made a typo here so credentials ID so I think that is a a typo so let's go back and take a look here what did we do here so if we 940 okay so let's take a look with sonar q b and V credential [Music] okay let's make sure we type this correctly correct whoops Shoals capital i d so that looks correct we can save that and there's one other way we can quickly validate the syntax so let me actually show you that so if we go back into art Jenkins and we go into um the main pipeline we can click on this pipeline syntax so let's open it in a new tab and then let's search for sonar cube in here so with sonar Cube and prepare environments so we're going to use this token let's generate that pipeline I could not find any tokens could not find any credentials with ID Jenkins okay so let's see what is going on here let's go back into our dashboard manage Jenkins let's take a look at our credentials so we do have our token here which we've got as a secret text and it is saying that it cannot find it okay so let's let's troubleshoot What's Happening Here before we do that let's actually just copy the the generated pipeline make sure that we didn't make a spelling mistake so let's go into Visual Studio code here okay so it looks like that is correct I have a suspicion that we need to change the type of [Music] um token or key that we created so let's go back in here again and it says that it cannot find any credentials so if we now go back let's delete let's delete this token or this credential rather and then what we can do is let's go back under credentials let's add a new credential and let's use a secret text which I believe is the right one but let's go in back over here and just copy and recreate it so let's Jenkins token and then we still have this here luckily and we also did copy and paste it so we we did have that saved but let's go ahead and recreate that secret and see what's going on here so if we go back into our pipeline token it might just remove that for a second let's now select okay so I'm not sure why it's giving us that warning but let's go ahead and try to run our our pipeline again so let's go back into manage Jenkins sorry dashboard I believe let's build it now and these are some of the the troubleshooting steps that you'll end up having to go through and we can see here that it's already failed again so let's take a look a little bit more closely at that message so and we did create that key let's make sure that we configured everything correctly so let's go back into our configure system let's search for sonar Cube so we can see that the let's uncheck okay so we know that that is correct we have this checked you don't need to specify any additional arguments so let's actually change this to server for a second this should not make a difference but we're just um troubleshooting what's happening here so let's go back into global tool configuration let's look for sonar Cube and we do see that that is correct changes name to scanner okay and I do want to just take a look at the pipeline again so let's just make sure we're doing this correctly so with sonar Cube okay so we copied that we know that that is correct let's make sure there was no difference there okay so let's let's add let's commit and push then let's go and rebuild that there could have been a typo that I just did not see when I copy and paste it so let's go ahead and build it again okay so it is still failing let's see multiple compilation errors hmm I don't think this makes a difference but let's try adding this in a script tag now let's make sure we close these correctly we've got that one that's correct and we'll tap these out so that can go there now I don't think this will make a difference but let's try okay I think that was it so I think since we're adding multiple lines inside of our Jenkins file we need to wrap it inside of a script so that's a an important lesson so again you know troubleshooting as we do in this live um we we found out that we need to wrap this inside of a script so let's see if that actually now completes two succession so it looks like it is testing the application we're running through the sonar Cube analysis now and then hopefully this will finish in just a few few moments and since we know the token is correct let's go back to our main sonarq page and once this finishes it should now post the results into our Sonic key which it has done and we can see here that that has passed okay all right so again like I said I'm doing this live no editing so I'll keep this all in the video so you can hopefully learn from some of the mistakes or some of the oversights that that I've been making here okay great so let's uh let's go back and continue so if we go back and look we've we've done our static code analysis um but we haven't introduced a quality gate so let's go ahead and introduce a quality eight uh what that means is if the project doesn't pass we can block the um the deployment in order to do that you do need to have a web hook that can respond back to Jenkins to do that we need to set up our web hook so let's go into Administration let's go into um actually it's right under configuration web hooks there's no web hook defined so let's create one we'll give it a name and then the URL that we need to to pass back so here it's important so in our case this is where our Jenkins and Jenkins is installed and then you need to add a sonar Cube Dash web hook and it's very important that you include the trailing slash we're not going to use any secrets so let's go ahead and create that web hook now we can go back into our Jenkins file and let's add an additional stage so let's make sure we copy the previous stage and then here what we can do is we can add a quality gate and then in order to do this we'll leave our our script in um and by the way just as a side note there's no harm in wrapping everything in a script even if you're just executing a single shell command so we can leave that in here and what we're going to do here is we want to wait for Quality gate so let's change this and we don't need that additional bracket so but here what we're going to do is we're going to wait for quality gate and then we want to abort the pipeline is false so we'll continue it'll just report back in our case we're going to use the credentials that we had specified earlier so let's go ahead and just copy the the same credentials now there are some additional options in here so again we're not going to abort the the pipeline we're just going to report back the status so let's just very quickly take a look at this uh so we're going to wait for Quality gate we're going to not abort the pipeline our credential is as specified so this time let's save and see if we got it right the first time so let's add and then let's Commit This so we'll add the quality eight check let's push that code and then let's build this again so hopefully you guys are are learning um or finding this useful uh you know we are building a complete pipeline so apologies for this being a long video but you certainly can pause this and come back to it at a later point if uh if it's getting a little bit long but I do want to make sure that I keep all the um the troubleshooting as part of this video because I think a lot of people will find this useful all right so let's watch our pipeline so it's going through its sonar Cube analysis and we can see it's now doing a quality gate and as you can see it has passed so now we can see that the quality gets faster we refresh this page we can actually now see that it's reported directly under this pipeline so we get the results of our quality gate now if we didn't create that web hook what would have happened is it would have just hung or stayed on that stage because it would have been waiting for a response to come back from sonar Cube so it's very important to make sure that that webhook is configured if you do find that your pipeline is hanging or appears to be stuck that could be one of the reasons why great so now let's go back and take a look at our pipeline here so we've already gone through we've uh built and test we've done our sonar Cube we've introduced our pipeline the next thing that we want to do is we want to now build and push our Docker container so in order to do that let's go ahead and install some plugins okay so let's head back into Jenkins and under manage Jenkins we want to install some plugins that'll help us build and push our Docker container so under available plugins we want to search for docker and here we're going to select a few plugins so we're going to select Docker Docker Commons Docker pipeline Docker API Docker build step and then finally the docker build and push so let's download and install after restart so we can go ahead and check this and while the plugins installed let's go ahead and start preparing our pipeline so if we head back into our Visual Studio code there are a few environment variables that we're going to use to help us build the container and these are again you know some best practices uh organizations may vary to vary but I like to set this up so that we can have a uh an appropriate way to version and tag our applications so the first thing that we'll do is let's go ahead and introduce some environment variables so under the tools section what we want to do is let's go ahead and add so environment and what we're going to do is the review that we're going to add so the first is we'll um we'll create a variable called app name and we're going to set the app name equal to the application name that we're building so in our case I'm going to just use this here which is the name of the git repository or the application itself so we'll call our application complete production end-to-end pipeline will also create a release and we'll use this in the the docker build stage here as you'll see in a second so let's start this with version 1.0.0 and then we'll append the tags to this release version we can then set our docker user so again make sure that you change this to match your your Docker Hub username so in my case I am going to use eye doctor username and then we can set a Docker password I will just call it Docker pass to make it a little bit shorter in this case what we'll do is we're going to actually create this as a secret because we don't want to leave this in our public repo so here what I'm going to do is I'm going to say that the docker pass environment variable is equal to a secret that is called Docker Hub and we'll set that up here in a second next what we'll do is we'll create a format for our image name so let's say our image name is going to follow a specific pattern so what we'll do here is we'll say that the image name is going to be r Docker username so in this in this case it'll be docker user so to do that what we need to do is we need to specify it inside of our these curly parentheses so Docker user and then it'll also be the docker uses so in this case that'll be dman Cloud as you can see here and then we're going to add a slash which is the the format that's required by Docker Hub hub and then we're going to add the app name so to do that we do just like we did before so we'll do dollar and then inside of the curly parentheses it's the app underscore name so our image name should be in this case Docker user which is dman cloud slash and then the app name which will be this so again hopefully this makes sense this is really just you know building out some environment variables so that we can use them later on um but it's following the the docker Hub that's uh you know standard uh for naming and then the next thing that we need to do is we need to create an image tag so this image tag will will be changing later on um once we update it inside of our second pipeline but for now what we're going to do is we're going to say our image tag is equal to and then we're going to say it's the release so and let's just copy and paste this because there's definitely room for error here so it's going to be the release followed by the build number now the build number is something that is a Jenkins environment variable so for example build one two three four five and you can see that here if we just very quickly jump in and show you what the build number is again I just want to be very thorough so you you have a full understanding of what is happening so once we go into the pipeline here our build number are these build numbers um so in our case when we create our image tag it's going to be our release which will be 1.0.0 and then Dash and then the build number and then we want to make sure that we put that inside of quotes so let me just double check that syntax to make sure that there's no mistakes and I'm sure that they Pro there probably will be um so let's see release okay so that does look good let's make sure that we've got our parentheses close correctly here so it looks like there's a mismatch so let's let's figure out what's happening here so we know that that one's correct that's correct that looks good and we do not need that double quote there so now we can see that everything lines up as expected and I wouldn't be surprised if there's some some problems here but let's uh let's go through it and we'll troubleshoot if anything um comes up okay so now that we've defined some of our environment variables let's go ahead and create our uh Docker image and then push it up into uh GitHub it's uh sorry into Docker Hub itself uh but before we do that let's save this and we do need to create a token before we uh before we forget so let's go ahead and go into manage Jenkins manage credentials and then this time we'll add a new credential username and password and then this is your Docker Hub sign in so again just like with uh GitHub we want to make sure that we create a uh an API token that that can be used as a password uh I've already gone ahead and done that um it's very straightforward you can just search on Google how to create an API token outside of the scope of this video but you've probably done that before so since I've already gone ahead and done that let me just copy what I have again I'm going to delete these tokens as soon as this tutorial is complete so it's okay if you see them and then let's match the the ID so if we go back and we take a look we said that we were going to call it Docker Hub so let's go back and create that new credential okay great so now that we've done that we can go back and actually start working on our Docker build stage so let's copy this and then let's call this stage our build and push Docker image okay so since we installed those plugins as uh a couple of uh shortcuts again if you look at a different YouTube videos you may see some different ways of this being done these are just the steps that I prefer to use when building a and pushing a Docker container I just find it to be very uh straightforward so in our case what we need to do is we need to tell it that we're going to use Docker with registry and since we're using just regular Docker hub and then we can pass it our docker pass which is our Docker password that we defined earlier and then here what we need to now do is we need to say our docker image is equal to am I going to do a docker build and then this is where we can start using those environment variables so before we had created this called image name and then we finish that stage there so once that's finished the next thing that we need to do is to push it so let's go ahead and create a another Docker with foreign the spelling correct here then what we need to do is again use our docker pass which is an environment variable that we had defined earlier and then this time what we're going to do is we're going to do a docker image push because this time we want to push it and we want to push it with the tag that we had generated so let's go the image underscore tab whoops tag and we need I think we're missing some quotes here so this needs to be wrapped inside of double quotes okay so let me just take a quick look at that again so we are going to push it and the image tag so I think I've got my quotes in the right place so this will push it with the um the image tag that we're generating based on what we use in our environment variables before but you may also want to just re-tag the latest build as uh the latest tag so you can do that by simply doing a Docker image push and then you can just tag this with latest as well now again you probably won't deploy with latest in in production use cases but this way you know that the latest build has will always have the latest tag as well so give that one quick look through everything seems to be okay but if not we will troubleshoot so let's save that let's go ahead and commit let's commit our changes so add it Docker build and push and then let's trigger that pipeline again so let's go back in here go build and then while that's building let's go into Docker Hub and you'll see that that repository does not actually exist so if I sign in you can see I've got a a complete devops pipeline that I'd push to two days ago but we don't see our complete end-to-end pipeline so if all is successful it should create a new repository with the tag um and push it up here so let's see what happens let's go back we can see that it's going through our testing of the application it's doing ours on our Cube analysis and we can actually follow along inside of the console just in case there's any errors that pop up and and we do have a error message okay so let's see so it is saying that it has failed to ah it would be helpful if we had created a Docker file so in this case uh what we forgot to do is we actually forgot to build a uh or rather create a Docker file so there is no Docker file for it to build which makes sense all right so let's go back into our Visual Studio code and then we need to create a Docker file so these are the things that you will probably run across as your building your your pipelines there's just a lot to to remember so okay so this is a Java application so what we'll do is we'll use a multi-stage build process and we'll build it first and then we'll package it up so let's see so what we're going to do is we'll use the uh the maven image and we'll use version 3.9.0 again you probably want to check with your developers to see if there's any specific versions that they want to use um since this is just a quick uh a quick example we'll just use the the 3.9.0 and what we'll do is we'll just go ahead and use the uh the tamarin 17. and we'll do this uh since we're going to do multi-stage as build again I'm not getting into the specifics of the actual Docker file here if that is something that you're interested in learning more about in detail um I'm more than happy to create some additional content so please just leave a comment below and I'll uh I'll certainly help you with that so here we're going to set our work directory to app and then we're going to copy everything into that working directory and then we're just going to do a simple nvn and then this time we'll do a clean install so that'll build our package and then the second stage what we want to do is let's again use our eclipse we'll use um the camera and I'm hoping I could just order complete here but it's not Rin and then we'll use version 17.0 6 and these are just versions that I've identified on Docker Hub so if you're wondering what where I'm coming up with these versions I just happen to know that these uh these are the versions that I want to use uh okay and then once we've done that we want to set our work directory to app which is where everything is and let's make sure we get that correct and then what we would like to do is we need to copy from the build and it's under at Target and the jar file that it builds is the demo app.jar now again I'm not getting into the specifics of the the Java side of this but this will be something that your your developers can help you with and we have a spelling mistake here because this should be from build and we're going to copy it from the applications Target the demo app dot jar into app and then we're going to expose the port Again by default the port is 80 80. and then we need to set a entry point so what we'll do is we'll run Java and then we'll do Dash jar and then we need to specify the jar file itself okay so let's take a look at that to see if it looks good so we're going to execute the build okay that looks good but again if there are any issues we'll troubleshoot it live here so let's go ahead and save that so okay so let's see what happens if we run that build now okay so let's follow along in the console and if there are other areas that you would like me to create any tutorials on please feel free to leave some comments below I'm always looking for Content ideas um I did have a lot of requests for this particular tutorial to go through an end-to-end uh devops pipeline so uh more than happy to create any additional content that you that you uh may feel is valuable all right so it looks like we're going through our Docker build stage here so hopefully we got our syntax correct in the Jenkins in the Jenkins file the build is done and then we should be on our way to pushing that container up into Docker Hub so let's see what happens hopefully you guys are enjoying what you're seeing so far you're finding it useful something that you can use either in your day-to-day jobs or potentially as you're interviewing uh and looking at uh maybe starting a career in the devops world okay so it looks like this has finished and is successful so let's go ahead and look you can see that everything passed if we go into our Docker Hub and give it a refresh we can see that it did indeed create the new repository it was pushed just a few seconds ago and we should see two tags so here we can see exactly what we expected to see which is the release which we defined as version 1.0.0 the build number which is 10 and then we also tagged it with the latest tag so this is great so everything so far is working as we expected it to so let's move on and see what the next stage is so we have a a trivi artifact scan that we can run um trivia against what I might do this video is getting a little bit long if this is something that's important to you uh just let me know and I'll update um yeah this video with uh with this build step for now but what we're going to do is let's just go ahead and move on to deploying this into our uh kubernetes cluster and in order to do that we're going to need to set up two more virtual machines one virtual machine that will run Argo CD another virtual machine that'll run kubernetes and um let's go ahead and do that now okay so just like before I've got this documented we'll quickly go through and set up our two additional servers we'll go through this a little bit quickly just to move this video along but I've got um two tutorials here that explain how to get Argo CD installed and then we'll just create a blank cluster for our kubernetes cluster so let's open up a another shell prompt so what we'll do is I'll open up a new terminal window and we'll we'll do two at the same time here so we can help move things along a little bit quickly so let's sign into our Argo CD machine and then on the right hand side we're going to build another cluster at the same time as well so if we go back into our documentation we'll just do a copy and paste and I'll move through this pretty quickly so let's update both virtual machines and then we want to create a kubernetes cluster so here I'm just going to use k3s which is a lightweight kubernetes cluster it'll install we'll copy the config file and we'll move that into our home directory so we can access the cluster itself so let's just go ahead and very quickly copy this command here and this will be the same for both our Argo CD setup as well as our cluster where we'll run the application so let's go ahead and install that here and I don't think the copy and paste worked correctly so let's just run that again let's become root on this side and we're disabling traffic traffic is a an Ingress controller and we're going to actually just use nginx um for our setup so you can see here that it's already installed and the next thing that we need to do is we need to then create a a folder for where our Cube config will uh will reside so let's go ahead and do this so let's exit out of [Music] both of these and then what we can do is just copy this block of code and actually I think there's a missing step here so let's create the cube directory on both folders and then we need to actually change into that directory and I'll be sure to update the documentation uh By the time this video is released so once we've changed into the directory we can and just go ahead and copy these commands and then now just to test we can do a Cube CTL get nodes and we can see that this Argo CD cluster is up and running and let's do the same on this machine and over here it's called app cluster so the the name of the cluster takes on the the host name of the machine and as you can see I've created the hostname of Argo CD cluster for this virtual machine and app cluster for this machine okay so now that we've done that let's go ahead and quickly get Argo CD installed uh we're just going to deploy it using um the Manifest files so let's copy and deploy Argo CD and then what we're going to do is we're going to change it from cluster IP to node port so once it's installed we can use this patch command and we'll edit service to to now be running as a node port and then we can fetch the password so this is the password that's going to be the default password that gets created for Argo CD so let's copy that and save that over here and then what we now want to do is let's go ahead and get everything set up for enabling with uh with TLS and exposing it via an Ingress controller so in order to do that we need to run or rather install cert manager so I will install cert manager and again I'm only doing this on the Argo CD node for now so once the cert manager is installed we'll need to create a cluster-wide issuer using let's encrypt again this is outside of the scope of this video but if you're interested let me know in the comments below and I can dive into some more detailed videos on on how this all works so now that cert manager is installed we're going to go ahead and create the issuer if you're following along and copy and pasting just make sure that you change your your email address here so let's save that and then let's apply that so this will go ahead and create a cluster-wide issuer and we can check that that's working by taking a look at the cluster issuer so let's do a cluster ensure and we can see that it's created and enabled great so the next thing that we need to do is we need to install the nginx controller I'm just using the nginx controller from kubernetes and not doing anything special here so let's just go ahead and apply that once that gets installed we should now get a an IP address that's mapped to the Ingress so we can check by doing a cube CTL get Ingress and we can see that there's no resources that are found yet oh rather sorry let's we didn't create an Ingress we need to check the services so let's check the services and we can see that the load balancer IP address is still pending but that will just take a moment and there we go we can see that the the load balancer has been created and the next thing we need to do is we need to actually create an Ingress to expose this so I just took this directly from the Argo CD documentation but I've included it in this guide just to make things easier again if you copy and paste just make sure that you update the references to the hosts that are defined here so let's create a file called Ingress and then we can copy this whoops let's make sure that we [Music] do this correctly and then we can apply that Ingress and then this time if we do I get Ingress we'll see that it is um trying to get a certificate from the Acme servers using cert manager and this usually just takes about anywhere from 30 to 60 seconds but if we just give this a few seconds in fact we can just put a a watch in front of this and we should see that this comes back and there we go so now we should be able to access our Argo CD using our hostname over a secure https connection so let's go back into our browser and then let's go and try and access our Argo CD and there we go we can see that Argo CD is up and running it it's your connection so here we can sign in using that password that we had collected earlier so I'm just going to go ahead and copy this and I'm going to sign in perfect so now we have a blank Argo CD there are no applications no git repos nothing has been set up yet so there's a couple things that we need to do so the first thing that we need to do is let's go ahead and add that second cluster that we'd created so in order to do that we need to copy over our cubeconfig file and use that context to register with Argo CD so again just bear with me as I go through this it may appear a little bit complicated but hopefully it will make sense so on this machine on the right hand side which is our application cluster which is what we want to register with Argo CD we need to take a look at the config file so let's take the contents of this cubeconfig file and we're going to go to our host where Argo CD cluster is running and we're going to create a new config here we're going to call this Argo CD sorry app cluster and then we'll paste the contents of that configuration in here now we need to change the IP address this is the IP address that Cube CTL will talk to so since we copied it from the local host of the APT cluster it won't work if we try to talk to it from the Argo CD cluster so let's go ahead and change oops let's change the IP address and in my case it is running on 192.168.1.20 and then we can save this now what we want to do is we want to export that Cube config so that we can talk to the Argo CD cluster so we can do that by doing export Cube config dot Cube and then we can specify the app cluster and then let's make sure that that indeed worked as expected so if we do a cube CTL get nodes we can see right now that it is using the Argo CD cluster so that is incorrect so what did we do wrong here did we made a typo in the cube con thick okay so it's always good to check so I'm glad that I'm glad that we did that so now that we exported the right config file we can see that we're we're now talking to the app cluster which is where our application will be deployed so let's go back and take a look at our installation notes Here so we can install the Argo CD command line utility which will help us do a couple of things we can change the password if we choose to do so but it'll also help us register that cluster so let's copy this and then let's install the Argo CD command line utility so this will fetch it from Argos GitHub and then it'll move it into the user local bin directory so once we've done that if you want to you can go ahead and change the password um in my case I'm just going to skip over that for saving some time and then what we'll do is we'll actually just go ahead and register the uh the cluster so in order to do that let's just do a quick search so if we say Argo CD add cluster and the top link here that comes up shows us the com the command that we need to enter so basically we can just copy this and there's some additional arguments that you can pass like the name if you want to override the cluster name which which we will use so let's copy this and then what we want to do is we want to add it's Now the default context and then we're going to call it we're going to use the dash dash mean and then we're going to call this the app cluster and then it's going to ask us if we want to give it the privilege to to do this we'll say yes it's going to create some cluster roles a service accounts and cluster role bindings and let's see why did that fail let's just make sure that that did not indeed create let's go into settings and take a look at clusters okay so it didn't create it let's take a look we must have made a typo here uh let's take a look at the documentation very quickly what did we do here let's take a look at this one so I think I'm missing okay so what we need to do let's actually take this because I think I may have typo let's see so the name is boot app cluster okay so app cluster and we need to make sure we specify the default context which is what it is set as so let's see why is it telling me service address foreign let's just make sure I've got the right context exposed here so let me do okay so let's see what is going on let's take a look at that config make sure we've got the right context okay so it is default and when we try to do a oh I know exactly what's going on again this is what happens when you do a live tutorial so what we didn't do is we did not sign in to Argo CD so what we need to do is we need to do an Argo CD login and then we need to log into our cluster so before we do that let's fetch that password again um and before we can do that we need to switch our contacts back so let's change our context back to the default fetch the password and then change our contacts back okay so now what we need to do is we need to log in to argocd.dab.dman Dot cloud okay so now that we've actually connected our CLI to the Argo CD instance we should be able to now register and there we go it was added so again sometimes these small things will will catch you off guard um but at least now we know that we we're in business okay so we can see that the app cluster has now been added with the correct IP address so what we now need to do is we need to set up a secondary repository where we will create a deployment and a service manifest and I've already gone ahead and done that so if you take a look at my git repo I've created a repository that is called git Ops complete end-to-end production and there are two manifest file rather a deployment and a service file that are listed in here so if we take a look at the the deployment you can see it's a very basic kubernetes deployment file again this is uh outside of the scope of this uh tutorial but I I would assume that you you have a good idea of how to create these manifest files and then here what I've done is I've created um a placeholder that basically tells us the image but then they change me and this is what will get changed with all the tags so what we need to do is uh clone this repository and then let's just first make sure that the application deploys as we expect it to so if I go back over here and I we no longer need to be over here but we'll we'll leave that window open so let's go back to our host machine and go into the projects folder and I've already cloned that Repository so let's edit the deployment file and then let's just very quickly update this tag so we we've already pushed a version of a an image that we built already so let's go back into Docker hub and then let's take a look at what it was right so this is our image name and then the tag so let's just make sure that we have set this name up correctly in our or deployment here and then we'll use this tag okay and then bear with me because you'll see in a second in our Jenkins pipeline we'll actually update this automatically through our build process but for now what I'm going to do is I'm going to add this and I'm going to commit it and I've updated the image tag so if we go back and take a look at our repository and we'll refresh this page you'll see that it's now got the right tag in here so let's go ahead and create a application so before we can do that we need to make sure that we specify the credentials for our GitHub repository inside of Argo CD so let's go under settings and then we'll go to repositories we want to connect a repository we're going to connect via https I'm going to deploy it to the the default namespace and then we can take this URL directly from here this is where our manifest files reside remember this is a separate repository from our application so let's go in here and create that repository for the username we want to use our username and then just like we had to in Jenkins we want to make sure that we use our personal access token to connect so let's specify that and connect and we can see that that's successfully connected so now we can create an application so let's create an application we'll give it a name so again let's give it the the same name as our application so if we get that deployment file we'll call our application simply this and again you could use whatever you like here and then we're going to deploy it into the default project we're going to automatically sync this we'll prune and self-heal and we're going to get it from the repository that we just configured we'll use the head and then our files are all located in the root directory so we can just do a DOT slash and then here we want to make sure we select the application cluster so let's select the application cluster we'll deploy it into the default namespace and then we can just go ahead and create that so once we do that we can see that it's already starting to sink if we click in we'll see that it has started fetching our image and is deploying it and while that's going through we can actually take a look on the terminal and take a look at what's happening so if we do a cube CTL get pods we can see that it is indeed pulling our container and it's entering into a crash loop back off but that could just be let's see what's actually happening here take a look at the events quick look at this so if we describe the pod let's see why it is back off restarting and it could just be because we don't have enough resources or allocated but let's take a quick look okay so we'll do some more troubleshooting and let's take a look at what's happening here let's take a look at the logs oh it looks like when we built our container that we had a problem it's unable to find the demo app.jar so let's see what happened here so if we go back to our project so we expect it to create this demo app.jar so let's take a look at where that actually is so if we go back to our application let's take a look at the complete project let's go to source Target so app Target demo app.jar so it is that is where it gets created but when we built this for some reason it did not work so let's let's take a look at the Stocker file one more time to make sure that we're building it correctly and I already see the problem here so the problem is is its app so again I'm glad that we we took a look at that so let's go back here and we can push that so if we do a get status we'll see that we modify the docker file so let's add that let's commit fixed typo and Docker file and then let's push and then we'll have to rebuild that again so this is the beauty of live demos what we're going through everything from the beginning and we can uncover any problems but again this is stuff that um you'll run into as a devop uh devop engineer so it's good that you see the troubleshooting steps and possible areas that that you could run into any issues so we know that this is now going to build we can see that it's running build 11. so when this finishes we can predict that our new container is going to have a tag of dot 11 as it gets pushed into Docker Hub so let's go back and get that ready so let's just go into here since we're doing this manually let's go back into our get Ops repository let's edit that deployment and we'll change our image name to 11. and then let's get this staged so we updated the image tag and then before we push it let's just make sure that everything gets pushed up as expected and this should just take a few more seconds okay so it's almost done and if we go into our Docker hub we'll see that the dot 11 tag has been pushed so now let's commit that let's go back into Argo CD and let's do a sync so this time hopefully we should be able to have a fully deployed application and it looks like it is indeed so if we go back and we can check our pods and we can do we can see that that is running and then these will fail off as um as the other application comes online so perfect so we can give that a second and everything will will sync itself back up again here and if we want to force it let's just go ahead and do that and force this application to sync so that is what the the problem was I'm glad that we were able to sort that out let's just go back and take a look here and make sure that that one container stays running and it does and we'll see that this one will um we'll drop off here in a second and we can actually probably just delete that and just go ahead and delete the pod [Music] okay so it's not behaving so let's do this the easy way let's just actually go ahead and recreate this application so let's delete the application from here we'll let this delete we'll make sure that everything is cleaned up on the cluster which it is so let's just go ahead and quickly create that application again deployed into default automatic prune for path is dot slash as we described earlier okay perfect now we are healthy and synced all right so now that that is running and we have our deployment and service manifests that have been configured what we now need to do is we need to make sure that every time a build occurs that it Updates this tag so in order to do that let's jump back into our Jenkins file and we're going to need to add another stage so what we'll need to do here is we'll need to use some shell script and we'll use set to replace the um the tag inside of our container so in order to do that let's go ahead and create a a new stage so let's copy the stage and we can actually clean this up so there's quite a few things inside of the script tag so let's clean that up and now we're going to call this Mom trigger continuous delivery pipeline okay so I'll go through this as slowly as possible again I do apologize I know this video is quite long but I want to make sure that this is a an important step to make sure that we can automatically update the manifests so here um Jenkins has the ability to trigger another pipeline by passing it parameters and there's a couple of things that you need to set up for this to work so we do need to create an API token for the Jenkins user and then we'll match it with a token in the build step for the second pipeline but first let's just go ahead and create this um uh the actual script that does that so it's a shell script and what we're going to do is we're going to actually curl over to our Jenkins machine to trigger it so I'm just going to go ahead and type this out so we're going to use the user admin and then what we'll do is we'll create a an API token so let's just put a placeholder in here for now we'll call it Jenkins Epi token and then we're gonna post and then we need to specify some headers here so again you can look up these headers um I mean it's not not too important uh just know that you need these headers to be passed [Music] so we need to pass uh cache control and we need to say no cash uh another header that we need to pass is the content type and what we're going to be setting is application x dot uww and it's a form and it's a URL and who did encoded let's just make sure we got the spelling correct here and then data that we're gonna pass along so now this is the actual image tag variable that we're going to pass so we're going to say image underscore tag and that's equal to what we had defined earlier which is image tag and then we need to specify our Jenkins server which in our case is running at Jenkins dot whoops jenkins.dev Dot email.cloud okay so here we need to specify which job that we want to trigger so what we're going to do is let's go ahead and go into Jenkins and then let's create a new item we'll call this get Ops complete pipeline so let's just make sure we we save that name we'll come back and adjust this but for right now it's let's just create that so we have it uh waiting for us let's go back in here so the job name so this is the second pipeline the pipeline that's going to do our updating of the Manifest um for us so let's pass the um the the pipeline name and then what we also need to do is we need to pass some additional parameters so here we need to say build with parameter let's make sure we spell that correctly power and enter and then a token so this is a token that will set up on our on our pipeline which I'll show you here in a second and then for the token we can just call it anything we want so we can just say for example get Ops token and we need to close that and then end it with a single quote perfect so that should be all we need here and if we've got all the okay so we're missing a a curly bracket somewhere so let me just make sure Jenkins we've got that I do not need that there I'm just going to go through and make sure I don't have any spelling mistakes in here and I'm sure I will which we'll find when we when we run the pipeline but so there's a cache control with no cash the header type is URL encoded we're gonna pass it the image tag and then that should be good okay so let's save that and then what we're going to do is let's go into our Jenkins and configure this pipeline so we'll do the same thing here so this remember this is the git Ops pipeline now so we'll keep a maximum of two builds this build is parameterized so the parameter that we used was image tag so this is what will be passed in and again this image tag will be passed in based on what we're building above so let's add a string parameter of image tag we don't need to specify any default values and then what we do need to do is specify a token so where just make sure that we have that correct and then we need to trigger remotely because we're gonna this is our first pipeline is going to call this Pipeline and then here is where we need to specify the token that we'd made up so in our case we just called it get Ops token and then here we're going to again use the SCM we're going to use get this time we want to make sure that we used our GitHub sorry our git Ops repo so let's specify that one and we don't have a Jenkins file here yet but we'll we'll create one we can use the same GitHub credentials that we'd set up earlier we want to make sure that we're using the main branch we'll call it Jenkins file so let's apply and Save okay so now we have this pipeline built what we need to do is we now need to create the Jenkins file that will replace the tags undeploy the file and push it to GitHub so let's go back into our let's go back into our command prompt and what we want to do here is let's take a look here for one second so what we need to do is we need to create a Jenkins file so let's see okay sorry for that delay what I was actually doing is from another machine I was copying over a Jenkins file uh just to save us some time instead of going through and building it so let me uh let me show you that Jenkins file so what we did is I copied over this file and again I'll walk you through it uh it'll just save us some time uh from building it out and it's quite straightforward so let's go ahead and enable word wrap so just like we did before we need to specify the the Jenkins agent in this case uh the label that we specified matches what we've created in our Jenkins UI we've created an App name so this is just the application name which will use a little later on again we'll clean up our workspace it's always a good practice to do that we're checking out the get up uh sorry the GitHub repository so we want to make sure we're checking out the actual repository for uh the Manifest files and then this is the the trick here the trick is really just a very basic shell script so if you want to run multiple commands on different lines you can just use three quotes so we're just going to cut the deployment file to see what it looks like and then we're going to use said to basically just search and replace uh the image tag with an image tag that we're passing in as the build argument and then we're going to modify the deployment.yaml and then the next stage will be just for us to push that deployment file to get this is very straightforward again we're just setting our Global username email address we're going to add that deployment yaml we're going to commit it with a message and then we're going to use our credentials so again this Jenkins file is not anything complicated so I just wanted to save some time and copy that over for us so there there's one more modification that we'll we'll have to make in our other Jenkins file so here you can see that we've got this app name that's been specified so we'll need to also do this in our Jenkins file for the application repository so let's go into that one which we can find over here so we'll just need to go ahead and add that in here and then we do need to create the application token so we didn't create the the Jenkins token before so that's the other thing that we need to do so let's go back into our Jenkins let's click on our username click on configure and then we want to create an API token so we can give it a token name of Jenkins API token so this is the token that we're going to use to allow the second pipeline to be triggered from so let's copy this to make sure that we have that saved and then we need to go into the dashboard manage Jenkins system is running just a little bit slow here and then for manage Jenkins we want to manage credentials we want to create a new credential we're gonna create a secret text we need to paste them that secret text and then here we can call this Jenkins VPI token let's give it the same name and then we can create that so now that we've created that we need to then go back into our Jenkins file and add an additional variable so we need to say that the Jenkins EPA token so this is what we were following it down here so if we go look we see with Jenkins API token but since we're calling it as a variable we'll need to Define that as well so let's also call that and we'll Define that as and we called it the same name so we can just do that simply great so now if everything works correctly we should be able to run our pipeline the last stage of this pipeline will make a call using the admin user and the Jenkins token that we just created it'll call that new job with the parameter and in this case it's the image tag which will be the new updated image Tag and Call that second Pipeline and that second pipeline should then trigger so if we go back and take a look at the second pipeline it'll clean up its workspace check out the repository where the deployment and service yamls are it'll search and replace the image tag with the build parameter tag that we just pass in commit that and push it to our git repository so let's go ahead and give this a run and see what happens so more than likely it'll fail the first time because nothing ever works the first time but it should give us some good troubleshooting steps so let's go ahead and let this let this run through so we're building the application okay so it's going to go through our sonar Cube analysis it's on to the build stage so now this should create a new Docker image it'll tag it with 1.0.0.12 if you remember that's what we had set up for our image tag to be it'll also tag it with latest and if all goes well it should then call our second pipeline job which will update the tag and then commit that to our git Ops Repository which will then be picked up by Argo CD and and and be continuously deployed so let's see we should be almost done so it looks like the build finished so let's go back to our job let's look at the get UPS Pipeline and see if this one actually triggered so it doesn't look like it has triggered so let's see why that did not happen maybe we didn't specify the right name so let's go back and take a look oh I know why because we did not save and commit those files so we need to commit everything before we change it so if we go back here so we're in the get Ops pipeline so this one we need to make sure that we commit our Jenkins file so let's commit so this is the Jenkins file and then we need to push that but then in our application we also made changes to our Jenkins file so if we take a look just to show you what happened here if we do a git diff on that Jenkins file we added that whole stage but we never actually committed it so that's why it didn't trigger the second job so let's add this as well and then added CD stage and then let's push that up so now we should have everything committed and if we go back into our pipeline let's go ahead and rebuild this again so this time it's going to run another job in this case it's going to be bill 13. I guess we'll see if uh the number 13 is going to be lucky for us um so let's see what's happening it looks like there's a failure so not so lucky let's see what happened here let's look at the console output uh app name equals to oh I do not think that we actually need that in here so let me fix that if we go back into our Visual Studio code and it is not this is one it's this one that it's complaining about so complete and let's just look at that error message again yep so we'd already specified this or is it duplicate so it's been a long video so you can imagine um so let's hit that again so it's a duplicate entry that we'd made which is why that had actually happened so let's remove foreign duplicate EMV okay so now let's go back in and build this job okay so this time it is running indeed so as you can see there are lots of moving Parts when you're building a complete devops pipeline so it is important that you're careful you use git as we've been doing so in case you need to go back and revert any changes but hopefully um you are learning um something by watching this tutorial if you do find it useful again please do comment if you found this too long um I'm happy to take negative feedback as well so any comment is uh is a good comment so please just uh let me know what you think if this is useful these types of videos are useful it certainly does take a a lot of time for me to put these together so if you're finding them useful I'm more than happy to create uh more content like this so let's see here we should be almost done with our pipeline a few more seconds and if we are lucky this time it will kick off that second job unless of course we've made any more silly little mistakes that we may need to fix okay so that has now finished let's go back to our dashboard let's take a look at this job and we can see that it has not triggered so let's go back and see why did that not trigger so let's let's look at the output of the console here and then we'll want to go all the way to the bottom so we can see that it did the docker login here so that's okay so it looks like it did try to connect and we're getting some specific warnings which is fine and it looks like it does not like our token so let's see what did we do wrong with that token let's go back and have a quick look so we created a Jenkins API token which is stored here we scroll down and take a look or signing in with the token that was created would be the problem here let's take a quick look okay so I see what the problem is the problem is that we didn't specify the environment variable correctly so this is actually incorrect the way that we need to specify this is we need to say credentials and then we need to specify the credential this way so let's make sure that we've got that spelled correctly let's save that and then let's take a look at our status here so let's go let's go get commit fixed typo but this time it was in trying to get this file okay so fingers crossed let's see if this is the last run that we need to do so let's go ahead and build this appreciate you guys hanging in there with me again this is a long video uh hopefully you did load up on some coffee or some Red Bull um I will be leaving um some chapters in the bottom of the video description so you'll be able to fast forward and look at certain parts of this but again I'm not going to edit any of this because I think it is important um I keep reiterating that but it is important to to see the troubleshooting steps that need to take place as you're building these pipelines because they can get very complex and there are a lot of moving parts and things that you need to remember okay so let's see what happens this time let's actually go ahead and watch the console while this is all this is running so let's look at the console output we should be nearing the end of our docker build here and then once it pushes it we're gonna hope that this time it actually authenticates to our second Pipeline and triggers the job so let's see okay so it's pushed successful okay so we didn't get any error messages this time now let's go back to our dashboard it looks like this pipeline has indeed triggered but we do have some errors Okay so we've made progress let's let's troubleshoot this one now let's take a look at what's happening here hopefully this is something straightforward so let's take a look and it is authenticating I want to make sure that we have the right repository that we're checking out here maybe that is the let's just make sure that we have the right repository so if we refresh this page we want to make sure that we are indeed using this Repository in our Jenkins file so let's quickly make sure that that we are okay so everything looks good here oh and very simple we have the wrong credentials so this is GitHub so we need to make sure we set the correct uh credentials okay so I am pretty confident that this will be the final run so let's change that in our git Ops so let's add and then let's commit fixed GitHub oh it's okay let's push this okay and this should be our final run let's go back in here build now let's give this just a see it takes about a a minute for each one of these to to finish so let's uh let's go through this again if you've stuck with me this far I appreciate it thank you so much if this has been valuable consider subscribing to the channel giving it a thumbs up sharing this video um I'm new to to YouTube so I'm sure there's certain things that I could be doing better so please leave comments below let me know what I can do to provide more value if there's anything that I can change if these are too long I'm happy to try and break these up into smaller videos as well so any feedback would be uh greatly appreciated so let's give this a few more seconds uh based on our prior runs it looks like it takes about 40 to 45 seconds for for Docker to uh to finish building and pushing and the good news is our second pipeline should only take a few seconds because all we're doing there is we're checking out the code and we're doing a search and replace of the image tag and then just pushing that back out again so that should only take uh less than 20 to 30 seconds so it looks like this is nearing its completion here so that has finished let's go back and take a look at our second job and this should trigger um which it did and it looks like it completed successfully so let's take a look at the console output and make sure that it did update our tags as we expected it to so if we scroll down so the initial the first time we we looked at the cadded rather the deployment file was tag 11 we did a search and replace it updated it to tag 16. and if all is well our Argo CD should now be able to pick up that latest change and run version 16 so let's see if we look at app details we can see it is indeed running version 16 okay so thank you all for bearing with me I know this was a very long video but hopefully you found this helpful um again appreciate your time appreciate you sticking through to the end if there's anything else that I could help you with leave a comment below with that thank you so much

Channel: Dinesh Mistry

Views: 104,929

Keywords: Jenkins, DevOps, CI/CD, Docker, Containerization, Sonarqube, ArgoCD, GitOps, Jenkinsfile, kubernetes, k8s

Id: q4g7KJdFSn0

Length: 169min 54sec (10194 seconds)

Published: Sat Apr 15 2023