Learn Jenkins! Complete Jenkins Course - Zero to Hero

welcome to my video on everything you need to know to get started with jenkins this is going to be one of the most comprehensive guides for jenkins on youtube it contains both theory and practical labs that you can follow along with jenkins is a hugely sought over skill to have for devops with engineers earning anywhere between 150 000 to 250 000 dollars per year hopefully that's enough to motivate you to take the time to learn jenkins but if you need another reason it's actually a really cool skill to have as it can be used to automate anything in this video we're gonna go over what you need to know to get a jenkins server up and running set up and manage jenkins agents as well as create a declarative jenkins pipeline using the groovy language as always this video includes chapters and everything you see me do in this video is available on my github in the description below so make sure to make use of that as you follow along so when getting started with jenkins there's plenty of documentation on the jenkins website but who wants to spend their day reading documentation off of a vendor website that's why i'm going to give you the too long didn't read version jenkins is an automation platform that allows you to build test and deploy software using pipelines but it's not just limited to creating pipelines for code it can be used to automate any task i commonly use it to run my bash and python scripts as well as ansible playbooks jenkins provides a web gui where you can create jobs and customize all the functionality that you want such as source control management pre and post build actions as well as build triggers this allows you to run tasks on demand by clicking a button or have them triggered automatically via web hooks so now that you know what jenkins is let's get into jenkins infrastructure which is pretty simple first you have the master server which controls the pipelines and schedules builds to agents and then you have the agents which run the build in their workspace so an example workflow may look like this a developer commits some code to a get repository the jenkins master becomes aware of this commit and triggers the appropriate pipeline and distributes the build to one of the agents to run it selects the appropriate agent based on labels which is something that you can configure through the jenkins ui the agent then runs the build which is usually just a bunch of linux commands to build test and distribute your code now one thing you should know when getting into jenkins agents there are two main categories of jenkins agents the first is the permanent node agents you can think of these just as standalone linux or windows servers that are configured to run jenkins jobs and these are just basic everyday servers the only real setup of them is you need to have java installed and you need to make sure that ssh is set up as the master server usually makes connections over ssh you'll also want to make sure that you have any type of build tools that you want to use on these servers installed as the agents are going to be the workhorses that run your actual builds so those are permanent agents but there's another type of agent known as cloud agents which are a much more popular choice in real world environments some examples of cloud agents are docker kubernetes and aws fleet manager in these scenarios jenkins can dynamically spin up agents based on the agent templates you configure in the lab portion of this video i'm going to show you how you can set up docker to be a cloud agent so whenever jenkins jobs are run they are run in a dynamically provisioned docker container this is going to be very similar to how you would set up a kubernetes agent as well as ec2 fleet manager agent to dynamically spin up pods or dynamically spin up ec2 instances now we're almost ready to get started and set up our own jenkins infrastructure but let's first talk about the two main build jobs that you're going to run into with jenkins these are the freestyle build projects and pipelines first is the freestyle build projects which is the simplest way to get started with jenkins and the way that i suggest you start out to learn jenkins basically just think of them as shell scripts that will be run on a server that can be triggered by specific events like a developer making a commit to a github repository freestyle projects are easy because you can use the ui and plugins to manage just about anything in the build most freestyle projects will be set up to execute a shell script so as long as you are familiar with shell scripting or the linux command line you should be good the next style is pipelines pipelines use jenkins files written in the groovy syntax to specify what happens during the build pipelines are commonly broken into different stages as you can see this pipeline is broken into five different stages the clone build test package and deploy stages these stages can differ depending on the project but this is a common workflow that i usually see the clone stage is usually made up of pulling down the code from a git repository and setting up the local environment on the agent the build stage takes the code and builds it which usually means generating some type of local artifact on the code like a jar file executable or container image the test stage yes you guessed it runs tests against the newly built code and in the package stage it gets packaged up so it's ready for deployment in the deploy stage is when i usually send out artifacts to my registry so a good example would be sending out a newly built docker image to docker hub so that's what jenkins does it's basically just a way to automate the work that developers don't want to do so they can save their time and do more productive things so that's it for the theory portion of this video because like morpheus says no one can truly be told what jenkins is you have to see it for yourself so let's go ahead and have a look at jenkins alright so we are here on the jenkins website with the installation instructions for jenkins there's a couple different options that they give you here you can either install it directly onto the os or you can run jenkins in a container environment such as docker or kubernetes the suggested method is to use docker so that's what we'll be doing there's instructions on the jenkins website how to do that but i have a more summarized version of those instructions in my github in the description below so make sure to check that out if you want to follow along i'm going to pull that up right now and you can see that there's three major steps for the installation the first step is to build the docker image the next one is to create a network called jenkins and then to launch the container i have commands for mac os linux and windows so you can use either one so let's go ahead and do this the first command is going to build the docker container using the docker file that i have here and just to check it out you can see it's just like any other docker file uh this one's pulling from the jenkins repository and you can see it's also installing blue ocean blue ocean is a popular add-on for jenkins it makes your ci cd pipelines look a lot nicer and just a little bit easier to manage and troubleshoot we'll be having a look at blue ocean later on in this tutorial so let's go ahead and pull up a prompt here and i'm going to start off by cloning my repository here and once i've cloned it i'm going to hop into that directory so i can create the docker image and i'm going to do that with my docker build command and you can see that it's building the container and it should just take a few minutes for you depending on the speed of your internet connection it's just going out and grabbing the docker image of jenkins and then running apt update install and installing a couple plugins so once that's done i'm going to clear the screen to make this a little easier to see and the next thing i'm going to do is docker network create jenkins and you can see i'm getting an error that the network already exists i already created this network but you should be able to see it and the way to verify that is just through a docker network ls command and you can see i have this jenkins network already created so i can move on to the last step which is actually running the container commands and again this is in my github below i'm going to copy that over for my other screen and paste it in here and this is just going to start the container and you can see exactly what it's actually specifying here you can see that specifying the network of jenkins there's a couple environment variables here specifying the location of the certificates the tls version and a couple mapped volumes and which ports should be published so let's go ahead and do a docker ps and you can see that the docker container is now running and it's listening on port 8080 so let's go ahead pull up a browser and see if we can get to this website all right so i have it pulled up here and you can see that it's on this unlock jenkins command this should be the first thing that you see when starting a new jenkins instance and basically what it's telling you is you need to put in the administrator password and that password is going to be at this location when you first install jenkins now since we're running this on a docker container we're going to have to use the docker exact command to cat out this file so i have that command saved in my github so let's go ahead copy it paste it into the terminal here and it spits out my password so i'll grab that put it in and we're in now we're at a screen where we can install jenkins i'm going to install with the suggested plugins and this should go pretty quickly it's just grabbing the most popular plugins that jenkins uses such as ant and gradle a couple other things here let's just wait for this to finish and then we're going to set up our user account all right so we're at create your first administrator go ahead and put in your details i'm going to put in my details now and then hit save and continue all right for this instance configuration you can just leave at default and hit save and finish and start using jenkins alright so this is the jenkins ui in all its glory i know it's not a lot to look at but it does get the job done let me zoom in a little bit here to make things a little larger and easier to see and let's go over what these items do the first item here is your breadcrumbs so it'll make it easy to navigate through jenkins and go down levels something we're going to be using a lot when we're administrating our jobs the next thing is new item this is where you create your projects and pipelines we're going to be going into it shortly the next thing is people this is where you administrate user accounts build history gives you a history of all your jobs that have been run manage jenkins has all your management stuff this is where you set up agents this is where you install plugins we're going to go in here as well and i'll show you the most popular menus that you'd be using as a jenkins administrator the my views and new view is just a way to organize your jobs and have things displayed nicely blue ocean is a popular plugin for jenkins that enhances your ci cd pipelines once we have our pipeline created we'll go into blue ocean and check out how it looks in blue ocean versus the regular jenkins ui for now let's go into manage jenkins and you can see that this brings up quite a few different options basically these are all your options for managing jenkins so at the very top you can see there's some notifications here let's go over them the first one is just saying that building on the built-in node can be a security risk this is just letting us know that we don't have any agents set up so if we were to make a jenkins pipeline it would be running on the jenkins master which is not recommended we're going to be setting up some agents so this is something that we can ignore for now the next one here is it's just a warning about one of our plugins basically wants us to go in and update our plugin so we'll just ignore that one for now and when we get to the plugin section we'll go ahead and fix this let's go over the main items that you'd be using as a jenkins administrator so i'll scroll down here and get everything in the view alright so let's see if we can speed run through the most popular items here the first most popular one is configure system this is going to give you a lot of information about your jenkins server and let you set some global parameters in regards to your jenkins server so you'll probably spend a little bit of time in here the next popular one is manage plug-ins as a jenkins administrator you're going to be spending a lot of time in here because jenkins is a beast when it comes to plugins and over time companies install plugins developers install plugins and different plugins break each other when you upgrade jenkins plugins break so being a jenkins administrator you're going to be dealing with plugins a lot so we're going to be spending quite a bit of time in here in this lab and i'll show you how to administrate plugins the next one is manage nodes and clouds this is where you're going to go ahead and set up your agents as well as clouds like kubernetes docker aws anything to do with agents will be in here the next section is the security section a lot of these are pretty obvious you can configure global security settings you can manage users and then you have managed credentials the manage credentials is probably the least obvious one in this list this is where you would store ssh keys or api tokens now you don't have to use jenkins as your credential manager you could use something like aws secrets if you want but jenkins does have a built-in credential manager so go ahead and use that if you would like down below we have some other pretty obvious things you can get some system information about jenkins there's some logs some load statistics i don't spend a lot of time here at the bottom here you have a couple tools uh mostly for troubleshooting jenkins but one that you may run into that you need to use is prepare for shutdown so you would use this in a situation where you need to take the jenkins server offline to perform some sort of maintenance you wouldn't want to just go ahead and reboot the jenkins server or just shut it down because you'd be interrupting current jobs that are running so what you'd want to do is set it to prepare for shutdown and what that'll do is make it so jenkins doesn't start any new jobs it just finishes the jobs that are currently running all right so now that we know the main management features of jenkins we're ready to create our first project so let's go up to dashboard and then new item and in here you can see there's quite a few different options but the ones that you're going to be sticking with mostly is freestyle project and pipelines pipelines are the more advanced style where you use the groovy script we're going to be getting into that later on in the video but freestyle projects are also pretty popular and uh they're great way to start learning jenkins and if you're administrating jenkins servers on the job doing any type of consulting you're going to run into freestyle projects so it's going to be something you want to learn so let's go ahead and create one we'll have to give it a name now the first thing i can say here is do not create a name that has spaces in it and the reason for this is because in the back end jenkins is actually going to be creating a folder based on this job and it's going to be something that we have a look at so you understand the file structure of a jenkins server but for now uh just make sure not to use spaces and use something like underscores or dashes instead so i'm going to call it my first job make sure freestyle project is selected and hit ok all right so we are in the project settings of our first build that we're creating here let me take you through the category of settings that you're gonna run into here the first one is just a couple build options at the top you can ignore those for now it's going to be something that you get into as you get more advanced into jenkins the next one is source control management so you're almost always going to be using something like get and then you just put in your repository url jenkins is going to go ahead and pull that repository down into your workspace and use that code the next one is your build triggers so your build triggers is what is uh triggering your job so popular options here is to have github send a web hook to your jenkins server now this can be a little more difficult if your jenkins server is behind a firewall you'll need to either open ports or use some sort of proxy service to make sure that web hook gets to your jenkins server another way to sort of cheat the system is to use this polling sem and what this is going to do is it's going to have jenkins reach out to github and just check periodically so that's one way to get around if your jenkins is behind a firewall and cannot receive web hooks another option is to just build periodically and this is just the same thing as setting up a cron job you just set a schedule and your jenkins build will run based on that schedule for build environments a popular option is to delete the workspace before the build starts this just makes sure that you have a clean environment before your build runs and it deletes any artifacts that have been left over by jenkins that were created during the last run of the job and then scrolling down we have the build these are your build steps that your build is going to go through and then we have our post build actions and these are just things that are going to happen when your build is complete so some popular options here might be to send an email notification and if you had the slack plug-in you could send notifications to slack so let's go over to build and the most popular one that you're going to use is just to execute a shell so this is just like bash scripting so whatever we run here the jenkins server is going to run within its workspace so the simplest thing that we could do here is just echo hello world hit save and then build now and this is going to run our job and we got an error here so let's go check that out and it failed because i opened that git repository thing so let's go back to our project go to configure and i left this empty let's just go ahead and set it to none and that should fix the error that we have there so let's go ahead and save it hit build now and now we can see that the job has run successfully so we can click into it and there we go it echoes hello world so that's your simple hello world jenkins build let's go back into the build just go up to back to my project and then configure and we're going to do something a little more complicated than just echo hello world let's get into the available environment variables so if you click here just open it in a new tab you can see all the environment variables that are set for jenkins that you can use through your build and you're going to be using these variables quite a bit whenever you're creating jobs the most popular one that i like to use is a build underscore id and this gives you the current build id i always use this when i'm versioning my docker images that i spit out from my build i like to specify the build id and then that way i know which build actually built the image so that's a popular one that i like to use another popular one is uh is the build url so you can see it here and you can see it gives the url for the build so that's a good one so let's go ahead and put these into our job so to use the jenkins variable i usually just go echo and we can see the build id of this job is dollar sign and then i usually throw it into these two curly brackets and then i go build underscore id and then because we can we'll go the build url is and then we'll put the build url go ahead and save it and then build now and then we can go into our job and depending where you click the icon in jenkins it's either going to bring you to this screen or bring you right into the logs so let's go into console output and there you can see that it is saying hello world because we have that at the start of our job and then the build id is three and then the build url is here now one thing that you may notice is it's spitting out everything twice uh to explain that uh this is the actual command it's running with this plus sign and then the next line is just the standard out of the console so since we're echoing hello world uh that's sending standard out so that's the standard out so that's why you're seeing everything twice in this job looking at this build url so that basically just brings us here so this is a really good one if you're sending slack notifications you can send the build url to one of your developers letting them know that the build is done and then they can click it and see everything that happened so if you click it you see it brings us here gives us all the information about the build okay so let's go back to our project and go to configure so the next thing that i want to do here is just show off the workspace so if we go ls this is just going to send the ls command to linux i'm actually going to go ls ltr and this should show all the files within the current workspace now since we don't have any files this probably won't output anything so let's create a file and we'll go echo one two three four and then we will send the output to a file called test dot text and let's do an ls after that we'll hit save hit build now and job number four is done by the way you can see there's two different links here this one brings you to the main build page this one brings you directly to the console output so i always hit the check marks or the x's so let's hit this you can see that it ran the lsltr first there was zero files and then you can see that it echoes one two three four out to the file and now we have that test.text now if we go back to our project you can see uh we have this workspace option and if we go into workspace you can see that that test.txt file exists and there you go click it you can see one two three now just for science let's go ahead and run the build again and it's now done and if we go there this time you can see on the first ls ltr that the file already exists so basically what this means is we are not clearing out our workspace and the files from the previous build are already in our directory so if we go back to our project we can change that by going delete workspace before build starts and you can see here that you can add patterns to delete specific files i'm just going to delete everything so i'll just leave the defaults hit save hit build now the job is complete go in there now you can see the first ls shows that there's zero files then we create the new file perfect now another thing that i would like to show you before we get into more advanced builds is the actual file system of jenkins so to do that let's hop into the terminal and i'm going to run a command here to get into the actual docker container so it's just docker exec dash i t the docker container name and then bash this will give us a bash shell into the container and now we're on the actual jenkins master uh we're just in the root folder so we'll want to go into the jenkins folder and we actually specified it in our docker run command here so it's this volume that we mounted so var slash jenkins home so we'll go in here and do an ls ltra quite a bit of files and directories here the one that we're interested in is workspace so let me clear the screen and then go into workspace and now if i do an ls ltra you can see that we have a folder here called my underscore first underscore job so this is the job that we created so whenever you create a build it's going to have its own workspace under a folder of the job name that's why you don't want to create spaces and you can actually go in here and you will see all the files in here so we can see our dot text and cat it out and there you go one two three four so actually sshing into the jenkins machine or doing a docker exec into the container and having a look at the file system is something you're going to do quite a bit with jenkins especially when you're troubleshooting builds and jobs sometimes it's a lot easier to actually just go into the file system of the jenkins server and have a look there so a really great troubleshooting tool let's uh actually go back to the jenkins home folder so we can actually just go cd uh tilde sign and that should bring us here if you do pwd you can see we're in the jenkins home folder and uh if we look at the files again you can see that there's other folders in here so there's actually quite a bit of troubleshooting you could do within the file system you can see that this plugins folder uh this would be where you would have all your plugins uh the updates folder where all the updates go and then you can see there's a couple like xml files for the jenkins configuration as well as logs so this is a good place to go to troubleshoot your jenkins master if you need to now that we've seen the file system of jenkins let's hop back into the jenkins console and i'm going to go back to the dashboard and you can see that we have this job coming up here nicely and you can see there's a couple things here you can run the job it tells you when it last successfully ran when the last failure was and the duration that it took the job to run now if we were to create some new jobs they would show up here as well so let's go ahead and do that i'm going to go new item and what we're going to do for this one is we're going to clone down our repository and run a python script so we'll call this one my python job hit ok and i'm going to go to this source control management click get and put in my github repository so you should do the same if you're following along there's a python script in this repository that we're going to run and you can see there's credentials here now this is a public repository so we don't need any credentials for cloning it down but if this was a private repository you would need to select your credentials or go ahead and add them so if you're using a private repository go ahead and make sure you're using credentials for that but for us we're okay so let's head down to the bottom and we're going to execute a shell and what we want to do here is i just want to see which version of python is running and actually one quick troubleshooting step i want to run is let's just see if python is installed hit python on the jenkins server and you can see that command not found so let's try just python3 and we can see that python3 is installed so let's go ahead and go to our job and make sure we use python3 and then we want to run our script called hello world dot pi and i believe that's what it's called in our repository yes hello world dot pi so let's go ahead and run that one go ahead and save it and hit build and check out the job and there we go we can see everything that it did so you can see that it went out to my github repository it cloned everything down and then it ran python3 hello world and we got the output of our script so something as simple as this is actually quite powerful instead of us having to log on to a server to run a python script and you know make sure we have ssh to the server to make sure we can run the script we can just create a jenkins job hit a button and we'll know that it runs we can also set up triggers for this job to be triggered like web hooks or when a repository is updated as well as sort of like a cron schedule and we'll also always get a history of when that script was run as well as full log output so this is just something that you can tie your python scripts too and then just sort of keep track of whenever they're run i also love doing this with ansible and running ansible playbooks instead of logging onto like a server and having someone run the ansible playbook i just set up a jenkins job to run the playbook and then you have a full history of whenever that playbook was run and if it was successful or not all right so now that we have the basics of creating builds with jenkins let's get into more advanced topics like setting up agents and then building pipelines using the groovy syntax so i'm gonna go back to the dashboard and then manage jenkins and then manage nodes and clouds and then on the left hand side there's two main options there's the new node and configure clouds so if you go to new node this is how you configure a permanent agent basically this would just be any sort of like linux or windows server that you have that is sort of always available you set it up as a permanent agent and jenkins just connects to it via ssh and then distributes the jobs to it now in my opinion this is sort of the older more deprecated ways of creating agents what people are usually doing now is configuring clouds so if you go to configure clouds this is how you set up agents using cloud platforms such as docker kubernetes aws so now that we're in here we can see that there are no current cloud implementations available to us right now unless we go to the plugin manager and that just leads us to the plugin manager so we'll go there and you can see that it's filtering on cloud providers here and this is basically just a list of all the cloud providers that you can use for your agents let's go ahead and install docker but if you're interested in kubernetes or using aws you could use those so i'm just going to do docker and i'll say download now and install after restart and then it's going to go through everything these look like just the first plugins that i installed during the setup down at the bottom here you can see docker and it'll be activated during next boot so i'm going to go ahead and click this which will restart the jenkins server all right once the jenkins server has restarted you should be asked to sign in again so you can just sign in and then you'll want to go over to manage jenkins and we should be able to go to our plugin manager and see that docker is installed if we go over to the installed tab here we can see that everything to do with docker has been installed and if we go back to dashboard manage jenkins manage nodes and clouds then configure clouds we can see that we have adding docker as an option so i'll go ahead and add docker and i'm gonna hit docker cloud details here and there's a couple things that you need to do to set up docker the first thing is you need to set the uri of the host that is running docker so if you wanted to run docker within the jenkins you could just use like 127.0.0.1 and then the the port for docker but if you want to run it remotely then you'd put in the ip address of the remote server now i'm running docker desktop on my local pc here so i don't want the docker to run within the jenkins master i want it to run within my uh actual pc that's running docker desktop so there's a couple things that we need to do here and there's actually a really good stack overflow article that outlines how to do this i'll put it in the description below and i also have details in my github how to do it and uh you can just follow along here on how to do it so what i first want to do is launch a container running the alpine socad image so i'm gonna copy this command over from my github and put it in and this should start up a container that's gonna help proxy the connection from our jenkins master container over to our local host here so you can see that the container is up i'm going to grab this container name and i'm going to do a docker inspect on it and what i'm interested in here is the ip address of the socat container and it's right here the 172.19.0.3 so i want to forward my connections for docker to this ip address so let's go in here and we'll go tcp throw in that ip address and then it's going to be port 2 3 seven five and then i'm gonna hit enabled and test connection just to make sure that everything goes through okay and you can see that it has connected properly now if i put something incorrectly here like the wrong ip address you're probably going to get a connection error like the one that's popping up here and you'll just have to take whatever comes up here and go show details and troubleshoot it go to google check to see what people suggest but most likely it has something to do with routing now let's go ahead and change this back to three then just have it test and then go ahead and hit save and that'll bring us back here let's go back to manage nodes and clouds back to configure clouds and uh we now have our docker cloud details set up but now what we want to do is create a docker agent template so i'm going to hit this here and then hit add docker template and there's a few things that you need to set up here the first one is the label and what labels are used for is for the master to use to determine which agent should run the build so it should be something descriptive about the agent i'm just going to call it docker dash agent dash alpine and then i'm going to say that this should be enabled make sure that you do the same if you're following along and then i'm going to take this label and just give it the same name as the agent template and then this is the important part this is the actual docker image that you're going to be using i'm going to be pulling one from docker hub and it's going to be a jenkins agent image running alpine with jdk 11. now if you want to use your own image you can point to it on docker hub or you can set up a private repository you'll just need to make sure that you put in your credentials so the jenkins master can authenticate it to it and pull down the docker image now for some container settings we can leave that as default instance capacity this is basically how many agents it's going to actually spawn so you're going to want to set this to a number i'm going to keep it really low for now just because if you run into any issues you might be spawning lots of agents and you don't really want to overwhelm your system another really important field for us to fill here is the remote file system root this is basically where your workspace folder is going to get created this is your jenkins home folder so i'm going to call it slash home slash jenkins as this is the default that you should be using for all your jenkins agents now if we scroll down i believe that's everything that we need to set here let's go ahead and hit save and let's go back to our dashboard and let's uh go to my first job configure and what we'll want to do here is select this restrict where this project can be run and then we're going to say docker agent alpine we can just select this one i'm getting a notification that this doesn't exist this label doesn't exist but sometimes it's just jenkins uh causing trouble you can see that the autocomplete had a space in it so when i remove the space that seemed to fix it also sometimes if you go in and out of the job that will fix it as well so go ahead and hit save and now when we build it should run the build on our docker agent so i'll go build now and you can see that it's looking for an agent that has the label docker agent alpine looks like it's still looking for it so let's go ahead and troubleshoot this make sure that this is the right label i'm going to open up another tab here and just go to configure oh go to a dashboard actually and then manage jenkins nodes uh configure clouds take a look at our agent template we can see that's the correct name here and it looks like it's enabled so it should be good and if we go back back to here go to cloud details let's go ahead and test the connection the connections working good so everything here should be working fine let's go back into the job and it looks like i was just being a little bit impatient as i was recording here the agent picked it up and ran the job if we run in here we can see uh it was building on the docker remotely on this agent that it created and uh the job completed successfully and we can also go here built on docker and then it tells you everything in regards to docker now let's go ahead and have a look at our other job that we created and you can see it's this python job so what would happen if we went in here and restricted it to our alpine image let's go ahead and i think that'll fix itself if we just hit build now we'll give it a minute to build go back into configure and i guess i just didn't read it correctly it's saying that it does match it so let's go back and you can see that it did not run successfully so let's have a look at why and you can see that it checked out the code but it couldn't run the python command because python 3 isn't available so the reason for this is because that agent template that we created is just pulling down jenkins image from docker hub that doesn't have python installed so what you would probably want to do is create your own image and that's exactly what i did and i have one available on docker hub that you guys can use as well so let's go back to configure clouds under manage jenkins and we're going to set up a second agent and give it a new label and then this second agent is going to run all our python builds so let's go to docker agent templates we have our first template here i'm going to take the first part of this label and then scroll to the bottom and go add docker template and then i'm going to give it a similar label but instead of saying alpine i'm going to say python make sure it's enabled i'll give it the same name as my label and then for the docker image i'm going to use my docker image that i created from docker hub so it's just devops journey one slash my jenkins agents colon python1 and this is what the docker file looks like if you're interested now i'm just going to complete the other settings here so instance capacity of two and then home slash jenkins for the remote file system that should be enough to get us started so i'll go ahead and hit save and then let's go to our dashboard go into our python job configure it and instead of using just our alpine docker image we're going to use our the label of docker agent python which has our python image so we'll save that and we will run it and there you go if you're patient enough to watch you can see that uh it selected the agent let's just go ahead and see and there we go the python script ran successfully on our docker agent that had python installed on it and there's actually a bug in my script that i didn't notice but maybe some of you guys did when you run python hello world it's saying hellworld instead of hello world so let's go ahead and fix that and one thing that we can add into our build is to have them triggered automatically so i'm going to go here and i'm going to configure and in our build triggers i'm going to say pull the scm pull sem is a way to have your jenkins master monitor the github repositories and just check for any changes it's more simple than setting up web hooks especially if your jenkins master is behind a firewall so that's the method we're going to take here and i'm going to paste in a schedule here so it's star 5 star star star star so this is very similar to setting up a cron job what this star slash 5 means it's going to do it randomly every 5 minutes and this is a best practice with jenkins because you don't want all your jenkins jobs going out once per minute all at the same time this just helps the jenkins master distribute when the jobs go out and look at their github repositories so we're gonna keep it as is there go ahead and hit save and then i'm gonna go into my github here and i'll have a look at this script and i can go ahead and easily fix this just in github change it to hello world go ahead and commit the change and then i'm going to go here and just wait for the build to trigger i have it running like randomly once every five minutes so it should trigger here within the next five minutes i'm just going to go ahead and speed up the video all right so the job has completed and you can see if we go in here you can see at the very top that it was started by an scm change so this build was triggered automatically when we did a commit on our repository so we didn't have to go in and press the build button it just went ahead and ran the script so something that you can add to any of your repositories if you want the build to trigger when a change is made okay so so far in this course we've learned how to install and set up jenkins master as well as jenkins agents we've also been over how to set up a freestyle project and create basic builds the next thing we're going to have a look at is setting up ci cd pipelines using the groovy language now to get started with this all you have to do is go over to new item and then instead of going freestyle project you create a pipeline so we'll call this my first build pipeline and we'll go ahead and hit okay okay so this looks very similar to what we've seen before with the freestyle project but if you scroll down you'll notice that you don't have a lot of the options that you did before and that's because they're all going to be built into this pipeline script which we need to define down here now there's two main ways that you can get started with doing this the first one is you can just create your pipeline script directly into the jenkins ui or the second option is to use what's called a jenkins file and that's the preferred method you just have a jenkins file in your source repository and you point this pipeline over to that jenkins file and it takes in your pipeline script there we're going to start off just using this pipeline script but then later on in this video we'll check it into our source control and have jenkins check it out from there and run the pipeline that way now to get started here i'm going to head on over to my github repository and in there i have this jenkinsfile.template so i'm going to take this copy it and put it in here and this is a good basic template that you can start off with for all your projects and uh if we just scroll to the top you can see that it's got some nice santax highlighting this is in the groovy syntax which follows a json format and in case you are wondering this is a declarative pipeline this isn't the scripted format of a pipeline this is the declarative so if you're looking up syntax for this make sure you're checking declarative pipelines those are the ones that you should be starting out with when getting started with jenkins but let's go through this pipeline there's a couple things that i want to break down uh the first one mentioned before is everything is wrapped in this pipeline parameter and then the first thing we do is we select the agent that we want this job to run on so this is very similar to what we set up before to restrict which agent that the build runs on and we're specifying it based on the label so for our pipeline we want to set this to the label of our agent so i believe that was docker agent dash python and now that we have our label selected the next set of code here is the stages and within the stages we have individual stages so this is a three-stage pipeline first to build next to test and the last stage is to deliver so this is good for a test build let's go through the steps so you can see that for the stage build each stage has steps and there's two steps within each stage the first step is just doing a basic echo from the jenkins agent and then the next step is to just run a shells command and this is just a placeholder to also echo here but we're going to be adding in our linux commands to actually do stuff here so i hope that makes sense it's going to make sense as we start running these builds and improving on it so let's go ahead and hit save and let's go ahead and build and then if we go into the build here let's actually go back here and you can see immediately that the pipeline looks a lot different than just regular builds you can see here that it's separated into the three different stages that we have set build test and deliver and then you can see that build is green but test and deliver are red so it looks like our build failed at the second stage of our pipeline here now you can go into the logs of each of these and it'll tell you exactly what it was doing so you can see the first thing it did was print the message building and then i ran our shell script which just echoes right now let's go into our test one and it says testing and then the shell script fails and it looks like it's an unterminated quoted string so that should be easy enough to fix let's go into configure and build is good it was this one the test stage so we'll fix the quotation and our deliver stage looks okay now the reason that deliver failed even though it looked okay is because test was an upstream stage and basically if an upstream stage fails your downstream stages are going to fail as well so let's go ahead and hit build and uh i'll stay on this page this time and here you can see that uh it's going through the stages and since our build is so quick uh it just immediately updated but if you're doing a longer build you would see sort of like a progress bar all right so that's pretty good let's up our game by using a jenkins file and just improving this pipeline so i'm going to pull up my visual studio code here and i'm just going to take this copy it paste it and then just rename it to jenkinsfile and i actually need to fix this quotation here because it was missing and this should be good enough for our job but i actually want to improve our job before i upload this to github and the improvement i want to make is to have jenkins pull the scm which we did with our previous project and basically this means that jenkins will reach out to github and if there's any changes on the repository it's going to go through the build so i just looked up the syntax by going through a stack overflow article and this it looks like the one that we want and by the way this is how most of us devops engineers work it's just searching for what we want on google and then finding the stack overflow article and putting it in devops is just like building uh legos so anyways this should work uh i want it to pull every five minutes actually so i just manipulated the parameters so it will pull once every five minutes if there's new code on github it's going to run the build so i'm going to take this and i'm going to push it out to github here just do my commit and then within my actual project i'll need to convert this you can just get rid of that and instead of using pipeline script we want to change this to pipeline script from scm we'll say get we'll snag this repository url if you're doing this on your own make sure to change this to your github url and if it's a private repository you're going to have to use credentials mine's public so this will be fine and then the script path we just have it to set as jenkins file but you can name the jenkins file anything that you wanted and put it in any directory that you wanted you just need to make sure that this was up to date so this should be good i'm going to go ahead and hit save and i'm actually just going to trigger this one manually just to get it going but for future builds next time we do like uh git commit and push to github this build should trigger automatically so let's just go ahead and make sure the build runs using the jenkins file i'm just going to hop right into the console output so it obtained the jenkins file from get so we're looking pretty good here and i actually forgot to update this so this job is just going to run forever let's go ahead and change my template sorry not my template change the actual jenkins file and change this to docker jenkins docker agent python all right i have the memory of a goldfish let's go ahead and these changes look good i'm going to put it up here and just go updated jenkins file and i'm just going to push this way double check github make sure our changes are there all right that looks good to me and this should run let's just go ahead and cancel this build and uh i'm actually just gonna go grab a drink wait five minutes and see if my build automatically triggers all right so i left it for about 10 minutes here and it didn't look like it automatically triggered if we go into configure uh you can see that it picked up these poll sem settings from the jenkins file and it looks like it should have run i'm just going to be impatient here and i'm gonna hit build now and uh in here my jenkins file i'm actually just gonna change this to star and then this way it'll just run once a minute and uh that's fine for our lab environment anyways let's go check on the build and see how it's going it looks like it was successful it ran on the docker agent python and uh everything ran successfully and if you look up there it used the jenkins file to do it now one thing you may notice is the output of this changed and that's because an additional stage was added it actually added the checkout stage and the reason it did this is if you go into configure underneath the pipeline we're saying pipeline script from scm so when you do that it automatically adds the step to clone down this github repository so that's the reason for that um one thing that you'll notice here is it cleared out the history of the old jobs because the additional stage was added but that's okay it's just one of the little quirks of jenkins let's go ahead and build out our pipeline more so i'm going to go back into visual studio code and right here we have our three stages the build test and deliver let's actually flesh this out so you can see in the repository i do have this application called my app and if we just look at it you can see it's a python file and as you can see this is a pretty simple script what it's doing here is it's first importing the fire library and then it's just creating a cli application that returns hello world now if we were to run this script on our jenkins agent it would fail because it doesn't have the fire library so let's actually go to our jenkins file and for build instead of echoing let's say cd into my app and then go pip install dash r requirements.txt so now the jenkins agent will make sure that it has the fire library installed and then for testing let's go ahead and run our script so we'll go python3 hello dot pi and uh there is actually two ways that i could do it it should work if i just run the python application by itself but it'll also accept a parameter so let's run it both ways so we'll go python3 hello.pi name equals brad and actually will need to be in the my app directory so that's perfect so the first thing it's going to do is install the requirements for this application and the next thing it's going to do is run the application and make sure that it's running okay so i'm going to save this and push it over to my repository and let's uh go here oh and it looks like it's already building our uh our job here it was automatically triggered through scm uh checkout went good now it's going through the build and test and deliver all right let's have a look at how these look so console output you can see the cloning went through fine the next thing it did was it did pip install requirements.txt so it installed the fire library and then it went into the testing stage and then it ran python hello hello world and then it ran it with a parameter brad hello brad and you can see both of them worked so let's go ahead and have a look at this in blue ocean and i'll just show you how the interface of blue ocean looks so much nicer than this sort of outdated jenkins pipeline so if you just go to open blue ocean you can see that the pipeline has all our different stages here it started build test deliver and if you click on each individual stage you can see exactly what happened in these stages so you can see the checkout from version control our echo message and then what it actually did and uh went into the my app installed the requirements.txt so blue ocean is pretty good for giving you a nice interface so you can go in and look at the build and have a look at what happened in each of the stages of the build a lot of your job as a devops engineer is going to be troubleshooting these pipelines so having a nice interface to do it is always good and blue ocean just really cleans things up as you can see so that's basically all i want to show for blue ocean if you follow along with this video then you should have it available to you as well to go in and here's the main page of blue ocean so we can see our actual pipeline here and if we look we can see oh it was started by me basically when i clicked the go button and then this one was automatically triggered and yeah you could run the build from here if you wanted you could disable the build and if you just go to configure it just brings you back to the old jenkins interface so that's all i really wanted to show for blue ocean it's a really cool thing to get into especially when you get to the multi branch pipelines but for our purposes it's just a nicer looking jenkins interface that'll help you troubleshoot jenkins jobs so that was your intro to jenkins in this video we went over the key skills such as installing and setting up a master server setting up docker cloud agents we also covered how to set up and manage freestyle projects and lastly we went over setting up pipelines using the declarative language groovy i hope you enjoyed this video course and found it gave you practical skills on how to use jenkins if you have any questions leave a comment below and if you haven't already please check out my other videos and subscribe to the channel thanks again for watching and i hope to see you in the next video

Channel: DevOps Journey

Views: 344,867

Keywords: ci cd jenkins, ci cd pipeline using jenkins, ci/cd jenkins, cicd using jenkins, devops, devops how to, devops training, devops tutorial, docker jenkins, jenkins, jenkins ci cd pipeline, jenkins ci cd pipeline tutorial, jenkins course, jenkins crash course, jenkins docker, jenkins for beginners, jenkins full course, jenkins pipeline, jenkins pipeline tutorial, jenkins tutorial for beginners, jenkins tutorial windows, learn jenkins, what is jenkins

Id: 6YZvp2GwT0A

Length: 68min 28sec (4108 seconds)

Published: Fri Jun 24 2022