Jenkins Multibranch Pipeline With Git Tutorial

in today's video we're going to talk about creating multi-branch pipelines in jenkins using git [Music] if you're new here welcome and if you are new here you don't know who i am my name is darren pope and i'm a developer advocate for cloudbees when you first start out with jenkins you're probably going to be using freestyle jobs or maybe pipeline jobs in today's video we're primarily talking about pipeline when you start with a pipeline job that job is probably meant to do some random things or maybe you're just building off of one branch but let's imagine for a moment that you're working on an application development team that requires you to have multiple branches within your git repository that would mean that every time you create a new branch you would have to create a new pipeline job inside of jenkins in reality that's a lot of wasted hours in creating and maintaining separate jenkins files this is one of the reasons that the multi-branch job was created so in our example today we are going to be using a variation of git flow maybe a little bit of github flow as we create our multi-branch pipeline job as we go through our example today we are going to be focusing on git now although my repository is hosted at github i'm not going to be using any github specific items today i'm just using get taking a look at this repository this is just a basic start.spring.io example right now all i have is just the code and notice there is no jenkins file in this repository at this point we'll add one in a little bit later so let's log in to our jenkins controller and we are going to create a new item and we're going to name it let's name it the same as our repository that way it's really clear and simple to know what our mapping is just looking at the job so we're going to do that and select on multi-branch pipeline and let's click on ok now what we're going to do is we are going to add a branch source of get now remember as i said a few moments ago even though my repository is hosted in github i'm just going to be using the get version of branch source today in another video we'll come back and do a another run through of this using the git hub version of this so just get today my project repository i am going to do code i'm just going to copy the https this is a public repository so you can look at it if you want to in the future i won't need any credentials since it is a public repository and we're just going to start out with the basics so i selected git from the branch source drop-down i've put in my https url i don't need to add any credentials because it's public and we'll leave all the defaults here now let's go ahead and click on save and see what happens so what's when we clicked on save there was a scan of that repository that was kicked off so it was able to go in and it fetched out all of the branches which right now there is only one branch in fact let's take a look at that real quick we can see there is only one branch and that is the main branch let's go back to here okay as it was checking the branches it checked the branch main and it did not find a jenkins file because well we haven't created one yet so processed one branch and it's finished so if we come back over and take a look at multi-branch sample app there are no jobs here notice this phrasing here this folder is empty a multi-branch job is effectively a folder of pipeline jobs it boils down to that and in future videos as we discuss some other job types you'll see this pattern continue to repeat itself so instead of having to create separate pipeline jobs for each of the branches that i would want to create jobs for now i can set up a single multi-branch job and based on how i configure this job it will find the branches that i will want to build but it boils down to a multi-branch job is a folder of pipeline jobs so going back over to the log notice it says jenkinsfile not found what does that really mean let's go back over here to configure and right below the branch sources section there is a build configuration and with build configuration there is a mode by jenkins file if you click on that drop down that is the only option and then the script path is jenkins file so i could change this name to anything that i wanted it to be but the convention is jenkinsfile okay so let's go ahead and create a jenkins file and push it up to our main branch so i'm going to go ahead and say vi jenkins file and i have an example pipeline that i'm going to use here and again it's fairly simple i'm just going to echo out hello and i've got a build discarder set up for keep the last five and i'm disabling concurrent builds and i'm using an agent that has a label of linux so it's a very very basic pipeline nothing really fancy here so let's go ahead and save this let's add it to the repo let's commit it and let's go ahead and push it up now although my repository is public i have authenticated with it so i can do all of my get interactions from the command line let's just go review that real quick so if i refresh this page we'll now see a jenkins file here added 23 seconds ago we're still we still only have one branch okay now that we've pushed that up we're now hitting a different item since i don't have any kind of web hooks set up this job was not notified of the changes that just happened in that repository so there are a couple of things you can do the thing that we're going to be doing today since i'm effectively air-gapped i am working on a network that has no inbound internet access so this would not naturally receive a web hook in future videos we'll go through some different options of how to do that if that's your setup if we take a look at the configuration one thing we could do is we could scan periodically if it's not otherwise run now what does otherwise run mean if we take a look at it sometimes there's an event to where we we receive a web hook or a job happens and new branches are found but since we haven't run anything and we didn't receive a hook nothing automatically happened for this job so what we're going to do i'm going to leave this page i'm going to do a scan and i'm going to click on log right underneath and now you'll see that checking branch main we have a jenkins file found and here's the other thing now here's main as it found by default as it found that new branch with the jenkins file it went ahead and fired off a job for that branch so it found the branch it found it got the jenkins file and then it echoed out the hello i want you to call out one thing here we just did a git checkout of this hash which is right now if you take a look at if i refresh this de7 dd is the same commit hash that's right here so keep this in mind as we go through the rest of these the rest of this example okay so now we've got our main branch building let's take a look at what we have here i've got a main branch going back to that concept of a multi-branch job is a folder of pipeline jobs so right now that jenkins file is on the main branch and that is the jenkins file that was used to build the main branch if i click into main notice over here on the left nav there is a build now so i could build just for the main branch i can view the configuration and it shows oh max number of builds to keep as five with artifacts 5 you see everything is greyed out here it goes back to this is not the job that can be configured you cannot configure main the only job you can configure is the multi-branch in a couple of minutes we'll get into how to deal with okay for certain branches i want to do this and for other branches i don't want to do this we'll get into that in a few minutes okay so the job ran it ran successfully and everything looks good now we're doing multi-branch so we need more branches so let's go create a couple of new branches first off and clear this out we are going to do a git check out dash b we're gonna call this fix dash one two three and you can see here that we are now on our fix 123 branch let's go back over to the main branch and let's do one more checkout and we're going to name it a little bit differently this time it's going to be named dev-456 and we're on the dev456 branch okay we've created the branches locally but now we need to get these branches pushed up to our git repository that the multi-branch job is going to be using so to do that we're going to do a git push set upstream origin dev dash 456 and we are on 456 here 456 that's correct and then let's check out the fix 123 and we'll do the same thing there set upstream origin this is fix one two three okay so now what i want to do so let's take a look at our git repository and now we have main fix and dev now let's go back over to our multi-branch job and let's rescan this to see what happens so we'll say scan multi-branch pipeline now and let's take a look at the log we can see that a jenkins file was found for fix 123. a jenkins file was found for main and a jenkins file was found for 456. and we can see that all three branches are now showing up inside of our multi-branch job but let's assume for a moment that the only branches that we want to build are branches that are main or start with fix dash now we could say i want to build everything except for dev dash how do we do this let's go back into configure and under the behaviors for the get branch source we are going to first under within repository we're going to go down to filter by name now there is a filter by regex but this is a little bit simpler to understand to begin with i could say i want to exclude anything that begins with dev dash and then include the star as the wild card so the way that this is currently written give me everything except for dev dash star but that's not really what i want i want to only include main and that's a space anything that begins with fix dash and then i include a star you can take a look at the help and it will explain to you what the requirements are so this way when we scan the repository the only branches that will be built will be maine and fix dash or any branches that begin with fix dash now there are a couple of other behaviors that i typically add in not always but i typically include them if we go under additional there are three i will do check out to matching local branch and if you take a look at the help it will always make sure that it's going to build as head on that branch so make sure you're checking out to the matching local branch the final two that i include are clean after checkout and clean before checkout that way i am working with a workspace that i know that is clean this isn't a full reclone there is a there is an option for that to where you could wipe out repository and force a clone that is not what check before check out or clean before and clean after do but if you wanted to do a full wipeout go for it it's right there you don't have to do any kind of weird deleting or anything else you can do that as a behavior i'm going to add clean after and clean before so let's review what we've done here we've added a filter by name with wild cards to only include main and any branches that start with fix dash i've also added in three other additional behaviors check out to matching local branch clean before checkout and clean after checkout let's click on save and as we save it a new scan happened we talked earlier about what could trigger a scan it could be a web hook or some other type of event anytime you save the job after changes that will also trigger a scan now what we have here is just a little bit of ui weirdness let's take a look and see what happened here if you take a look and read real carefully everything is normal through all of the get items here but during the checking branches there's the fixed dash no changes so it's still at the same one it found main again no changes detected still at the same commit but notice right here at the bottom evaluating orphaned items since there was a branch and we'll see in a moment that deb is now gone after it goes through and does the scan it will check for anything that's orphaned so since we added in that filter by a wild card now it's going to go through and find anything that doesn't match main or fixed dash and we can see here that it would it will remove dev456 so let's take a look at the job now and see what we have we're back to just having main and fix 123 and everything is still showing good there okay so now we're working in our fix branch so let's treat this like we would normally treat a file in either git flow or github flow what i'm doing right here is probably closer to a github flow than git flow because i don't have all of the branches that are necessary from a definition perspective for git flow but the concepts are at least this level fairly similar all right so let's click into here we are currently in the fix 123 branch i want to modify my readme file i'm going to say new content okay and then what i want to do here is i'm going to add that change and i'm going to go ahead and commit it but i'm not going to push it up just yet because i want to make one more change i want to make a change to my jenkins file and what i want to do here is i want to add a new stage and this stage is only going to apply to the branch whoops to the branch or any branches that start with fix dash so again i'm following this wild card here so every branch will get hello but only the branches that start with fix dash will cat out the contents of the readme.md file so let's go ahead and save this file oh before we do that make sure i clean that up yeah get rid of that space there we go nice and clean okay and we will do the same thing here to where we'll get add jenkins file get updated change file okay so let's take a look at the get status so we're all good there by two commits let's go ahead and push this up now if we come back over to our git repository oops that's not what i wanted when we flip over to our fix 123 branch we can see that the readme has new content main does not and also on fix 123 we have our change for our win branch fix dash okay so everything is the way i expect it to be and just reiterating if i had web hooks eventing something set up to where this repository would have seen those changes then this job for fix 123 would have already fired off but since we don't i'm just going to run this manually so if we click on fix 123 let's watch this log and see how this looks a little bit different so it gets into the normal starting point of checkout but notice something down here that we called out earlier before we saw a git checkout dash f and then the commit hash but now we see get checkout branch name fix 123 and b6ea well if we take a look at our fix 123 branch there's the latest branch for fix 123 which is b6ea we have hello which is what we expected and since we are on a branch that starts with fix dash then it echo or it catted our readme.md file everything's working as expected now if i go through and run my main branch job again we'll see a variation of what we just saw on the fix branch we see a git checkout dash b main and we don't see anything regarding the cat stage so everything's working as expected now again going back to the the basis of today we are working with get although my repository is at github i do not want to use a pull request to merge in the changes from fix dash 123 into main just think of this as a normal plain git repository so what i'm going to do and there's numerous ways to do this i'm just going to do it the way that i would typically do it so what i'm going to do is i'm going to get checkout main let's clear this up here so i'm now on the main branch i'm going to merge in the changes that i have in my fix branch so i have my jenkins file and my readme those were the two file changes that i made on the fix branch i am now going to delete my fixed branch locally and i'm also going to and let's go and push that change up so now main will be fully up to date and i'm also going to go ahead and remove my remote branch okay so let's go back over to our git repository you can see we now have two branches we have main and dev but remember from our multi-branch configuration we are ignoring any branches that do that are not either main or start with fix dash so we we've completely removed fix dash let's take a look at our sample app now in order for this branch to disappear we have to rescan if i click on the log much like we saw when we had the dev dash branch removed we also see now that the fix dash was an orphaned item that has now been removed so the only branch that we're finding right now is the main branch so if we come back here to multi-branch the only branch that shows up in the listing is the main branch now let's go ahead and run this main job one more time if we look at the output this time we're going to see something a little bit different same thing check out dash b there's the commit hash b6 ea which came in from the merge from the fix dash branch but check this out there's hello we now see the stage for the cat but this stage was skipped due to the win conditional remember that the win conditional was only used this stage when the branch started with fix dash and that was a quick look at how to set up multi-branch pipeline jobs using git in future videos we're going to be looking at different ways to integrate with different providers such as github if you have any questions or comments you can reach out to us at cloudbeesdevs on twitter or you can reach out to me directly on twitter at darrenpope if this video was helpful to you today please consider giving us a thumbs up and if you haven't subscribed to cloudbeast tv yet why not take a moment click on that subscribe button and then ring that bell and you'll be notified anytime there is new content available on cloudbees tv thanks for joining us today and we will see you in the next video

Channel: CloudBeesTV

Views: 11,028

Keywords: darin pope, jenkins multibranch tutorial, jenkins, github, git, jenkins tutorial for beginners, jenkins pipeline tutorial, jenkins training, devops tutorial for beginners, introduction to jenkins, introduction to jenkins pipeline

Id: B_2FXWI6CWg

Length: 29min 53sec (1793 seconds)

Published: Tue Feb 16 2021