How to Create a GitLab Multibranch Pipeline in Jenkins

today we're going to talk about how to create a multi-branch pipeline in jenkins using get lab [Music] are you new here if you are welcome and if you are new here you don't know who i am my name is darren pope and i'm a developer advocate for cloudbees in the past when you were creating pipelines for gitlab projects anytime you created a new branch you would have to create a new pipeline job with a multi-branch pipeline all of that is now in the past let's talk about the tooling that we're going to be using today i have a jenkins controller that is running version 2.27.1 when it was installed it was installed using install suggested plugins and at the end of that once we logged in i also installed the get lab branch source plugin and the version of that plug-in at the time of this recording is 1.5.4 if you want to be able to do multi-branch pipelines with get lab get lab branch source currently is your only option i also have an agent connected to this controller with a label of linux and i also have a tunnel set up via ngrok so i can receive web hooks from gitlab.com now you might be using self-hosted gitlab all of these setup steps should be the same but if not be sure to refer to the documentation for the get lab branch source plugin the link for that documentation is listed below this video and as we get started taking a look at my account within git lab i have two projects i have a micro one project and a micro two project in today's video we're only using the micro one project if we take a look at it all that exists in here right now is a readme file there's no jenkins file or anything else just a readme file the first thing that we're going to need to do is to create a couple of credentials within get lab we're gonna need to create a personal access token and we're also going to need to set up an ssh key so we'll be able to check out this repository the micro one repository is a private repository so to do this we'll go up to my icon we'll click on edit profile and i am going to click on access tokens i'm going to give it a name of jenkins dash darren pope i'm going to let it expire on this friday and according to the documentation for the get lab branch source plugin the only scope that we need to add is api so we'll select api and then click on create personal access token the access token will render at the top so i'm going to copy that and save that for later usage the other thing that we need to set up is an ssh key so while we're still here in our user settings we're going to go to ssh keys and we need to add in the public key of a ssh key pair so i've already created one off to the side so i'm going to copy in my public key so we'll come back over here and i have now pasted in my public key i'm also going to expire this on the 26th you may not want to expire things that's up to you let's click on add key and we can double check that by scrolling down and see that we have our ssh key here so now that we've created our personal access token and our ssh key or at least the public side of it let's go into our controller and let's create our credentials so we'll go to manage jenkins manage credentials global and then add credentials for the gitlab personal access token we are going to use a credential type of gitlab personal access token so the token value is going to be the value of the token that was generated for us and i'm going to give it an id of jenkins-darin pope named the same as what the personal access token is that's purely by coincidence but it helps me keep things straight we'll click on ok the second credential that we need to add is an ssh username with private key now in this case i'm going to give it an id of darren pope dash ssh key i'm going to give it the same description the username is going to be the username of gitlab that i'm logging in with so in my case that is darren pope and then the private key is going to come from the private key that i generated from the key pair okay i've copied that and let me paste that in so there is my private key and my ssh key pair does not have a passphrase or the private key doesn't have a passphrase so we'll click ok so what we've done so far is we've created our personal access token and the public side of the ssh key over on git lab we've now created within our controller a github personal access token and an ssh user with a private key before we can create our job we need to do one more thing we'll go back to dashboard manage jenkins and configure system this last thing that we need to create is we need to set up our gitlab configuration with the credentials so it can manage our web hooks for us so as you scroll down configure system you'll see get lab and then get lab servers by default there is a default one here but it's pretty empty so we'll say credentials and we'll select our get lab personal access token that we set up we're going to check manage webhooks we are not going to check or check the managed system hooks i don't care about system hooks for today we don't need to do anything with secret token but the root url for hooks even though we don't really need to set it up just for clarity i'm going to set it to the value of our jenkins location url this ngrok url is already set up above but i'm doing it here just to be redundant and for clarity now that we've done that let's click on test connection and what this tells us is that the credentials are verified for user darren pope this is what tells us that the get lab personal access token credential that we set up is working as expected let's go ahead and click on save now we're ready to create our multi-branch pipeline so we'll go to new item we're going to give it a name of micro1 the same name as our gitlab project we'll scroll down to the very bottom at least in my case and click on multi-branch pipeline we'll scroll down to branch sources and select get lab project the server is the default one this is the one that we just set up and right now it's the only option the checkout credentials are going to be the ssh key pair that we set up so in this case it's darren pope ssh key the owner is going to be the account on get lab so in my case that is darren pope and i'm going to tab out of this field and watch what happens inside of the projects drop down right below as the tab was clicked it made a call and gave me a list of all of the projects that are available to me today we're doing micro one and at this point we're completely set up we don't need to make any other changes for this first part now you'll notice that we have just a handful of default behaviors discover branches that's how we'll find our branches and we'll also be able to discover merge requests from either origin or from forks so let's go ahead and click on save and let's see what happens so we're scanning our gitlab project and this message is a little misleading it says jenkins file found but is a directory not a file so what that actually means is jenkinsfile was not found because if you remember back over in our micro one repository all we have right now is a readme and that's it so this message is a little confusing so if you see this that basically means jenkins file was not found if we click back on micro one in the breadcrumb what you're going to see is a couple of things happened number one is it changed to include the account name the description of the name which in my case is first space last name and then a slash and then the project name you'll also note that this folder is empty because it wasn't able to find a jenkins file going back to our log it was looking for a file named jenkinsfile where did this come from if we go back over to configure and scroll down below branch sources we are going to see something called build configuration the mode that we have selected and it's the only mode that's available to us is by jekins file so what's happening is when that scan occurred it was looking for a file named jenkinsfile since there was no jenkins file found in the root of that project then that's why nothing was set up for us now that we know why the jenkins file is important let's go and create a jenkins file in micro1 so i'm going to say vi jenkinsfile and i have a simple declarative jenkins file so all that this jenkins file is going to do is it's going to run a shell step to say echo hello on our agent that has a label of linux this is a very very simple pipeline let's go ahead and save that we're going to add it we're going to commit and we're going to push now one other thing that we didn't review after this was created you know even though we did not have any branches found the job was still created so if we take a look back at micro one and go over to settings and web hooks we can see that a web hook was set up for us at b95 which is what our controller is so what i expect to happen and what should have happened when we did that push is that the branch would be found so we now received a web hook from get lab we can see that main was found and then if we take a look at the number one we can see our echo hello so the jenkins file did exactly what we expected within the main branch so now let's create a new branch off of main named fix 123. so we'll say get check out dash b fix dash 123 and now we've done that but we don't want to make any changes to this branch just yet so let's go ahead and push this up to git lab so once we push that new branch up to get lab a web hook was fired we're listening for it let's see what happened so if we go back to micro one we now see fix 123 is here and we can see that number one is running if we take a look at the output it's exactly the same output that we had for main which is again what we would expect at this point because we didn't make any changes so to recap we now have a main branch we have a fix 123 branch and no merge requests now let's make a change to the jenkins file that is on the fix dash 123 branch we're going to add in two new stages we've added in a stage for fixed branch so anytime we are on a branch that begins with fixed dash we're going to cat out the contents of readme.md anytime we're on a branch that starts with mr dash in our case merge requests then it's going to echo out this only runs for the mrs so let's go ahead and save this add the changes and let's push this up and see what happens so what we should see now is fix 123 is currently running or almost running let's see what happens there's two it's doing the pull there's our echo hello there's the cat out of the readme because we're on the fix dash 123 branch and the stage for the mrs were skipped because fixed dash does not equal mr dash so this jenkins file for this fixed branch is working as we want now that we've done that let's go and create a merge request so we'll go back over to git lab scroll back to the top click on micro one at the top here we can just click on create merge request we want our changes from fix 123 into main we also want to delete the source branch so in our case fix 123 we want that deleted when the merge request is accepted so let's go ahead and click on submit merge request so now that we've done that let's go back over to our controller click on micro one we still have main we still have fixed 123 but now you'll notice that we now have a merge request and that is mr5 well the 5 can be referenced here in this path so we can see this is mr5 we take a look at mr5 and take a look at the output then what we have here is our echo hello our fixed dash stage is skipped but then the mr stage does run because this branch started with mr dash now that we've reviewed all of this let's go back to micro one and recap this one more time we have main fix 123 and the mr5 merge request let's go ahead and merge this merge request also notice that we have delete source branch checked so when we click on merge what's going to happen is we can see that the changes were merged into main if we go back over to our job and we click on micro one here what we're going to see is now that fixed dash 123 is now obsolete as indicated by the strikethrough we can also see that the merge request mr5 is now obsolete and if we want to clean those up instead of waiting for them to be cleaned up we can just say scan gitlab project now if we click back on micro 1 what will happen is we will see now that all we have left is main and there are no merge requests finally let's take a look at main look at build number two and we can see here that we have our echo hello we have the stage for the fix branch was skipped because main doesn't match fixed dash we also see that the mr was skipped because main does not start with mr dash so now we have all the changes to our jenkins file that we want on our main branch and that's it to recap in order to use git lab with a multi-branch pipeline job within jenkins we had to install the gitlab branch source plugin and once we did that we needed to create two credentials on the gitlab side a personal access token and an ssh key because we have private repositories then over on our jenkins controller we needed to create two credentials one for the personal access token and one for the private key part of our ssh credentials and then we set up our configuration for gitlab using our personal access token so that web hooks could be managed for us once we did all that setup we were able to create the multi-branch job we went through a few scenarios to where we added a jenkins file to a branch we created a new branch off of that branch we created a merge request merged it all back together and now you can see instead of having to create new jobs for anytime you have a new branch within a gitlab project consider using the gitlab branch source plugin to simplify your work either as a jenkins administrator or as somebody that just has to maintain jobs it will be a great time saver for you if you have any questions or comments you can reach out to us on twitter at cloudbeesdevs if this video was helpful to you consider giving us a thumbs up and if you haven't subscribed to cloudbees tv yet why not take a moment click on that subscribe button and then ring that bell and you'll be notified anytime there's new content available on cloudbees tv thanks for watching and we will see you in the next video

Channel: CloudBeesTV

Views: 4,220

Keywords: darin pope, jenkins, jenkins webhooks, gitlab webhooks, jenkins multibranch, jenkins gitlab project tutorial, jenkins gitlab merge requests, gitlab branch source plugin jenkins

Id: y4XGFluzPHY

Length: 20min 32sec (1232 seconds)

Published: Tue Mar 30 2021