How to Create a GitHub Branch Source Multibranch Pipeline in Jenkins

in today's video we're going to talk about how to create a multi-branch pipeline in jenkins using github [Music] are you new here if you are welcome and if you are new here you don't know who i am my name is darren pope and i'm a developer advocate for cloudbeast you may have watched a previous video where we did a multi-branch pipeline in jenkins but we were using git now in that video i used github as my git repository but the primary purpose of that video was to show how to integrate with a vanilla get repository as we go into today we're going to be integrating with github and we're going to be using a github app to provide the authentication for us let's set up where we're starting at i've got a jenkins controller that is 2.263.4 i've got an agent connected to that controller with a label of linux and finally i have a tunnel set up via ngrok so i can send my web hooks from github into my jeekens controller for our application today we are using the multi-branch sample app repository and that will be feeding our multi-branch job to start with we're going to be creating our github app we'll be leveraging that github app within our jenkins controller to be able to do our authentication the step-by-step instructions for this are included in the cloudbeads documentation a link to that documentation is included down in the description okay let's get started on setting up this github app let's go over here in github i'm going to go over to my profile i'm going to go down to settings and i'm going to open in a new tab okay let's go to this new tab cool let's go ahead and scroll down to developer settings and here is github apps now currently we don't have one set up let's click on new github app you may have to enter your password okay here we go let's give it a name and this name is just a name we will reference it when we set up our credentials over on our jenkins controller but it's just a name but just for consistency i'm going to say jenkins darin pope that just tells me okay hey this app is for jenkins and i'm mapping it for my darren pope organization within github okay we don't need to fill out a homepage url we don't need to fill out a callback url we don't need to do setup but we do need to add in a web hook now if you're not using web hooks you don't have to do this but because i want to use a web hook so we can see how other things work i'm going to enter one in here so let me grab my webhook url and this is my current ingrock tunnel url so i'm going to be sending it to internally my jenkins is jenkins 8080 but this ingrock tunnel is forwarding to that jenkins 8080 port and then we also have to pass in slash github dash web hook don't forget this trailing slash i was wrong you don't have to specify a homepage url but i'm going to and mine's actually going to be super lazy i'm just going to make it just my home page on github okay so we've filled out three things so far github app name i gave it a sluggified name you can give it a human readable name if you want to homepage url and i've also included a web hook that i'm going to use now for the web hook i'm also going to leave the enable ssl verification turned on okay now we need to get into repository permissions if you don't follow this as i'm going through the video refer to the documentation and it will walk you through each one of these things that you need to set okay so for administration i need to set to read only for checks i'm going to do read and write for contents i'm going to do read only for metadata i am doing read only which was already set for me pull requests i am doing read only commit statuses which should be right here commit statuses or read and write scroll down a little bit and then optionally you could set web hooks now since i am not going to have my jenkins controller manage webhooks for me i'm not going to set the webhooks since i'm i'm going to be manually setting up the webhook for this example if you have access and you already have are allowing traffic in from github you and you want the jenkins controller to be able to manage your web hooks for you then you would enable that to read and write so right now i'm going to leave it no access if you want the controller to manage it read and write okay so those are the permissions that we needed to set on the repository so there are a total of one two three four five six that i set and there was an optional one for web hooks that i did not set now that we've set our permissions let's go ahead and scroll down so organization permissions we don't need to do anything there all of those can be left as no access user permissions again no access so the only thing that we have to set from a permissions perspective is the repository permissions okay so we'll go past org we'll go past user now we need to subscribe to events if for some reason you do not see these events available to you where you see a much smaller list what that means is that you did not select the correct repository permissions so what you want to do is rewind re-watch the permissions pull up the documentation go through it okay so let's go through and set up our subscribe to events we have five that we're going to subscribe to the first one is check run the second one is check suite the next one is pull request which is down here then push then repository so let's go through those one more time we have check run check suite pull request push and repository so those are the events that we're going to be subscribing to let's go and scroll down a little bit more where can this github app be installed only on this account now most people would probably do this because you may not have multiple organizations you could select any account it can be installed by any user or organization but i don't need that much exposure so let's say only on this account okay so let's go ahead and create this github app now we are landed if you take a look at the breadcrumb you can see where we're at so we're we're in settings developer settings github apps and this is basically the detail page for the github app jenkins-darin pope now what we need to do here is we need to generate a private key and what you need to do to do that is you'll scroll to the bottom of this page this is the detail page keep going keep going keep going and here you would generate a private key and then it says oh hey won't we go ahead and save this so you need to save this file so i'm going to save it locally and then once you have saved it you still need to do one more thing and the details for this are in the documentation so you need to go refer to that okay so let's take a look at the key so there is the key that we downloaded what we need to do is we need to change this into a different format and again this is in the documentation so i'm going to say open ssl all the different options the n is going to be the file that i just downloaded and then the out i'm going to give it a different name and then i need to give it one more option no crypt so now i can see that i have a converted github app pim that was just generated and we are going to use the contents of that newly created private key the converted github app key we're going to use that value inside of our credential in a few moments before we leave github we need to install this github app into our organization so right now we are still on our detail page for the github app jenkins dash darren pope so from here we're going to click on install app and you'll see that you'll have your organization here or you may have a list of organizations so i'm going to click on install where i want it installed and for me i'm only going to install it i could do it for all repositories but from a security perspective i may not want that app to apply to all repositories because maybe my organization is not set up in a very nice and clean way much like my personal organization isn't set up in a nice and clean way so i only want to select a specific repository and it's going to be the one that we're using today which is multi-branch sample app and then it shows us the permissions those were the permissions that we gave the repository during our setup of the app and click on install okay so now it dropped us back out to our profile page excuse me that's not yeah it is uh so we're it back in our po top of our profile page and it shows the app being set up here and the repository that it has access to if you needed to add in other repositories in the future you could just come here and add them or remove them depending on what you're needing to do okay so now that we have our github app set up we're going to go over to our jenkins controller and set up the credentials for the github app so it's actually pretty simple we'll just go over here to our dashboard click on manage jenkins go to manage credentials click on global click on add credentials the kind that we're going to use is github app for the id i'm just going to give it the same name as what my github app is just so i can look at things and understand it the app id this is where you're going to have to go back over to github we need to go to the details page of our app so we'll click on developer settings we're going to click on jenkins darren pope we want to get the app id right here from the top so in my case it is one zero two one zero five so i'll copy that paste that in here and then the key is the secret key that we generated but not just the one that we generated but the one that we actually converted so i'm going to grab the value for that private key and i'm going to drop it in here tab out and then i'm going to click on test connection and we can see here that it was successfully authenticated and we have a remaining rate limit of 5 000. you may have watched in another video about using a personal access token go read the documentation that's down in the description and it will give you the reason why a personal access token may not be the best choice for your scenario today at the time of this recording using a github app is going to be the best solution for you in the future it may be something different but for today github app is what we want to use okay so let's go and click on ok so now the credential is saved let's click back over to dashboard and now we are ready to create our job let's create a job we call it multi-branch sample app scroll down and select multi-branch and click ok we'll go under branch sources we'll select github we're going to select our credentials that are the credentials for our github app we can see that github app verified remaining rate limit 5000 we're going to grab our https url let me copy that i'm going to click on validate credentials okay connected to app and that's all that we're going to do right now so credentials give it a url no other changes just click on save as it scans it found one branch if we take a look back at the job there are no branches found because if you remember we do not have a jenkins file in the root of our repository looking back at our job configuration our build configuration is by jenkinsfile and it was looking for a file named jenkinsfile since we didn't have one it didn't find the branch so what we want to do now is create a jenkins file in our repository so let's go back over here i'm going to add a file create a new file i'm going to call it jenkinsfile and the contents of that file are going to be pretty straightforward if you watched other videos this is the exact same pipeline that we've used before gonna agent label linux a couple of options and we're just echoing out hello good enough to get started so let's click on commit new file so that will add and commit all at the same time let's go back over to our job and we can see that this job has now started so the web hook is the web hook so the web hook was received and it's being processed in fact we can see that push event to branch main and we're just waiting on this to do the clone and see how it goes so we got a hello and everything is done so this is very very good so we made a change to our repository in this case we added jenkinsfile a webhook event was sent the jenkins controller received it processed it the job was started and now we have main what we want to do next is we want to create a new branch in our repository so let's do git checkout whoops check out that should be we'll call it fixed-123 and now that we've done that we want to go ahead and push this back up to github which is git push setup stream origin fix 123 cool so if we take a look now at our repository in github we can see that we now have two branches we have main and fix 123. if we come back over to our job what we should see now is that fix 123 was automatically created for us this is the wonders of a multi-branch pipeline we add a new branch a new job is automatically created for us we did not have to go in and manage this job so let's go ahead and go into fix 123. come back over here let's switch to fix 123. we'll just do our editing here in the ui be a little a little bit easier a little bit cleaner let's modify our jenkins file so we're on the fix 123 branch let's edit this we are going to add in two stages we're going to add in a stage that is meant to only be run when we are on the a fix dash branch and not too surprising we're adding in a stage to be able to handle pr branches let's click on save now let's get rid of the empty space first okay let's take a look back over here first at fix 123. let's we can already see that build number two is starting but let's take a look at number one everything was what we expected there there was just the hello because we had not added that other change yet now that build number two has completed and you can sort of see this in the output here we have the hello stage for the fixed branch for the pr so if we take a look at number two we can see that we catted out the contents of readme which was these contents for the stage that was the contents for the stage for the fix branch and we can see that the stage for the pr was skipped due to the conditional because the fixed dash is not a pr branch so so far we have a main branch a fixed branch and nothing currently in pull requests but let's change that let's get ready to create a pr so what we're going to do is go back over here to multi-branch fix 123 had recent pushes a minute ago so let's compare the pull request we'll do a compare and set it up so fix into main we updated the jenkins file and these are the two stages that we added one for the fix one for the pr let's go ahead and click on create pull request now we're not going to click on merge just yet let's go back over and see what this looks like let's refresh this page we have our two branches and if you were paying close attention you saw now that we have one pull request this pull request is pr7 well if we take a look back here this is number seven let's take a look at the output of pr7 we have our hello stage we skipped the stage for the fixed branch because pr isn't the branch and then here is the output for the pr and that's it so now we've got a jenkins file that is branch aware including pull requests not that you would do that differently but you might this is how you would implement that okay so what we've got now before we continue on we have main branch fix branch and we have a pull request let's go back into our pull request and let's go ahead and merge it confirm the merge now the way i have my repository set up in github i have it set up to automatically delete the branches once a merge is complete or once the pull request is merged into the main branch in this case so let's head back over here let's refresh this we can see that even though it's reflecting there's still one pull request pr7 actually has a strikethrough same thing for fix 123. so if we click on main we can see here that we have hello we've skipped the fix branch and we've skipped the pr branch due to these conditionals so now everything has been merged up into our main branch if i wanted to force everything that is struck through and what the strikethrough means is these will be garbage collected we'll say what this means is that these branches and the prs will be deleted the next time a scan occurs now we're not going to wait for that scan to happen we're just going to click on scan repository now and it's done so if i click on multi-branch sample app here again you'll see that we only have main and we have no pull request left in the queue so that is it when using a multi-branch job when you're connecting to github you will want to use a github app to take care of your authentication if you can get a tunnel between github whether it's github.com or maybe you're running github enterprise and it's in a different segment of your network if you can receive those web hooks things will magically happen but it's not magic those hooks will be received the different branches will have their jobs appear and disappear and you'll be able to manage all of this through basically one credential a github app and one job if you have any questions or comments you can reach out to us on twitter at cloudbeesdevs or you can reach out to me personally at darren pope if this video was helpful to you please consider giving us a thumbs up and if you haven't subscribed to cloudbees tv yet why not take a moment click on that subscribe button and then ring that bell and you'll be notified anytime there is new content available on cloudbees tv thanks for watching and we will see you in the next video you

Channel: CloudBeesTV

Views: 2,086

Rating: 5 out of 5

Keywords: darin pope, jenkins, jenkins webhooks, github webhooks, github app, jenkins multibranch, jenkins tutorial for beginners, jenkins pipeline, jenkins tutorial

Id: aDmeeVDrp0o

Length: 26min 44sec (1604 seconds)

Published: Tue Mar 02 2021