How to Create a GitHub Organization in Jenkins

in today's video we're going to go through how to create a github organization job in jenkins [Music] are you new here if you are welcome and if you are new here you don't know who i am my name is darren pope and i'm a developer advocate for cloudbees you may have seen previous videos where i walked you through how to create a multi-branch pipeline with github we're going to take it one step further today and we're going to take a look at the github organization job type if you're unaware of what this job type is this is much like a multi-branch pipeline job except it's much much bigger much like a multi-branch pipeline is a folder of pipeline jobs a github organization job type is a folder of multi-branch jobs if you step back and think about it if i create a github organization job and i point it at the organization within github whether that's github.com or github enterprise and if everybody follows certain rules convention over configuration then your job as a jenkins administrator is greatly simplified so let's recap where we are starting from today i have a jenkins controller that is running 2.263.4 i have an agent connected to that controller with a label of linux and i also have an ngrok tunnel running so i can receive webhooks from github.com also today unlike previous videos i have two repositories that we're working with today they both start with the name micro i have a micro 1 repository and a micro 2 repository at this point the only thing that both repositories have is a readme and that's all that we're going to need for this example today if you've watched the previous videos about multi-branch pipeline with github you might want to skip over these next couple of sections so you can follow the time codes down below in the description if you haven't watched it you're going to need to watch this section even if you have watched it in the past with multi-branch you might want to watch this maybe at two speed just to refresh what is going on also there are knowledge base articles down below in the description in case you have any questions about what is going on in today's video the first thing that we need to do is create a github app so to do that we'll go to our github profile page click on developer settings we'll land on github apps let's click on create a new or it doesn't say create but new github app let's give it a name the name for our app is going to be named jenkins github darrenpope paste that in right there we have to give it a home page url i'm just going to give it my organization homepage url and then under webhook because i am going to use web hooks in order to trigger off a lot of the changes that are occurring within my controller my web hook url is going to be my ngrok url which i'm going to grab here and i have to also include github webhook so let's make sure all that looks good except for the double slash now that looks good okay so up to this point we give it a name a homepage url and a web hook that ends in github dash web hook we'll leave ssl verification turned on because i have a good certificate through ngrok now let's go ahead and set our repository permissions administration we need read only checks we need read and write contents read only metadata read only which is already selected for us pull requests i won't read only and commit statuses i want read and write now as i said before i am going to go ahead and allow github to send me webhooks number one but i'm also going to set up my controller to manage webhooks for me so i'm going to say webhooks read and write okay let's go ahead and scroll down some more under organization permissions we do not need to make any changes there they are all no access for user permissions again no changes all no access based on the permissions for the repositories that you selected there are a set of events that showed up down here under subscribe to events so for the events we want to subscribe to check run check suite pull request push and repository let's go through that one more time check run check suite pull request push and repository we're also going to set it up to where this app can only be installed on this account so it's only going to be associated with the darren pope account let's go ahead and click on create github app and if parts of that weren't clear to you go review the documentation below and it takes you through more step-by-step how to do these settings now we've landed on our roughly the detail page for jenkins github darren pope go ahead and scroll down and we're going to the section that is called generate a private key so i'm going to click on generate a private key and i am going to save this file and that has now been saved locally on my machine we're going to need this private key to use as part of our credential within our controller however we cannot use the key as it is and this is where you're going to need to refer to the documentation or take really close notes as we're going through this what we're going to do is go over to my downloads directory and what we're going to do is run the following command so the command is this open ssl command with a pkcs8 and a few other switches along the way my file name is going to be jenkins whoops i probably did not have so that is the key file that we just downloaded i'm going to change this well not necessarily change it but i'm going to set up a an out file called converted githubapp.pim and we're also including a no switch on the end so openssl some commands some switches we have an n file which was the private key file we downloaded or was downloaded to us we're turning it into a different format so we're giving it a different name and then no crypt let's hit enter so now we have a converted file that was created at 1409 which is what time it is right now before we leave our github app we need to make one more change so what we're going to do is we need to install the application within our organization so we're still on our details page we need to click on install app or you can also well we can see here we must generate a private key so we just did that let's refresh this page just do a straight up refresh see what happens yep so it's all good now but now we need to install the application so let's click on install application remember that we said this application is only good for the darren pope organization so i'm going to click on install and it gives you these options i can either install this on my account so it applies to all repositories meaning everything that currently exists in anything in the future or i only want to select certain repositories and this is the decision that you need to make in my case i have over 200 repositories that are within my darin pope organization i do not want this github app to have control where you have access to any of those at least from the permissions that we've set up so i'm only going to select certain repositories and the two repositories that i want to select are micro 1 and micro 2. so there's micro one and i can say micro two whoops if i can spell micro two in your team in your company in your organization you may have multiple github organizations that have that are scoped to certain teams you may have a single organization that has thousands of repositories so you want to be very careful in how you're setting up this github app and that's also going to carry over into how we create our job so for right now the only thing that we're going to do is we're going to allow this github app to have the permissions to only these two repositories and you can see the effective permissions against these repositories so let's click on install and now we can see that everything is set up the way we want now that we've gone down this path in case we ever wanted to add more repositories remember i selected repositories to go along with this app we will go back to install app click on the gear and within the gear that's where we would add our extra repositories or if at some point you would clean up your github organization to only have the repositories in it that you need then you could go ahead and switch back over to all repositories now that we've finished configuring our github app let's go over to our controller and set up the credentials so let's go to our controller manage jenkins manage credentials click on global let's add a credential we're going to change the kind to github app for the id i am going to name it the same as my github app you can name it whatever you want i'm naming it so i can look inside my credential here and map it back against what is in github i'm going to do the same thing for the description for app id we need to come back over click on developer settings click on the app and the app id right here at the top of the page so in my case it's one zero four two nine and for the key that is going to be our converted key that we created using openssl so let me cat converted and i will copy this private key tab out of that and then let's click on test connection and if everything works correctly we should receive this message success remaining rate limit it may say 54999 somewhere in that ballpark is what is the number that you should be seeing that's at least for github.com you may see different numbers if you're using github enterprise let's click on ok so now we see our credential set up now that we have our credentials set up let's go ahead and create the github organization job we'll click on dashboard we will click on new item i'm going to enter an item name just called darren pope i'm naming it the same as my organization within github again you can name it whatever you want you'll see in a few moments that the name of this job doesn't really matter we're going to click on github organization and click ok so what we can see here is we have an api endpoint of api.github.com because we are integrating with a organization on github.com this is the correct api endpoint if you're using github enterprise you will have had to have set up a github enterprise entry over in managed jenkins configure system so that it would show up here as an option let's click on credentials and select our github app credentials we can see that the github app verified with a remaining rate limit of 4999 the owner is the owner or the id of the organization in my case my organization id is darren pope so i'm going to leave that as is typically i would just start with the behaviors that exist and come for a default which are discover branches and discover pull requests one for origin and one for forks but you may remember that our github app only applies to two repositories micro one and micro two so in my case today i also want to add in a filter for the repositories which you'll see here at the top is a repositories section within this drop-down there's a repositories there's a within repository there's general we're focusing right now on the filter by name under repositories now note there is also a filter by name under within repository that's not what we're doing right now we're doing filter by name with wild cards under repositories and in my case i only want the two micro reposito or the two repositories that start with micro said differently any repository that starts with micro will be included in this job let's go ahead and click on save and what's happening right now is it is making an api call to github and here are all the different repositories but note that it's ignoring all of them except for micro one and micro two the reasons why all of those were being ignored is because of that filter by name that we just added for filtering out the repositories so the only two we care about right now are micro one and micro two now let's take a look at this real quick before we take a different look at what happened we can see that it found a remote branch of main but it did not find a jenkins file on that repository on the main branch same thing for micro2 it found the main branch but no jenkins file found and that's okay because if you think back to the beginning we only had a readme in both micro1 and in micro2 before we leave this page i want you to notice one more thing look at the breadcrumb notice that the breadcrumb now has the organization name i'm going to go ahead and click on that so we can see from the view of the github organization job it pulled in the logo the icon from github it gave me the full description name or the full human name and then it also shows the folder name remember what i talked about in the introduction much like a multi-branch job is a folder of pipeline jobs a github organization job is a folder of multi-branch jobs so let's get these repositories showing up in our job in order for these repositories to start showing up within this job we need to add a jenkins file to those repositories let's take a look at how this works so there's our basic configuration and here we have a project recognizer and what this is telling us is anytime i find a jenkins file this repository will be picked up and in fact as you can see here underneath the ad the only option we have is pipeline jenkins file we will revisit this in a few moments but for right now we're just going to have just the one and just because i'm going to go ahead and click on save now let's flip over to our shell and let's go back over to our micro one repository we are on our main branch and let me grab a jenkins file that we are going to use so i'm going to say vi jenkins file and i'm going to paste in a very simple jenkins file i have a pipeline agent label linux again we have an agent connected to our controller that is has a label of linux and then we're just saying hello from jenkinsfile this will become more clear in a few moments let's go ahead and click on save and then what we're going to do is take that file and get it pushed up to github so we'll get add we'll get commit so add jenkinsfile and then we are going to get push okay now that we've done that let's go back over here i'm just going to click on my name and now we can see that micro1 has been found if we click into micro1 you're going to see that we have the main branch because that's where the jenkins file exists we only have a single branch and the jenkins file exists on the branch so let's think about this again for a second darin pope is a folder that's a special type of folder but it's a folder nonetheless of multi-branch jobs note the icon here when i click into the multi-branch job then we also now have a folder of job pipeline jobs that are associated to branches if we take a look at main when we take a look at the output we can see that this was triggered based on a branch index and then we have our hello from jenkinsfile which was the contents of our jenkins file okay now what we want to do is we want to create a new branch in micro1 so let's do a get checkout or actually let's do this let's do a get pull yep we're on main so let's get checkout dash b we'll call it fixed-123 and now what we're going to do is we're going to push that up to github again no changes to the branch we're just creating a new branch let's see what happens now if we refresh micro 1 we can now see that fix 123 is there and it's being built due to the push event to branch fix 123. and we have hello from jenkinsfile again all good exactly what we expect to have now i want to make a change to the jenkins file that is within the fixed-123 branch so we are on fix 123. let's edit our jenkins file and we are going to add two new stages and let me grab those here real quick so what we have here is the jenkins file this stage the hello stage will apply to all branches the second stage will only apply to branches that start with fixed dash which in our case we have a fixed dash 123 and the final stage is for pr so what we're going to be doing is we're going to be making a pull request from fix 123 into main in a few moments and we can do something different just for pr builds let's go ahead and save this and now we want to add it and i'm going to commit my change okay and we are going to push this up what we're going to see here if i click on micro one momentarily we should see fix 123 has now gone into a pending mode and as it starts to run here there we go there was a push event here's our hello from jenkinsfile we cated out our readme which was our fix dash step and we can see that the stage for pr was skipped because fixed dash is not the same as pr dash so so far so good we have our main branch our fixed branch and we have no pull request as of yet let's go back over to github and let's create a pull request so we're in micro1 we can compare and create a pull request we can see here that we're merging from fix 123 into main and the updated jenkins file are the two changes that we just made or the two stages that we just added let's go ahead and create a pull request but we're not going to merge it just yet let's go back over to our controller if i click on micro 1 we can see now that we still have our main and fix 123 branches but we now have a pull request that exists and that's pull request 4. if we take a look at pull request 4 we can see that that pr4 was opened the hello output we're skipping the fixed dash and then we have the stage this only runs for the prs so right now we have a main branch a fix 123 branch and a pull request so now let's go ahead and merge this pull request we'll go back over to github we will scroll down and click on merge pull request and confirm merge once the merge completed the way i have my repository set up within github is to automatically delete the branches once they are merged so fix 123 has now been deleted let's go back over to our controller click on micro one and what we're going to see is that fix 123 is now obsolete that's what the strikethrough represents and we can also see that the pull requests pr4 is also obsolete if we click on main we can see it's still running and should be finishing momentarily we have our hello from jenkins file and then we have stage for fix was skipped and stage for pr was skipped so all of our changes that we had in fix 123 through the pull request are now living inside of the main branch if we click back on micro we could wait for a while to have these branches cleaned up but let's go ahead and click on scan repository now and that will clean up the branches for this repository so we can see all we have left is main on branches and the pr4 is now gone okay we've spent some time in micro one let's go over to micro two and make some changes there so you can see how this works now remember when we set up our github organization job type we said we only want to bring in repositories that start with micro in our case we have a micro one and we have a micro two micro two currently does not have a jenkins file in its file system so let's make a change so we can have a jenkins file there however we're going to name it a little different let's go over to micro2 so let's pull up this let's go over to micro2 and we are going to create a file named jenkinsfile dash m2 you're thinking darren wait what are you doing it'll become apparent in a few moments let's grab a jenkins file that is m2 in fact the contents of it say as much it says hello from jenkinsfile dash m2 if you think back to the other jenkins file that we started with earlier today it was hello from jenkinsfile so just go with me here a little bit let's go and save this and now let's do a get add jenkinsfile m2 get commit dash m added jenkins file and two and then i am going to push that up now we've added a jenkins file although not named jenkin's file but it is it is a legitimate jenkins file it just doesn't have the name of jenkinsfile if we click on darin pope we still do not see micro2 here but think back a moment to project recognizers let's click on configure if we scroll down to our project recognizers let's add a new project recognizer it's still pipeline jenkins file but in this case we're going to say jenkinsfile dash m2 let's click on save as it scans what we will see and i clicked away from the scan log just so it's not there we can now see that micro2 showed up and we can see that the main branch is building and if we take a look at the output we can see that it says hello from jenkins file m2 because we set up a project recognizer for jenkins file dash m2 when it went through to find when it was doing the scan and was trying to find files project recognizers it did not find a jenkins file but it did find a jenkins file dash m2 and the job was able to create the micro2 multi-branch job underneath the github organization but let's say this wasn't communicated very well within your organization and somebody just created a dash m2 but at a certain point it's like oh you know what i really want to use a jenkins file but i'm going to leave the m2 file in there as well this could cause some confusion and we're going to walk through one of the ways you could solve that now what we're going to do is we are going to make a copy of our m2 jenkins file and clean it up to look like the other jenkins file so i'm just going to do a copy of jenkinsfilem2 to jenkinsfile oops and i am going to edit jenkinsfile and i am going to get rid of the dash m2 so if we look here in the repository we have a jenkins file and a jenkins file m2 so now that we've created this jenkins file what do you think is going to happen is the jenkins file going to run and render or is it going to be the jenkins file m2 that's going to run in render let's see so let's take a look at status okay so let's add jenkins file because nothing will happen if we don't add it we'll commit it and we're going to push this to main so now we have pushed to main for micro2 if i click on main again we can see that 2 is pending wait for it to warm up and 2 is done if we scroll down it still says hello from jenkinsfile m2 and here's why we have not had another scan of the organization done yet to pick up that the jenkins file now exists inside of the micro2 repository so let's go back to the org level i'm going to say scan organization now give it just a second or two it doesn't take that long at least with my setup i'm going to click on micro2 and now i'm going to click on main before we actually run this job or do a build now let's go back and take a look at 2 real quick we can see here that the jenkins file that was picked up and wren was the hello from jenkinsfile m2 let's go back over to micro2 excuse me let's go back to main in micro2 and click on build now when we take a look at three what you're going to see is now we have a hello from jenkinsfile so when we rescanned the organization it found the micro2 repository it found that it had a jenkins file in it so therefore the jenkins file is what was going to be run instead of the jenkins file dash m2 so let's go back and look at this one more time go back to the organization click on configure and what we have we have two project recognizers in our case we have a jenkins file and a jenkins file dash m2 whichever file first matches is going to be the file that will be used from that point forward after that scan to run the job just to prove that what i'm going to do is i am going to reorder the files so i'm going to use a jenkins file m2 first so if jenkinsfilem2 is found it's going to be used if that isn't found then it's going to look for a jenkins file if no jenkins file is found then the repository will not be found so i have m2 first i have just jenkins files second let's click on save because of that change it will do another scan let's take a look back at micro2 let's click on main since that was not an initial scan we need to do a build now and with four what we're going to see is we are now back to hello from jenkinsfile m2 as a recap remember you can have a pipeline job you can have a multi-branch job which is effectively a folder of pipeline jobs in today's video we learned that a github organization job is a folder of multi-branch jobs so now in a perfect world if your github organizations were nice and aligned and followed either lines of business or product or however you want to align your repositories you could create a single job on your controller point it at that organization and making sure that you either name things correctly or just grab all of the repositories your choice from that point forward any changes that are made within the repositories with the correctly configured web hook you will never have to in theory go back into your controller to make changes to jobs from that point forward you could manage all of your configurations primarily within your get repositories if you have any questions or comments you can reach out to us on twitter at cloudbeesdevs and before you click away today could you go ahead and give us a thumbs up on this video and if you haven't subscribed to cloudbees tv yet why not take a moment click on the subscribe button and then ring that bell and you'll be notified anytime there is new content available on cloudbees tv thanks for watching and we will see you in the next video

Channel: CloudBeesTV

Views: 2,541

Keywords: darin pope, jenkins, jenkins webhooks, github webhooks, github app, jenkins github organization tutorial, jenkins github pull requests, github branch source plugin jenkins

Id: LbXKUKQ24T8

Length: 39min 51sec (2391 seconds)

Published: Tue Mar 16 2021