How to Restart a Jenkins Agent

how to restart a jenkins agent [Music] what happens when you lose connectivity between a jeans controller and an agent basically what you're going to have to do is go through a process to reconnect the controller to the agent in this video we're going to take a look at a few different ways that you can reestablish that connection here's today's starting point i have a jenkins lts controller version 2.303.3 and attached to this controller i have a linux based agent down in the description of this video is a link to a gist that has all of the commands and documentation that we look at in this video one way that we can reconnect an agent to a controller is pretty simple we just need to manually reconnect it so in our case let's go to agent one and let's disconnect the agent so if we say disconnect the computer agent yes absolutely we can see that we were disconnected by the admin and if we take a look at the nodes we can see that right now this is showing a red x and it's also showing over here in the build executor status offline so to simply bring this one back online we just say launch agent and within a few moments that agent will be reconnected and ready for work again so in that first example we manually disconnected the agent and then reconnected the agent but what would happen in case the agent had been restarted or some other process that had disconnected that agent well if we go over and take a look at our agent if we take a look at the processes what we have here whoops is a process that is a bash process in my case that is starting up the agent process well let's go ahead and kill 4123 so kill dash 9 41 23. you'll also notice that there's also a 4132 so these two processes here should go away when we kill 4123. so there's 4123 let's do our process check again and we can see that now those two processes are gone if we go back over into our controller and take a look at nodes first off since we never left this we see that the connection was terminated if we take a look at nodes we now see that agent one is offline and again to bring this controller back online at this point we would come back over and click on launch agent so we click on launch agent and then we see here that the agent starts back up and is successfully connected and online if we take a look at the nodes everything is looking good on this agent at this point so we've looked at the two different options so far for manually bringing an agent back online even though it's really the same resolution launch agent but what would happen if i needed to take an agent offline temporarily let's say i'm doing maintenance on that agent the example that we just looked at is if the agent got kicked over for whatever reason or i'm not in control of that agent and it was taken offline without my knowledge or control but what if i know that i'm going to take an agent offline well what i can do is if i go to agent 1 i can mark this node temporarily offline so if i mark it offline what's going to happen is it's been disconnected and the only way that that could be brought back on notice we're seeing offline but it's actually yellow here i would no longer launch the agent because actually the agent process is still connected but it's no longer accepting jobs so when i'm ready to bring the agent back online i would say bring this node back online and from that point we can see that our agent is now idle and is ready to accept work but again all three of these examples that we've looked at whether i go in and disconnect the agent and then launch the agent again or if i manually kill the process i still have to go back into the controller and click on launch agent or if i've marked an agent temporarily offline i have to go back into the console and click on bring it back online what if i want to automate all of that well there is a link in the gist to a document on the jenkins.io site that has a script that we can run and i've taken the liberty of pulling that script in and the way that we would do this at least for the moment is to go into the script console let's say i had many agents connected and all of these agents were taken offline for weekend maintenance or whatever the case may be and i just want to be able to run this script and bring them all back online that way i don't have to go in and click on launch agent launch agent lodge agent so the way that we would do that is manage jenkins and we would go down here to script console and let me go grab my version of this script so the link to the script that's in the gist also allows you to send emails in my case right now i don't have email set up so i pulled out all of the email examples if you want to use email that's great if not you can use the example that i have also fully down in the gist so if i paste this in what we're going to see here is once we get through a little bit here it's going to iterate over the agents that are connected to our controller and it's going to check it to see if it's offline or online or if it's marked offline temporarily because if it's marked as offline temporarily i don't want to bring that one back online because it was taken offline on purpose so i would not want to from an automation standpoint bring that one that was marked temporarily offline now you may make the change here's the script you can do with it what you want but in our case here's our script let me go back over here to agent 1 which i should have already taken offline let's go here again i'm going to disconnect to this agent and right now this agent is marked offline let's go back over to manage jenkins script console paste in our script and let's click on run and when we click on run what we're going to see here is we can't get a path from agent 1 that's fine is offline is true number of offline nodes was 1 number of nodes is 1 but now we can see here on the left nav that agent 1 has been reconnected so the script did exactly what we wanted but again everything up to this point has required me as a jenkins administrator to log into the jenkins controller and click a button run a script do something along those lines what if we were able to take this script and schedule it to run against our controller we can do that using the geek cli so how do we do that here we go what we're going to need to do is first off get our jenkins cli now in my example that i'm going to show you i'm going to be doing the work on my jenkins controller so having the cron run on my jenkins controller against itself is okay but it might be better to go ahead and have that cron job running somewhere else that you have that you can cli on and it's reaching out and running this command and running this check against the controller remotely either way is probably okay i would prefer to do it remotely but i have this on the machine so it's okay for now so first things that we're going to have to do is we're going to need to create a token to be associated with the user that's going to be running this script so let's go up to admin and configure and let's go and create a token i'm going to say this is going to be cron and i'm going to click on generate and i'm going to copy this token so i'm going to put paste this over here so i don't lose it i'm going to click on save if i was to take a look at that again we can see that the token now exists never been used yet that's all good now what i'm going to do is do something a little contrived but i'm going to do it on the controller so i'm on my jenkins controller i'm going to create a folder structure that is called and i already have this over here for myself and this will be in the gist so you can follow along if you want to so what i'm going to do is i'm going to do a make dir paste it in i'm going to create a cron jobs directory and then within the cron jobs directory i'm going to create a reconnect agents directory just that way i can keep everything in one place and then i'm going to cd into that directory and then finally what i need to do is i need to go and get the jenkins cli jar file and put it in this directory and if you're not familiar with getting the jenkins cli if we go back to dashboard manage jenkins if you scroll down here to jenkins cli there is a link to the jeek and cli jar that is coming from this specific controller and you always want to use the cli that is available from your controller meaning that you want to keep the version of your jenkins cli matching your jeans controller so in my case what i have here is a little script i can run just a wget command to download that jenkins cli into this directory so if i take a look at this directory i now have my genecon cli jar sitting there next up i'm going to create a file called reconnect dot groovy and the contents of this file is going to be the contents that we pasted over in script console so i'm just going to copy and paste that into here and save that now that we've created the groovy file there are actually three more things that we need to do we need to create a shell script to run the groovy file using the jeacon cli we need to create a credentials file that we'll be using within the shell script and then finally we'll be making a cron entry so next up let's go ahead and create our shell script which i'm calling reconnect sh and then inside of that what we're going to have is a java dash jar and specifying our g concli fully qualified to the location dash s jenkins 8080 that's the location of our jenkins controller we're specifying websocket and then off now here i could use admin colon and the token that we generated but it's more secure if we specify a file within the home directory of the user that's going to be running this so we'll be creating this file dot jenkins-cli and it will have the contents for our user name and token then from here we're going to be running the groovy script and we're going to be passing that in from reconnect.groovy so let's go ahead and save this and then let's go ahead and create our in cli so i'm saying in my home directory create this g cli file and inside this i'm going to say admin colon and the token that we created a few moments ago so that's there the other thing that we need to do is set the execute bit on our shell script okay so now if we take a look at reconnect agents we have three files we have reconnect groovy reconnect sh and jenkinscelli.jar if we double check our chickencli file that file now exists in the home directory so the reference to that file should be fine from our shell script now finally we need to create our cron tab so we're going to say cron tab dash e and i'm going to insert my cron tab and what this says is every minute the five stars every minute we're going to run the reconnect sh file and then we're just passing out so the standard error and nothing none of that is getting logged so that's what the devnet21 is here at the end so every minute this reconnect sh is going to run so let's go ahead and save our cron if we take a look at it crontab dash l we can see that it is scheduled and should be ready to go so now let's go back over to our controller click on dashboard click on agent one and let's disconnect this agent we'll click on yes and so now that has been disconnected by the time we got back over to this the cron had already run so therefore the agent has already been reconnected if we take a look at the log for this we can see that it was restarted here just a few seconds ago and finally just to prove out that this is working in the other scenario to where the agent was just taken offline outside of our control let's go back over here to our agent let's look for our bash c which is this one so we're going to say kill dash 9 46 75 so now that has been taken offline if we take a look at our controller we can see it's offline and within the minute this agent will be brought back online and we can now see on the left now that this agent has been reconnected we take a look at nodes everything is back in running order if you have any questions or comments you can reach out to us on twitter at cloudbees if this video was helpful to you give us a thumbs up and if you haven't subscribed to cloudbees tv yet why not take a moment click on that subscribe button and then ring that bell and you'll be notified anytime there's new content available on cloudbees tv thanks for watching and we will see you in the next video you

Channel: CloudBeesTV

Views: 275

Keywords: darin pope, jenkins, jenkins tutorial, how to restart jenkins slave, restart jenkins agent, restart jenkins agent linux, jenkins restart agent, restart jenkins slave, jenkins restart slave, jenkins slave restart, jenkins ssh agent, jenkins ssh slave, jenkins slave ssh

Id: MTLgbp0GH8w

Length: 14min 20sec (860 seconds)

Published: Thu Dec 02 2021