GitLab CI/CD | GitLab Runner Introduction | 2022

hey what's up everybody my name is moss and in this video i'm going to introduce you to gitlab runners i'll walk you through how to install register and use a self-hosted runner in a gitlab pipeline we'll begin with a short presentation to get you familiar with what gitlab runner is and how it works and then we'll start the tutorial if you haven't already go ahead and grab a coffee and let's get started okay let's begin by getting familiar with what gitlab runner is and how it works gitlab runner is an open source application that is used to run jobs in a gitlab ci cd pipeline and it's very similar to the concept of a jenkins agent if you've ever worked with jenkins before when a gitlab pipeline is triggered the jobs defined in the gitlab pipeline are assigned to available gitlab runners the gitlab runner program can be installed on your local machine a virtual machine a docker container or cloud infrastructure and it's supported on multiple operating systems get lab runners execute the work defined in gitlab pipeline jobs and there are multiple types of gitlab runners first you can utilize a shared runner shared runners are available to all projects in a gitlab instance then there are group specific runners which are available to all projects in a specific gitlab group and then there are project specific runners which is the type of runner that we're going to use in this tutorial these runners are used on a single gitlab project or a set of projects with specific requirements the next key component of the gitlab runner program is the executor when you install and configure a gitlab runner you select which type of executor you want to use with this runner and the executor determines the runtime environment in which a pipeline job will run for instance you could select virtualbox as the executor so that you can execute jobs inside of a virtual machine you can also use the shell of the host machine that get labrunner is installed on as the executor so this is almost equivalent to writing a shell script with the same commands that are included in the pipeline job but instead we're having the gitlab runner execute those commands in the shell of the host machine we can also use remote ssh as the executor and in this case the gitlab runner program will connect to a remote machine and send the job instructions to that remote machine and the remote machine will execute those pipeline job instructions this executor is not commonly used and in gitlab's documentation they actually encourage encourage you to use other executors instead of the remote ssh executor docker is another executor option that we can use the docker executor option is commonly used and in this workflow the gitlab runner is installed on the host machine of the docker container and when a pipeline job is assigned to the gitlab runner the gitlab runner will pull the docker image that's specified in the gitlab pipeline it will start the container and then execute the job instructions inside of the docker container similar to the docker executor we can also use the kubernetes executor and in this case the job will be executed on a kubernetes cluster and the last option is to write your own custom executor for environments that are not natively supported by gitlab runner something worth mentioning is the ability to monitor gitlab runners the runner program has an embedded prometheus metrics http server for monitoring and you can monitor metrics like the get lab runner business logic go specific process metrics general process metrics and build version information tags are a very useful feature of get lab runners when you install and register a gitlab runner program you can add tags to the runner program that can be referenced in a gitlab pipeline when you reference a tag inside of a gitlab pipeline yaml file you're filtering for only runners with that specific tag so when the gitlab pipeline runs only runners matching the tags that you specified in the pipeline will be able to execute the jobs in the pipeline and so this is really useful if the pipeline requires specific dependencies in the runtime environment of the jobs so for instance if you're trying to build a python application or running a python script you would need the runtime environment to have python installed as a dependency so you might have a python tag for gitlab runners that are installed in an environment that includes python as a dependency so why do we want to host and manage our own gitlab runners well first it could potentially save us money by hosting gitlab runners on our own infrastructure as opposed to utilizing get lab runners hosted and managed by gitlab.com the second reason i can think of is customization you might have specific requirements for your product or your organization has specific requirements that might push you towards self-hosting your gitlab runners and the last reason is security you might be required by your organization's it to host the git lab runners inside of the corporate environment for security reasons so the next question is how do we self-host gitlab runner and the first step is to install gitlab runner on a target host and remember that this could be on a virtual machine it could be on your local machine it could even be inside of a docker container so you can install the gitlab runner program itself inside of a docker container the next step would be to configure the gitlab runner program on the target host which would include choosing an executor and also registering the gitlab runner with the gitlab instance that it's going to be running jobs for and then finally we would run a gitlab pipeline that utilizes our runner maybe via a tag that was assigned to the runner at configuration time now if you intend on following along with the tutorial there are some prerequisites first you'll need a gitlab account you'll also need a virtual machine or some other infrastructure like your local machine in order to install the gitlab runner program in this video i'll be using a linux machine as the gitlab runner's host a gitlab project and finally you'll need docker installed on the target machine that the gitlab runner program is also installed on and this is because i'll be using the docker executor and one other thing i'd like to mention is that the gitlab project that i'm going to be using is linked in the video description below so now that we have some additional context on the gitlab runner program let's get started with the tutorial okay so the first thing that i want to do is navigate to the gitlab project that i want to create a gitlab runner for and remember that during the presentation i mentioned that we are going to be creating a project specific gitlab runner and to do that i would navigate to the project which i'm currently in the project and then on the left hand side menu i'll scroll down here to settings and then select ci cd and then i'm going to select expand in the runners section and if i scroll down here we have two sections we have the specific runners and the shared runners section and if you remember from the presentation runners are shared across an entire gitlab instance and in this case we're using gitlab.com so all of the runners that you see in this list here can be used by anyone on gitlab.com and in contrast we have specific runners and as you can see here these are git lab runners that are specific to this particular project and it also lists the steps that we need to take in order to install the gitlab runner program and register the runner with gitlab.com and this project so let's go ahead and open up this documentation for the installation process and there's multiple ways that we can install the git lab runner program on a linux machine in this video i'm going to utilize the gitlab runner binaries to install it so i'm going to select the binary section and then i'll select install on linux and we can install the runner program through a debian package but i'm going to use the binary file i'm going to download the binary file so i'm going to select this link here and the first step is to download the binary file here it says download one of the binaries for your system i'm going to select the top url here since uh i'm on a 64-bit system but be sure to check your architecture before selecting one of these urls and confirm it's it's the right um the right binary that you're about to download so i'm going to copy this to my clipboard and once i have it copied to my clipboard i'm going to open up a terminal session and i'm going to paste that command into my terminal window so this is going to download the gitlab binary into user local bin and now that we've downloaded it the next step is to make it an executable file so i'm going to copy this command here and navigate back to my terminal paste that in and once we've made it an executable file the next step is to create a gitlab user so the program is going to be ran as root but the jobs themselves will be ran as this user that we create here so the user that we're going to create is going to be called gitlab hyphen runner so i'm going to copy this command to my clipboard and i'll paste that in and after the gitlab runner user has been created we'll then install and run the gitlab runner program as a service so that's going to be these two commands here so in the first command we invoke the install command and we pass in several options the first one is the user option and we set that to be the newly created gitlab runner user and then we also set the working directory as home gitlab runner and then immediately after installing the gitlab runner program as a service we start the service so i'm going to copy these commands to my clipboard and navigate back to my terminal and we'll paste them in and it looks like both commands ran successfully if i want to check the status of the service i can do sudo gitlab runner status and then we can see that the service is currently running and let's also take a look at the service file in the system folder so if i do sudo cat okay we have this gitlab runner service file and if we take a look at the service block you can see the execute start command it invokes the git lab runner binary with the run command and it specifies the working directory the config file which is a file that you can manually change if you if you want to and then we specify the service name and the user that will be used to run the gitlab runner jobs so of course we could manage the service just like other services using the system control command we could start and stop the service and check the status of it but as you can see we can also just use the git lab runner binary to to manage the service now after we've started the gitlab runner service the next step would be to register the gitlab runner with our gitlab project and to register the gitlab runner what we're going to do is type in sudo gitlabrunner and then register and it prompts us to enter the gitlab instance url and in our case we are using gitlab.com so i'm going to type in gitlab.com and then we have to input the registration token and to get our registration token we have to navigate back to the gitlab project so if i go back to the browser and then to ci cd settings here we have the registration token so i'm going to copy this to my clipboard and then navigate back to my terminal i'm going to paste in that token and hit enter and i'm going to call this runner testrunner and now it prompts us to enter any tags that we want associated with this runner now the pipeline i haven't showed the pipeline in this project but the pipeline includes an invocation to python so i'm going to add the python tag to this gitlab runner because it's going to run a docker image that includes a python installation so i'm going to say python and hit enter i'm going to leave the maintenance note blank and the registration is exceeded but now we have to select what kind of executor we want to use with this gitlab runner and you can see the possible options that we can select here but as i mentioned before we're going to be using docker so i'm going to type in docker and hit enter and after selecting docker it prompts me to enter a default docker image that should be used when running a pipeline job and i'm going to enter the example they show here ruby 2.7 it doesn't matter because we're going to override the docker image in the pipeline script so i'll hit enter and as you can see from the message it says that the runner was registered successfully so we can verify that at the command line and we can also verify it in the ui so at the command line i'm going to say sudo get lab runner verify and in the message it says verifying runner is alive and then it gives the runner id and if i navigate back to the cicd settings page in lab i'll refresh the page and i'll navigate back to the runners section and as you can see under the specific runners section we now have a runner listed here and its status is green we can also see the description of the runner that we entered at the command line as well as any of the tags that we associated uh with this particular gitlab runner and in this case the only one that we added was a python tag and if we wanted to update any of the details of the runner we could select edit here and then adjust the configuration of the runner we can see things like its ip address we can update the description we can set a maximum job timeout and this would be used if a job lasted past a certain period let's say it lasted over an hour we can set a timeout period where the job will automatically be you know killed after that timeout period and we can also adjust the tags as well and now that we have an available runner i'm actually going to turn off shared runners for this project so the only runner that's going to be able to be used in a pipeline is going to be this runner so i'm going to turn shared runners off and i'm going to navigate to my gitlab pipeline so under the ci cd section i'm going to select editor and let's take a look at the pipeline that i've written so if you're not familiar with gitlab pipelines already i definitely suggest taking a look at my other video introducing you to get lab pipelines we actually use this specific pipeline in that video it's slightly modified in this video but it's generally the same i'm going to quickly walk through the pipeline and then i'm going to run the pipeline to verify that our gitlab runner is being used to execute the pipeline jobs so at line one we define the docker image that should be used by the gitlab runner when executing this pipeline and the image that i've selected is a docker image that includes python and directly after the image keyword we have the default block the gitlab default keyword allows me to set global defaults for other keywords which means that if a job doesn't explicitly specify a particular value for a keyword such as tags then the value that is defined under the default block will be used for that job and as you can see in this particular default block i've defined a tags keyword and we've set the python tag which is the tag associated with our self-hosted gitlab runner now if i wanted to use different gitlab runners per job i could also do that and set the tag value at the job level rather than under the the default keyword so instead of under the default keyword we would include the tags keyword and associated tag values under a particular job declaration so if this pipeline runs correctly it should utilize our runner for all of the jobs that are defined in this pipeline and if we take a look at the other parts of the pipeline we have three stages to find the build test and deploy stage and then we have the build job and the build job is doing some kind of arbitrary steps just echoing to the console making a build directory creating an artifact and then we have two test jobs again these two jobs are just performing arbitrary steps in the script section and then for the last job the deploy job we're actually invoking python 3 and checking the version so remember that we're running the job inside of a docker container that has python installed and that's one thing that i should have mentioned earlier as kind of a a prerequisite or a requirement if you're following along with this tutorial since we selected docker as the executor you'll need to have docker installed on the machine that the gitlab runner is installed on if you don't have docker installed on your machine but maybe you have python installed on your machine and you'd rather the executor be shell instead of docker as the executor you can actually modify the configuration of the gitlab runner program by just modifying the config.tomo file which is located here and as you can see in the config file the executor is defined here as docker and i think this is also uh the settings that are specific to the docker executor but i think we would want to just simply change this to shell if you wanted to um run the jobs just in your host machines shell so that if you had python installed on your host machine it would just work by running directly on the host machine as opposed to inside of a docker container so i'm going to navigate back to gitlab and then to pipelines and we're going to run a new pipeline to verify that our runner is being used when the pipeline is ran so i'm going to select run pipeline and let's take a look at the build job so the build job uh runs pretty quickly and as you can see at the very top here it says running with gitlab runner uh on testrunner which is the runner that we created and then on line four it confirms that we're using the docker executor with the python image and it pulls the python image down and in this particular job we're not invoking python so it's not necessary to have the python image in this particular case but still uses it and then on the right hand side we have the tags that are referenced by the job which in this case is python so we are referencing the correct gitlab runner which is the runner that's installed on this machine now it might also be worthwhile to take a look at the deploy job as well just to confirm the python invocation worked correctly so if we take a look at the deploy job console output we can see where python was invoked and it does print it to the console so it looks like everything is working correctly now i've demonstrated one possible way of installing and running the gitlab runner program with a particular executor configuration in this case we use the docker executor but if you're interested in a particular configuration for instance maybe running a gitlab runner inside of a docker container as opposed to having it installed on the host machine and maybe not using the docker executor but using the kubernetes executor or something like that please let me know in the comments and i will definitely consider doing another video on a particular configuration i hope you enjoyed this video and found it valuable if so please consider throwing a like on the video and subscribing to the channel for more videos like this thanks for watching and i will see you in the next video

Channel: Tech and Beyond With Moss

Views: 112,913

Keywords: gitlab ci cd pipeline tutorial, gitlab ci cd tutorial, continuous integration, gitlab runner, gitlab ci, gitlab tutorial, gitlab ci cd, continuous deployment, gitlab ci tutorial, gitlab ci cd runner

Id: -CyVpfDQAG0

Length: 23min 27sec (1407 seconds)

Published: Mon Apr 04 2022