Register Docker Runner/Executor with GitLab Server to Run Pipelines

hey everyone continuing the series of setting up your own gitlab server inside a Docker container in this video I'll show you how to set up gitlab Runner as a container alongside the gitlive server using Docker compose if you guys have watched my previous videos on how to set up the gitlab server in a Docker container we were at the position that we have actually added the gitlab repository I have added this gitlab demo repository here and then I've added a gitlab CI file but if I go to my pipelines and if I try to run the pipeline the pipeline is submitting into the queue but if we go to the build job we can see that the build job is stuck and the reason is we don't have a gitlab runner and even if we try the CI settings we don't see any gitlab Runner dedicated to the project or any shared Runner and when we try to visit the admin area to look for the shade Runners we don't see any runners here as well so now if I click on this new instance Runner and I select the default option like the Linux one I am adding a test eggs and if I run the create Runner this guy showed me some command that we need to run this command to run or get live Runner and when we tried this command inside our gitlab server it was not working and it says that the gitlab runner command is not found so this means that we need some other external service which we call the gitlab runner and inside that gitlab Runner we can run this command to register or get lab Runner so if we look at our Docker compose file we have the gitlive server we have some configurations here and the gitlab server is running on Port 8088 so the piece missing here is we need to add another service called gitlab Runner images separate then the gitlab server the gitlab server images Docker gitlab CE and the runner images gitlab Runner to minimize the download size and to speed up the download we are using the Alpine version here but if you guys are okay with the standard version is you guys can just uh skip the Alpine here I'm giving the name gitlab Runner to The Container so we will be using the docker executor to run our pipelines and your up so if we go to our CI file we can see that we are using a Docker image called python Alpine and then once that image is loaded and the script is run we will be just checking which python version we are using for this Docker image so to run your pipelines and your jobs inside a Docker container you need the docker service or you need the docker engine to be running inside your gitlab Runner so to use the getlive docker engine inside the gitlab runner you need to Mount Your Docker socket file from your host machine to your container which is the gitlab runner when you install a Docker in a machine there are actually two different programs comes in one is the docker client and the other is Docker server so the docker server actually receives commands or socket either on a network or through a file and the client actually communicates over a network and send messages to the docker server to make a container start a container or stop a container when the client and the server are running on the same computer they can connect through a special file called socket when the docker client and the docker server are running on the same computer they can connect through a special file called socket and since they can communicate through a file and Docker can efficiently share files between host and containers it means you can run client inside Docker itself the docker demon listen for the docker engine apis request we have three different types of socket when it's called the Unix now that is called the TCP and then the other is Ft by default I Unix demand socket is created at this path called War so if you guys have installed the docker engine on your on your host machine you guys can inspect this file this is the socket file for Docker through which the docker client and the docker server is communicating so this means that to run and communicate through this socket file from within the gitlab runner to create container and stop container when we want to execute our jobs we need to mount or we need to sync this file from our host machine to the docker container so for that I will add another another attribute called volumes and then inside the volumes I'll try to mount this file from my host to The Container so this is the docker socket file from my host machine and this will be synced to this file inside the docker container once you add the gitlab service to the docker compose file and mount the required docket socket file you need to stop your existing gitlab setup and since we have updated the docker compose file for some new changes we need to do Docker compose up with hyphen hyphen build to actually start building the gitlab runner as well so after you build the docker compose including the gitlab runner and you reload the gitlab server you can go back to this Runner section click on the new instance Runner create the runner and now this is the command which we need to run inside the gitlab runner container so I'll go to my terminal I'll do a Docker Docker ps-a to list down all the running containers this is the gitlab runner container I'll copy this container ID I'll try to log into this container and now I'll go and copy this command and I'll try to run this command inside the gitlab runner container so if you guys look at this diagram this is our host machine then we have two containers one at the gitlab server container the other one is the gitlab runner when we are running this command here gitlab Runner register we are actually running this gitlab Runner register command inside this container which is the gitlive runner container and once we successfully run this gitlab Runner and register command this will actually create this job executor it will not create the container after we successfully run this gitlab Runner register but when we actually run our Pipeline and the docker executor at that time the gitlab runner will actually create another nested container here which will be actually the executor of that job so to access the getlive server like this one from inside this nested executor container we need to put this container on this host machine networking because uh from within the container you can can access another container by using the container name but if you guys notice here this container is actually on a nested level it's running inside the gitlab runner so we need to move up one level to be on the same level as we have the gitlab runner and the gitlab server container so once we put the networking of this executor container on our host machine this means that we can access nuggetlab server through localhost from within the job executor container so what I'll add here to this container is that I'll add Docker Network Mode post this means that add the networking of this Docker container on the host machine networking now this is asking for the default okay type instance URL which is the localhost so now there are two scenarios one is when we are running a job inside a Docker container which is the job executor and this job executor is trying to pull the repository from the gitlab server then we need to put the docker networking of this container this executed container on our host machine and the other one is when when we are registering a runner with the gitlab server for that you need to put the getlab runner container this is also a container so you need to put the networking of this container on the host machine as well so if you guys see here this is a gitlab server this is the gitlab runner and we need to put the networking of this container on this wash machine networking because here it needs the gitlab instance URL once we complete this command this will actually the gitlab runner will actually send all these details to the getlive server to register this Runner and once that is registered and we want to run our Pipelines this this networking will be actually used so now we have two options here's one is to since we are communicating between the container to container from the gitlab runner to the getlive server one way is to use the container name which is bitlab server and in the port name and the other ways if you want to use the localhost you need to put the networking up your gitlab or Runner container on the host machine as well for that you need to update your Docker compose you can just add here the network mode to host and if you add this network mode host then this means uh you can use the localhost 8080 here because since both the containers will be on the same network they can access each other through localhost but if you don't want to go into this details then you need to put the gitlab the gitlab container name here alongside the port so the gitlab runner can communicate to the gitlab server when it's trying to register the getlib runner now I'll give it just a dummy name to this Runner to choose an Executor I'll try to run my pipeline jobs inside the docker so I'll put the docker here you can select any from these options and in case you want to provide a default image for this here I have explicitly mentioned that I'm using the python image but if you want to set a default image for this if I don't set an image here you can use the default one so you can set them default image here so for this I'll use the python Alpine version and my gitlive Runner is registered now what I'll do that I'll go to my Runner sections from the admin area my git library is registered here and if I go to the details I can see all the details I have provided so the executor is Docker this is the runner where we have registered our executor I'll go back to my repository I'll go to the pipeline section you guys can see these pipelines have already ran the moment I registered the gitlab runner and the executor here these jobs were in pending and were already in queue waiting for some executor so they were already run but I'll just show you guys again by running a new Pipeline and if I go to the build stage and yeah this pipeline is running with the getlab runner and it's using the docker executor image and the default images python Alpine uh this has pulled first the docker image for the python and here it's actually cloned the repository here and then it has dumped the python version which we have mentioned inside the pipeline which we have mentioned inside the CI script python data version so that was it for this video now if you guys have seen like this one is obvious to add the get live Runner service here and then Mount the docker socket file into the Container but if you guys look at these tips to First you need to click on this button to ins to create a new instance Runner and then once you create the runner this will give you a command and after this you need to log into the gitlab runner container and then you need to paste this command here and give all the required options we can also automate this process completely so the moment we add our gitlab server here and we do a Docker compose up from the command line we can pass all these options and all these command lines here inside the gitlab runner command so the moment we actually start up our containers or gitlab Runner will be registered automatically here so instead of going and creating generating those commands and running those commands in the in Syndicate lab Runner manually we can also automate this process through the local compose file so in the next video I'll show you how to automatically register the job executor inside the get lab Runner using docker compose so the moment we start or get live server and the runner we will have automatically a runner added here and the moment we create our pipelines the jobs inside that pipelines will run automatically stay tuned for the next video and I'll see you in the next video

Channel: Build With Lal

Views: 5,108

Keywords: gitlab, docker, gitlab-in-docker, gitlab-server, custom-gitlab, gitlab-local, gitlab-in-container, dockerize-gitlab, gitlab-in-docker-compose, gitlab-ci, gitlab-pipelines, gitlab-runner

Id: Rvh7OZbDJ_o

Length: 11min 0sec (660 seconds)

Published: Fri Sep 08 2023