How to test a Docker container in GitLab CI

in this tutorial i wanted to show you how you can test a dockerized application in your gitlab ci pipeline and briefly explain the concept of services which allow you to link one or more containers in your jobs [Music] so let's begin this tutorial with an overview to understand like the big picture so what i have here is the existing pipeline for a node.js application that exposes an api the pipeline is building the software there are some unit tests that are running and then finally the application is being dockerized here in this docker stage and then you can see here inside the packages if i go inside the container registry it's just pushing this image with the latest version that's available there's no other text so things are pretty pretty simple so now we have this container containing the node.js application that exposes an api just to take a look the api is pretty simple it has only a single endpoint i'm here inside postman i'm trying out the status endpoint it's returning status up so i know that the application is working properly locally but now i just want to make sure that this docker container has been built properly and that i can deploy it somewhere else this is exactly what we're going to do right now we're going to take a look inside the gitlab ci pipeline and figure out how we can start this docker container and how to run a curl command to see if this endpoint is working properly but also how to run this postman test there so here's the pipeline as it stands right now and it goes through different stages for the build test and package state so inside the build stage things are pretty simple we're just installing any npm dependencies that this project has and we're publishing the ms artifacts then in the next stage for the unit test we're simply running the test to make sure that all the unit tests inside there are working properly and then finally what we're doing here is we're logging into the gitlab container registry we're building this docker image and then we're pushing it so pretty pretty simple and you have seen the image inside the registry now the next step would be to start this image and to run for example a curl command to see if we're getting a status app from this image if everything has been built properly just starting a node application inside here wouldn't make sense because we want to test the container that we have built so let's go ahead and maybe define another stage and call this acceptance and let's just call this curl api testing that's the job i'm going to assign it to the stage acceptance and the next step would be to start the docker container so locally the way i would start it is with docker run specifying the ports and then the docker container is started so we can try the same approach here as well i'll have to use a few things from building a docker image just gonna go ahead and copy some of the lines that you see right here on top so we're copying the image the services thing and we also have to log into the registry this will ensure that we can download this image from the registry and the command that we have to run is docker run and write in a detached mode i'm going to specify here the ports so application is running on port 3000 and locally i'm going to run it on the same port as well now to specify which image you want to run i'm going to use this predefined gitlab ci variable and i already know that this docker image that i'm using doesn't have curl installed so i'll also have to go ahead and install curl and the command would be relatively easy so http localhost on port 3000 i'm going to check the status endpoint and i'm going to pipe it and i want to check if the response contains the word up so that should be pretty much enough i'm gonna go ahead commit this file i'm gonna take a look at what the pipeline is doing in a second if we're jumping back at the pipeline execution we'll see now that our acceptance tests are actually failing so we can go inside the job and we can try to understand like what's going on you will see here that it says failed to connect to localhost port 3000. let's try to understand a bit like what happened here and where this execution started these are just the initial logs at one point we should find the point where we are locking to the registry this part worked without any issues then we're using docker run this image is not available so it's being pulled takes a few seconds to do that as well and you will see here what's being displayed here this is the container id so the container has been started properly and i already know that locally this container worked i'm adding here curl and then i'm running this command so there's something wrong with this container why is this container not working in gitlab let's go back to the pipeline and i'll explain why it is not working so you will see here that we have defined two things we have defined an image which is docker and we also have defined this docker in docker image and this is under services admittedly to understand this is not easy but i will do my best the idea of using services is when you're using the docker image you just have the docker client it's pretty similar to when you're opening a terminal on your computer and you're running a docker command and it's saying something like cannot connect to the darker demon well this ducker in docker is essentially the darker demon and this is another docker image that's being started and the docker client is communicating over a network to the docker demon now this docker daemon is the one responsible for starting this container so when you're using here docker run in detach mode specifying these ports you're not starting this docker container in this image you're actually starting it inside this service and this service is also accessible over the network so we can connect with it but it's not going to be available under localhost localhost is just this image here but this service here will get another address and by default it is the name of the container the name of this container is docker all we have to do here is replace localhost with docker the port should be the same i'm gonna commit this and in a few minutes we're gonna take a look at the pipeline as well and after a few minutes the execution of this entire pipeline has been successful so now obviously we're reaching our container we're getting back the response that we expect so everything is working properly i mentioned in the beginning that i also have some postman tests that i need to run so going back to the pi plan this postman tests are running on node.js i need newman as a cli tool so in this case i'll have to install here node.js i'll have to install this tool until i'm able to run these tests but actually it's not so efficient to start here a docker service that again starts another docker container to which i'm connecting we could use this services functionality to start our own docker image and to link it essentially to our base image whatever that is so in our base image we can specify for example newman as a dependency for running this postman test or we could run curl directly from another docker image we could use other dependencies to connect to this image and to run any kind of test that we need so i'm gonna go ahead and modify this a bit so let's go ahead here and remove docker in docker and instead of this we're gonna start our own image so the image that we have built we can specify it as a service and gitlab will start this container and link it to our base image our base image we don't need to docker because we don't want to do something like docker run all we want to do here is run this command so i want to do a curl command ideally on localhost or somewhere we don't know but ideally on localhost to get this response so we are starting our container and the image that we can use here is for example directly a curl image gonna use here curl images slash curl this will just grab us the latest curl image that only already contains curl this is just for demonstration purposes i'm gonna get this image from the service and ideally curve will be able to automatic connect now what's important to understand about services is that we can start here different docker containers but we don't have shell access to them everything that is inside here in services we can only access them over a network in this case with curl we are accessing them over http this is how it works but we cannot run any commands for example i know that this is a node js image i cannot run any node commands or something like that from the script here so everything that i'm doing is over a network previously i mentioned that the hostname of the images is determined from the name of the image now in this case i'm not really sure like what image i'm going to get back generally to avoid any confusion i highly recommend defining an alias what will happen here is we can define an alias so we can see here this is the name of the image that we want to use and here i'm going to specify an alias let's say for what i'm working here this is a banking api so i'm going to write here banking api so this will be the host name so this will be available over different communication protocols in my case over http so i'm going to modify my curl command to go to this alias and to run this command unfortunately the pipeline is still successful we can go and take a look at the logs and this time also try to understand everything that happened one of the first things that i'm noticing is that this entire job only took 22 seconds the previous execution with starting docker and docker and everything took 61 seconds so already a huge improvement in terms of performance what's happening here let's try to understand how this job is actually reading the configuration that we have provided first of all the runner will start downloading this image so curl images curl this is our base image that we're starting additionally it's going to also start the service and this is what we have published inside the gitlab registry so this is being downloaded as well it's automatically authenticated we don't need to worry about authentication or anything like that this is where the entire job execution is starting and directly we have access here to the banking api so everything has been set up the services have been started we're making this http request it is successful you can see here the response and we're done okay so now that we understand the basics around running these services let's use a more complex image so i was saying that i want to run this postman test so for the postman test i need to use the newman image from post map so let's call this step postman api testing it's still part of the acceptance tests so i'm gonna use the same stage and the image that we're using here i'm gonna use here the name postman slash newman and we can also override the entry point because we don't want to run the command that's automatically given when starting this container again for the services all we have to do is again we want to start our docker container and we can use as a base image anything that we want we can use as a base image another image inside our registry that maybe contains testing tools or all the dependencies that we need for testing purposes or for other purposes you can use a public image from docker hub so anything works both inside the image or inside the services you don't have to feel limited by anything like that in this case we're using this postman human image it has everything i need in order to run the postman test i'm gonna define here the script and the command is human run and my tests are here inside test collection.json and i'm also specifying here variable and essentially i'm overriding the base url and the base url will be this banking api on port 3000. so now i have everything in place i'm gonna make these changes and take a look at the pipeline now if we go back to the api we'll see that the postman tests have also been executed successfully newman is available as a dependency this is really a very clean way on how you can combine different docker images and bring them to your pipeline by using the services and in this case i've only started the api as a service but you can also start databases or any other services that are available over a network so hope this tutorial was helpful and that you have learned something new and that you can use this amazing functionality in your own pipelines if this tutorial was helpful give it a thumbs up leave a comment in the section below and consider subscribing to this channel for more tutorials like this one see you next time thank you for watching

Channel: Valentin Despa

Views: 16,942

Keywords: gitlab, gitlab ci, gitlab tutorial, gitlab ci services, gitlab services, gitlab docker, gitlab docker registry, gitlab build docker image, gitlab build docker image and push to registry

Id: fymJsLIwrFU

Length: 15min 47sec (947 seconds)

Published: Thu Nov 25 2021