Running docker jobs inside Jenkins running on docker
Jenkins is a free and open source automation server, which is used to automate software building, testing, deployment, etc.
I wanted to have a quick and easy way to run Jenkins inside docker, but also use docker containers to run jobs on the dockerized Jenkins. Using docker for jobs makes it easy to encode job runtime dependencies in the source code repo itself.
The official document on running Jenkins in docker is pretty comprehensive. But, I wanted a version using docker-compose (on Linux).
So, I started with a basic compose file:
When using this (
docker-compose up -d ), things came up properly, but Jenkins did not have access to the docker daemon running on the host. Also, the docker cli binary is not present inside the container.
The way to achieve this was to mount the docker socket and cli binary to inside the container so that it can be accessed. So, we come to the following compose file:
But, when trying to run
docker ps inside the container with the above compose file, I was still getting the error:
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock. This is because the Jenkins container is running with the
jenkins user, which does not have access to use that socket.
From my research, the commonly recommended ways to solve this problem were:
- Run the container as root user
chmodthe socket file to
sudoinside the container and give the
jenkinsuser access to sudo without needing to enter password.
A more secure way is to create the
docker group inside the container, and add the
jenkins user to that group. But, this requires us to build a custom image.
Also, the group id of the
docker group inside and outside the container have to be the same, so I had to add an extra check which deletes any existing group inside the container which uses the same group id, then creates the new
docker group with the passed group id, and then adds the
jenkins user to the
So, the final
And the final
docker-compose.yml file is:
docker_group_id argument can be edited in the compose file. Command to get the group id of docker:
$ getent group docker | cut -d: -f3
With the above, everything works:
$ docker-compose up -d Creating network "jenkins_test_default" with the default driver Building jenkins Step 1/6 : FROM jenkins/jenkins:alpine alpine: Pulling from jenkins/jenkins 801bfaa63ef2: Pull complete 2b72e22c6786: Pull complete 8d16efe80b55: Pull complete 682cd8857a9a: Pull complete 29c6010e8988: Pull complete fa466f5d199d: Pull complete e047245de0ff: Pull complete 0cfb53380af7: Pull complete c29612b1a095: Pull complete cd7d4bd47719: Pull complete 21cd3d960a1f: Pull complete f3962370d584: Pull complete bd6f35a1ea17: Pull complete bd0c271b250f: Pull complete Digest: sha256:1c3d9a1ed55911f9b165dd122118bff5da57520effb180d36b5c19d2a0cfe645 Status: Downloaded newer image for jenkins/jenkins:alpine ---> e14be04b79e8 Step 2/6 : ARG docker_group_id=999 ---> Running in f1922fa97177 Removing intermediate container f1922fa97177 ---> 79460069fb98 Step 3/6 : RUN echo "Assuming docker group id: $docker_group_id" ---> Running in 11809f4ae767 Assuming docker group id: 999 Removing intermediate container 11809f4ae767 ---> e89b345f6c74 Step 4/6 : USER root ---> Running in b2e311372bc9 Removing intermediate container b2e311372bc9 ---> 9d4d8c3ad5b2 Step 5/6 : RUN old_group=$(getent group $docker_group_id | cut -d: -f1) && ([ -z "$old_group" ] || delgroup "$old_group") && addgroup -g $docker_group_id docker && addgroup jenkins docker ---> Running in 357046a8ac49 Removing intermediate container 357046a8ac49 ---> 865b942324eb Step 6/6 : USER jenkins ---> Running in dbc2976f62c0 Removing intermediate container dbc2976f62c0 ---> c7e6fac0187c Successfully built c7e6fac0187c Successfully tagged jenkins_test_jenkins:latest WARNING: Image for service jenkins was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`. Creating jenkins ... done $ docker-compose exec jenkins docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 6c05ee1315e4 jenkins_test_jenkins "/sbin/tini -- /usr/…" 47 seconds ago Up 47 seconds 50000/tcp, 0.0.0.0:8081->8080/tcp jenkins
Here is an excellent guide on how to setup Jenkins configuration as code. This will make this setup even better because nothing will need to be configured inside Jenkins manually - it will all be driven by code / files.
srijan's blog Newsletter
Join the newsletter to receive the latest updates in your inbox.