GitLab Runner allows you to run CI jobs.
Can you run Runner on the same server as your GitLab instance?
- It's possible however I wouldn't recommend it. It's better to keep your DevOps tooling (source code management, etc.) separate from code execution due to security and performance concerns.
- The security concern is due to malicious code execution in your jobs, for example, it may be possible for someone to write a unit test that traverses your server's file system providing them with access to other data on that server.
- The performance concern is simply that your jobs will be competing for the same resources (CPU, memory, storage) as the rest of your GitLab instance and you may notice a slow down in the responsiveness of your GitLab instance whenever there's a job running.
See https://jeromezng.com/how-to-install-docker-on-digital-ocean
Setup VPC
It might make sense to setup a VPC with my GitLab Instance on Digital Ocean to limit network access into thet runner, however, I need to think this through a bit more.
Docker volumes allow you to persist state data across containers. We'll be creating a volume to allow us to persist gitlab-runner configuration data across our containers.
docker volume create gitlab-runner-config
Inspect the full details of your volume by running:
docker volume inspect gitlab-runner-config
This will specifically show you the mountpoint:
[
{
"CreatedAt": "2021-12-19T09:30:54Z",
"Driver": "local",
"Labels": {},
"Mountpoint": "/var/lib/docker/volumes/gitlab-runner-config/_data",
"Name": "gitlab-runner-config",
"Options": {},
"Scope": "local"
}
Change directory to the mountpoint to see what data is persisted across containers.
cd /var/lib/docker/volumes/gitlab-runner-config/_data
Start the GitLab Runner container and expose port -p 8093:8093
docker run -d --name gitlab-runner --restart always \
-p 8093:8093 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v gitlab-runner-config:/etc/gitlab-runner \
gitlab/gitlab-runner:latest
List all your docker containers
docker ps
Ensure that the gitlab-runner container is running and exposing port 8093.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1234567890 gitlab/gitlab-runner:latest "/usr/bin/dumb-init …" 3 hours ago Up 46 minutes 0.0.0.0:8093->8093/tcp, :::8093->8093/tcp gitlab-runner
Once you've confirmed the gitlab-runner container is running, you will also need to confirm that gitlab-runner's session_server is listening and advertising on port 8093.
Open your config file
cd /var/lib/docker/volumes/gitlab-runner-config/_data
nano config.toml
Add the following session_server configuration and save
[session_server]
listen_address = "[::]:8093"
advertise_address = "code.example.com:8093"
session_timeout = 1800
Restart gitlab-runner
docker restart gitlab-runner
Now that the gitlab-runner container is started, you will now need to register the runner with your GitLab instance.
You can choose to register your runner as a (1) shared runner for your entire instance (2) a group-specific runner (3) a project-specific runner. Since my GitLab instance is for personal use, I decided to use shared runner which can be used by all my groups and projects.
You can register your gitlab-runner using
docker run --rm -it -v gitlab-runner-config:/etc/gitlab-runner gitlab/gitlab-runner:latest register
You will be prompted for a few details. Your registration token for a shared runner can be found in GitLab by navigating to Admin > Overview > Runners (https://code.example.com/admin/runners). These details will be saved into config.toml.
Enter the GitLab instance URL (for example, https://gitlab.com/): https://code.example.com
Enter the registration token: 1234567890
Enter a description for the runner: `code-example-com-shared-runner`
Enter tags for the runner (comma-separated):
Enter an executor: virtualbox, docker+machine, docker-ssh+machine, kubernetes, docker-ssh, parallels, ssh, custom, docker, shell:
docker
Enter the default Docker image (for example, ruby:2.6):
ruby:latest
Open your config file
cd /var/lib/docker/volumes/gitlab-runner-config/_data
nano config.toml
Ensure that your config file now contains the registration details you previously entered
concurrent = 1
check_interval = 0
[session_server]
listen_address = "[::]:8093"
advertise_address = "code.example.com:8093"
session_timeout = 1800
[[runners]]
name = "code-example-com-shared-runner"
url = "https://code.example.com"
token = "1234567890"
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "ruby:latest"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
Restart gitlab-runner
docker restart gitlab-runner
Check the logs of gitlab-runner
docker logs gitlab-runner
Your logs should look something like this. I'm not sure why the logs say listen_address not defined, metrics & debug endpoints disabled builds=0 but it seems to be working and I'll look into that another time.
Configuration loaded builds=0
listen_address not defined, metrics & debug endpoints disabled builds=0
Session server listening address=[::]:8093 builds=0
Verify your runner was successfully registered by navigating back into your GitLab instance. For shared runners, navigate to Admin > Overview > Runners (https://code.example.com/admin/runners). Your runner should now be listed on this page. Ensure that the Last contact is showing a recent time, otherwise it means your GitLab instance hasn't been able to make contact with your runner which could be for a number of reasons including having the incorrect config.toml or not exposing the right ports in your docker container.
Type/State: Shared
Runner: code-example-com-shared-runner
Version: 14.6.0
IP Address: 0.0.0.0
Projects: n/a
Jobs: 1
Tags:
Last contact: 1 min ago
In order to verify your runners were successfully setup, you will need to run a test job on an existing project to see if a pipeline is successfully created. The easiest way I found to do this is to clone an existing project which already has /.gitlab-ci.yml defined.
I chose to clone the GitLab Pages Jekyll project https://gitlab.com/pages/jekyll into my GitLab instance. After cloning the project, I navigated to the Project Name > CI/CD > Jobs (https://code.jeromezng.com/my-projects/jekyll/-/jobs) and pressed the "retry" icon on an existing job. If there are no existing jobs, you may need to make a commit to trigger the pipeline.
When the job is running, click into the job and view the logs. You should see the job being run on the runner you provisioned code-example-com-shared-runner
Running with gitlab-runner 14.6.0 (1234567890)
on code-example-com-shared-runner 1234567890
Preparing the "docker" executor 00:05
Using Docker executor with image ruby:latest ...
Pulling docker image ruby:latest ...
...
That's it! You now have GitLab's runner setup. You can now modify your .gitlab-ci.yml to according to your integration or deployment needs.