GitLab

Configuring & Managing GitLab Runners with Reusable Templates: My Real-World Workflow

By Binh No Comments #bash scripting #ci cd solutions #git bash #gitlab cicd #optimize ci cd

If you have ever worked with or managed GitLab CI/CD, you may be familiar with the concept of GitLab CI Runner. It basically is the worker that we use to execute CI/CD jobs, and this worker uses executors (e.g., Docker, Kubernetes) to do that. Sometimes, relying entirely on Runners provided by GitLab is not enough for multiple reasons. For example, if you run build/test jobs that require a powerful machine, the available Runners from GitLab will not provide the expected resources and will not produce the result you want, likely slowing down the build/test speed.

Having a dedicated runner managed by ourselves will give us advantages in terms of speed, security, and many optimisations. I used to write 20 tips to speed up your GitLab CI/CD pipelines in 2023 blog to talk about this. Feel free to have a read if you are interested.

When I started building my first GitLab Runner many years ago, I felt overwhelmed by the concept and the GitLab documentation for creating a GitLab Runner. Nowadays, the documentation has improved significantly, but I think it’s still a heavy learning curve if you’re new to GitLab, and you also have to dig into the documentation to build your own GitLab Runner completely.

In this article, I want to write about my workflow for configuring GitLab Runner and managing multiple Runner configurations if I have to.

Prerequisites

  • Prepare a Linux server with Docker CE installed. I’m going to use Ubuntu 24.04 in this demo. We can actually install GitLab Runner on many platforms, but Linux seems to be strongly adopted in the realm of DevOps/Cloud. Therefore, I chose to build up GitLab Runner on Linux. For other platforms, please check it on GitLab docs.
  • I’m using GitLab SaaS (https://gitlab.com) in this example. If you use your own self-hosted GitLab, the process is almost the same. There’s a minor difference in the URL which you use to register a runner.

Installing Docker

As a GitLab Runner requires us to choose executors. The most common executors preferred to use are Docker and Kubernetes, I would say. I will use Docker in this setup for simplicity.

For your corresponding OS, please consult Docker’s documentation for its installation.

Installing GitLab Runner

On your Linux server, copy the following commands and paste them into your terminal to run, I’ll do the same with my Ubuntu server:

# Set up the official GitLab repository
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash

# Install the latest package (change "apt" to the command that suites your OS)
sudo apt install gitlab-runner -y

Registering GitLab Runner

Before we register a runner, I just want to mention that we have 3 types of Runners, they are:

  • Instance runners: are available for us to use in all groups and projects in GitLab. If you use GitLab SaaS, these runners are known as shared runners, and they’re managed by the GitLab team. If you build a self-hosted GitLab, you’re in charge of managing these runners.
  • Group runners: We can use these runners in all projects and subgroups within a main group.
  • Project runners: These runners are associated with specific projects. E.g. we have a project-a, and register a runner to this project. The runner is used with this project-a only. Ideally, these runners shouldn’t be used in other projects except you enable the runner to run cross-projects.

I’m going to register a project runner in this article, for other runner types, the process is almost similar. And if you use GitLab SaaS, Project runner and Group runner are the two main types you should care about. You just need to follow the corresponding sections in Manage Runners. To register a GitLab Runner on your machine, we need to do two things:

  • Create a Runner in a project on GitLab. That is to get the generated runner authentication token.
  • Use the generated token to register the created Runner on your machine where the runner runs.

NOTED: Since GitLab 16.0, a new runner creation workflow has been introduced, and the two steps above are part of the new workflow. Before version 16.0, the authentication token was enabled by the Admin rather than requiring the creation of a new runner to obtain the token. Please see this docs if you installed a runner before version 16.0. I would recommend using the new workflow because the old workflow will be removed in version 20.0.

NOTED: One machine can have multiple runners. You just have to create new runners to get their corresponding authentication tokens that you will use to register the runners.

Create a GitLab Runner

We need to log in to GitLab, and follow:

  1. Go to Search or go to on the left sidebar and find your project and go to it.
  2. Select Settings > select CI/CD.
  3. Expand the Runners section.
  4. Select Create project runner.

It redirects you to the Create page

  1. In the Tags field, enter the job tag(s) to specify jobs (specified in the .gitlab-ci.yml file) that the runner can run. This could be one tag or multiple tags separated by a comma. If you want to run the job without a tag, just select Run untagged jobs. I named it demo-runner-1 in this example.
  2. (Optional) In the Configuration section, you can choose what you like.
    • We can skip the Runner description and the three options it’s asking for if you don’t need them.
    • We may want to set the Maximum job timeout. I set it to 3600 (1h).
  3. Select Create runner.

Next, we need to select the platform on which we want this runner to run. I’m using Ubuntu, which is Linux. At Step 1, just copy the generated runner authentication token and save it to a temporary place securely. We will use it in the later steps. DO NOT REVEAL THIS TOKEN TO THE PUBLIC.

Once you click View runners, it takes you back to the original CI/CD settings page. You will see your created runner there.

Register the created runner

At this step, the easiest way to register the created runner is just to run:

sudo gitlab-runner register

It’s going to prompt you a few questions that require you to enter. You can follow the Register with a runner authentication token to do.

This way is fine if you just want to configure a few runners. However, if you have to configure many runners on the same host machine or on different host machines, I think the best management is to create config template files where we can store them in a Git project. That’s where we can trace back what we have configured for those runners without having to remember all selected options.

Also, in worst cases, when the host machine collapses, and you want to replace it with a new one, having those config files ready makes your life easier. You just run the gitlab-runner command to register the runners with the same configurations you have done manually on the previous machine.

To register GitLab Runner with the config template file, we need to do three steps:

  • Step 1 (Optional): Create a basic bash script to run gitlab-runner command with supplied arguments. This is just a wrapper script of gitlab-runner with my custom way. You can tailor it based on your preferences. Or if you know other programming languages, feel free to do it yourself.
  • Step 2: Create a runner config template file that we can set options for the [[runner]] section only in this file.
  • Step 3: Use the script to register runner with the created template config file.

For options in the [[runner]] section, please check advanced-configuration for more details.

Step 1: Create a register-runner script.

I’ve added the content of this script below at https://gitlab.com/-/snippets/4891991 in case there’s something wrong with the WordPress format in this blog.

The reason we create this script instead of using the gitlab-runner command directly is that there are a few arguments, such as --url, --executor, and --docker-image (maybe more) which we will likely set the same for all runners, for example, in my case. Then I need to provide the template file and token for each runner I register with reusable commands. If you don’t like this way, please run the gitlab-runner command with your own options. You can refer to my script in the gitlab-runner part to see how I register the runner.

Run sudo nano /usr/local/bin/register-runner, and add the following content:

#!/usr/bin/env bash

show_help() {
    echo "Usage: $0 [OPTIONS]"
    echo
    echo "Options:"
    echo "  -n, --name     VALUE      Set name for the runner. Default to 'docker-runner'"
    echo "  --template     FILE       Specify a runner template file. Should contain options from [[runner]] section only"
    echo "                            Please visit https://docs.gitlab.com/runner/configuration/advanced-configuration/ to know more"
    echo "  --token        TOKEN      Specify runner authentication token"
    echo "  -e, --executor VALUE      Set executor to the runner. Default to 'docker'"
    echo "  -u, --url      VALUE      Set url for the runner. Default to 'https://gitlab.com/'"
    echo "  -i, --image    VALUE      Set default image to the runner. Default to 'alpine:latest'"
    echo "  -h, --help     VALUE      Show this help message"
}

# Parse options with getopt
OPTIONS=$(getopt -o "n:e:u:i:h" \
    --long name:,template:,token:,executor:,url:,image:,help \
    -n "$0" -- "$@")

if [ $? -ne 0 ]; then
    echo "Try '$0 --help' for usage." >&2
    exit 1
fi

# Reorder arguments as parsed by getopt
eval set -- "$OPTIONS"

# Default values
name="docker-runner"
template=""
token=""
executor="docker"
url="https://gitlab.com/"
image="alpine:latest"

# Extract options
while true; do
    case "$1" in
        -n|--name)      name="$2"; shift 2 ;;
        --template)     template="$2"; shift 2 ;;
        --token)        token="$2"; shift 2 ;;
        -e|--executor)  executor="$2"; shift 2 ;;
        -u|--url)       url="$2"; shift 2 ;;
        -i|--image)     image="$2"; shift 2 ;;
        -h|--help)      show_help; exit 0 ;;
        --) shift; break ;;
        *) echo "Invalid option: $1"; show_help; exit 1 ;;
    esac
done

# Validate required options
if [[ -z "$template" || -z "$token" ]]; then
    echo "Error: --template and --token are required." >&2
    show_help
    exit 1
fi

echo ">>> Registering runner: $name...."
gitlab-runner register \
  --non-interactive \
  --template-config "$template" \
  --url "$url" \
  --token "$token" \
  --executor "$executor" \
  --docker-image "$image" \
  --name "$name"

if [ $? -ne 0 ]; then
    echo "Unsuccessful Runner Register. Please check manually!" >&2
    exit 1
fi
echo ">>> Done"

Press Ctrl + O and press Enter to save the content. Then press Ctrl + X to exit.

I’m setting default values for name, url, executor, docker-image. You can modify the script based on your preferences or add more options if you need.

Grant executable permission to the script

sudo chmod +x /usr/local/bin/register-runner

Step 2: Create a runner config template file

Create a directory to store the config template files. In the future, this directory could become a Git project that you clone from GitLab, GitHub, etc. And it should store all the templates that you used.

sudo mkdir -p /data/runner_template

We will create a basic config template file for the demo-runner-1 runner with a few custom settings, run

sudo nano /data/runner_template/demo-runner-1_template.toml

And add the following content:

[[runners]]
  request_concurrency = 1
  environment = ["DOCKER_DRIVER=overlay2"]
  [runners.docker]
    memory = "256m"
    memory_swap = "256m"
    memory_reservation = "128m"
    cpus = ".2"

Press Ctrl + O and press Enter to save the content. Then press Ctrl + X to exit.

Having a separate template file for configurations of each runner gives us the ability to add more custom configurations that we need. For example, we can bind a directory on the host machine to a directory inside the container

[[runners]]
  ...
  [runners.docker]
    ...
    volumes = ["/path/to/directory/from/host:/path/to/directory/in/container:rw"]

Again, please refer to advanced-configuration for more explanation of settings defined here and other settings as well.

Also, you can use one template for multiple runners. You may question why do I need that? It’s just a way that I manage the runners. For example, you may have a generic template file (e.g. /data/runner_template/generic_template.toml) where multiple runners can use to register with the same settings. And you also have a specific template file for a particular runner with its own resource requirements.

Step 3: Register the runner

Next, to register the runner, we just need to run the script with supplied arguments (change values to your own configs) as follows:

sudo register-runner \
    --name "demo-runner-1" \
    --template "/data/runner_template/demo-runner-1_template.toml" \
    --token "PUT-YOUR-COPIED-AUTHENTICATION-TOKEN-HERE"

Output:

Ok. We registered the runner successfully on this host machine.

If you register GitLab Runner to a self-hosted GitLab. Please provide your own URL. For example:

sudo register-runner \
    --name "demo-runner-1" \
    --template "/data/runner_template/demo-runner-1_template.toml" \
    --token "PUT-YOUR-COPIED-AUTHENTICATION-TOKEN-HERE" \
    --url "YOUR-OWN-GITLAB-DOMAIN"

Because we use sudo to register the runner, it runs as root. Therefore, it’s going to merge the content of the template file into /etc/gitlab-runner/config.toml. We can check the config by running

sudo cat /etc/gitlab-runner/config.toml

Output:

We can also edit this file directly to change the settings, and it’s going to take effect right after a few seconds. However, I would recommend that whatever you change here under [[runner]] section, you update the content of the template file corresponding with the runner you changed as well. That is to have a record of what you have changed in order to register the runner again with the same configuration in the future.

If you go back to the CI/CD settings section on GitLab’s project page. You will see the runner status now shows a green circle icon, which indicates we registered the runner successfully.

Testing

To test the new registered runner, in the project where the runner is registered, I create a .gitlab-ci.yml under the project root path on the local machine with content:

stages:
  - build

test:build:
  stage: build
  # We have to specify the runner's tag here to let the job pick the runner
  tags:
    - demo-runner-1
  script:
    - echo "Test registered runner - demo-runner-1"
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

And then commit this change and push it to the GitLab repository. Then, I create a merge request to trigger the CI job named “test:build” to use the demo-runner-1 runner.

Output:

Noted that we have to specify the runner’s tag on jobs that we want them to use this runner, as shown demo-runner-1 in this example. Otherwise, it will pick a random shared runner available on GitLab.

Conclusion

For now, every time we want to register a new runner, just repeat the process above.

With the DevOps/Automation mindset, I don’t think that we should do a lot manually like this if you have to manage many runners (maybe fine if you just have 1 or 2 runners). Later in the future, you could eventually extend the script above to automatically create a Runner by using the GitLab REST API, and also register the runner after that. See this doc for reference. The main idea is to speed up the registration process by simply running the script, which automates everything without requiring manual configuration. I recently extended the Bash script to do this, and wrote Automate GitLab Runner Registration and Unregistration blog to do this.

Furthermore, if we need to rebuild the host machine, even with a script to automate the runner registration process, what if we have hundreds of runners? We still have to run the script manually for each runner, sounds not good, right? I think that’s when we need configuration management. We can also integrate the registration process with tools such as Puppet, Ansible, Chef, or something similar to achieve this. Once we launch a host machine, we just use one of those tools to deploy the registration process for all the desired runners that we need. I just want to share what it looks like in reality when we want to automate the whole process, and this requires a deep understanding of either the tools or your own system. Once you’ve done that, it serves you in the long run. Also, if you use Kubernetes, that’s a different approach. I don’t have much experience with Kubernetes, so I’m not able to share much here. Perhaps, you may have to find what solution works best for you then.

Discover more from Turn DevOps Easier

Subscribe to get the latest posts sent to your email.

By Binh

Leave a Reply

Your email address will not be published. Required fields are marked *

Table of Contents

→ Content on this page