GitLab CI/CD: Automate build and publish docker image to a private container registry.

When you work with CI/CD pipelines, there are some cases in which you want to build a Docker image with your own configuration and dependencies. In this blog, I’ll show you how to automate building and publishing a Docker image to a GitLab Container Registry (known as a private container registry).

Step 1 – Getting started

1. Create a project access token

First, we need to create an access token that allows our CI job to authenticate GitLab Container Registry. GitLab provides 4 ways to authenticate with their container registry:

• Personal access token.
• Deploy token.
• Project access token.
• Group access token.

I’ll use the Project Access token for this example. I created a project called “docker-build-demo”, you can create your own project. Then, on the GitLab project console, go to Settings -> select Access Tokens -> Project Access Tokens -> Click “Add new token” (if you don’t have one).

A new pop up shows then, you’ll enter the token info with fields:

• Token name: the name of the token (enter a meaningful name you want), I put “demo-build-and-push-docker-image-token”
• Expiration date: specify an expiration date that you want
• Select a role: you can put either “maintainer” or “developer” according to the person who works with this project
• Select scopes: You’ll need “pull” and “push” permissions so that the CI job can communicate with GitLab Container Registry. Thus, select write_registry and read_registry.

Then click “Create project access token”

After created, you’ll have a message saying that your token has been created successfully as below:

Please note down your created token, we’ll need to add this token into CI/CD variables to be used in the CI job in the next step.

2. Create a project access token

We go to Settings -> select CI/CD -> select Variables section -> click Add variable -> An pop up showing, untick Project variable and select Mask variable -> enter Key: <is variable name> – Value: <is your created token>. I have entered Key: CI_AUTH_REGISTRY_TOKEN

Step 2 – Configure build and push docker image to GitLab Container Registry

1. Create a Dockerfile for building a custom Ubuntu image

Under the project directory, we create a demo/ubuntu/20.04 path, and create a Dockerfile inside this directory

cd /path-to/docker-build-demo
mkdir -p demo/ubuntu/20.04
cd demo/ubuntu/20.04
touch Dockerfile

Add the following content to the Dockerfile:

FROM ubuntu:focal
LABEL org.opencontainers.image.authors="example@gmail.com"
LABEL author="Binh Do"
LABEL description="Base image of Ubuntu OS 20.04"

ENV TZ=Pacific/Auckland
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

RUN DEBIAN_FRONTEND=noninteractive && \
    apt-get -y dist-upgrade && \
    apt-get update && \
    apt-get -y install git tzdata

We’re building a custom Ubuntu image that has a git package as dependencies, also need to install tzdata package which can be used by other packages such as PHP or Python as dependencies in case we want to use this image as a base image.

2. Configure build and push in the .gitlab-ci.yml file

Under the root project directory, we create a .gitlab-ci.yml file.

cd /path-to/docker-build-demo
touch .gitlab-ci.yml

And add the following content to the .gitlab-ci.yml file

image: docker:20.10.17

variables:
  DOCKER_REGISTRY_HOST: $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME
  DOCKER_TLS_CERTDIR: "/certs"

services:
  - name: docker:20.10.17-dind
    alias: docker

stages:
  - build

build: 
  stage: build
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_AUTH_REGISTRY_TOKEN $CI_REGISTRY
  script:
    - echo $DOCKER_REGISTRY_HOST
    - docker build --pull=true --cache-from "$DOCKER_REGISTRY_HOST/custom-ubuntu:20.04" -t "$DOCKER_REGISTRY_HOST/custom-ubuntu:20.04" demo/ubuntu/20.04
    - docker push "$DOCKER_REGISTRY_HOST/custom-ubuntu:20.04"

Explanation:

• To build a custom Docker image by using a CI job, we need to use docker-in-docker to achieve this.
• $DOCKER_REGISTRY_HOST: we need to name our custom docker image with the naming convention so that we can push this image to GitLab Container Registry. The image should have a naming convention like the format below:

    <registry server>/<namespace>/<project>[/<optional path>]

In my example, it is registry.gitlab.com/general-projects4/docker-build-demo/custom-ubuntu with a tag of 20.04

• We should authenticate with the GitLab Container Registry before we start building a custom Docker image. To authenticate, we run

docker login -u $CI_REGISTRY_USER -p $CI_AUTH_REGISTRY_TOKEN $CI_REGISTRY

• $CI_REGISTRY_USER is the username that you’re using to push containers to the project’s GitLab Container Registry. $CI_AUTH_REGISTRY_TOKEN is the variable that we created for our project access token. And $CI_REGISTRY is the address of the container registry server, formatted as <host>[:<port>]. Click here to learn more about predefined variables. You will see a warning in the log that says that you’re running by CLI; you can ignore that warning as it’s just a default behaviour of Docker. You’re safe to do it.

• In the docker build command, the --cache-from option helps build only the changed layers, using a build cache to reduce the CI job runtime.

Result:

After pushing the code changes for your Docker image in the “docker-build-demo” project to the remote repo. You’ll see the pipeline is created with a CI job called “build”.

We click on the “build” job to see more details. Here is my result; you should have the same if you follow the correct steps.

Ok, you have published your custom Docker image successfully to the project’s GitLab Container Registry. On the left GitLab console, you can go to Deploy -> Container Registry -> you will find your docker image there.

That’s all!

To use your image. Simply run:

docker run [options] registry.gitlab.com/general-projects4/docker-build-demo/custom-ubuntu:20.04

BONUS – Avoid running unnecessary instructions.

When you want to build a custom Docker image based on another custom Docker image. You may want to avoid running unnecessary steps that have already been run in the version-based Docker image. For example, you have a Ubuntu-based Docker image with git installed as the steps we did above. Then, you want to build a custom nginx Docker image based on this custom-ubuntu:20.04. You definitely want to avoid running the git installation step again in this Nginx docker image. That seems to be redundant because the custom-ubuntu:20.04 image hasn’t been changed at all so we don’t need to install git again.

We can build this requirement into the GitLab CI/CD. I’ll show you another example of building a Nginx image for production based on the custom-ubuntu:20.04.

1. Under the project directory, you create another demo/nginx/ path and create a Dockerfile inside this directory

cd /path-to/docker-build-demo
mkdir -p demo/nginx
touch demo/nginx/Dockerfile

Add the following content to the Dockerfile:

FROM registry.gitlab.com/general-projects4/docker-build-demo/custom-ubuntu:20.04
LABEL org.opencontainers.image.authors="example@gmail.com"
LABEL author="Binh Do"
LABEL description="Base Nginx Image for Production"

RUN apt-get update && \
    apt-get -y install nginx-full

2. Next, we create a build.sh script under the root project directory

touch build.sh && vi build.sh

Add the content below to the build.sh file:

#!/bin/bash
set -o errexit -o nounset -o pipefail

function build_image {
    tag=$1
    shift
    docker build --pull=true --cache-from "$DOCKER_REGISTRY_HOST/$tag" -t "$DOCKER_REGISTRY_HOST/$tag" $@
    docker push "$DOCKER_REGISTRY_HOST/$tag"
}

# Build Ubuntu-based image
build_image custom-ubuntu:20.04 demo/ubuntu/20.04

# Build a custom Nginx image
build_image nginx:production demo/nginx

This script instructs which image should be built first and then which one should be built next. Here, the custom-ubuntu:20.04 image is built first, and then the nginx:production image is built afterwards.

3. Modify the .gitlab-ci.yml file, we change the script part to run ./build.sh script instead. We also need to install “bash” for the docker:20.10.17 image to be able to run the bash script.

image: docker:20.10.17

variables:
  DOCKER_REGISTRY_HOST: $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME
  DOCKER_TLS_CERTDIR: "/certs"

services:
  - name: docker:20.10.17-dind
    alias: docker

stages:
  - build

build: 
  stage: build
  before_script:
    # Install bash to use the build.sh script
    - apk add --no-cache --upgrade bash
    - docker login -u $CI_REGISTRY_USER -p $CI_AUTH_REGISTRY_TOKEN $CI_REGISTRY
  script:
    - ./build.sh

4. Now, commit the changes and push them to the remote repository. Remember to add execution permission git add --chmod=+x to the build.sh script. Otherwise, it can’t be run by the command ./build.sh.

git add .
git add --chmod=+x build.sh
git commit -m "Build a custom Nginx image"
git push

5. Once you push the changes, you’ll see the pipeline created then. If everything is correct, you should have a new image called nginx:production. Here is my result.

That’s all we have for this post. Enjoy your testing!