Puppet Series – Automate GitLab CE setup

In this blog, I’ll show you how I automate gitlab-ce (Gitlab Community Edition) installation with Puppet (we can also do this for gitlab-ee known as GitLab Enterprise Edition). This is another use case of Puppet from Puppet Series that I’m writing about.

Prerequisites

• Have Puppet Server installed with Roles and Profiles architecture that I implemented in the previous blogs, you may want to check it out at Puppet Series.
• Also, prepare an Ubuntu 24.04 with a minimum of 4GB Memory and 4 vCPUs to install GitLab. I also provisioned this server by Vagrant, and let it register Puppet Server as a Puppet Agent already.

I’ve added all the code written below in https://gitlab.com/binhdt2611/puppet-demo if you want to use this as a reference.

Adding puppet-gitlab module

Under the project root path in https://gitlab.com/binhdt2611/puppet-demo, we create a branch named feat_gitlab_setup, and start developing on this branch.

Next, to use Puppet to install/setup GitLab, we will need to add puppet-gitlab module and its module dependencies to the Puppetfile file (still under the project root path).

mod 'puppetlabs-stdlib', '9.7.0' 
mod 'puppetlabs-apt', '10.0.1'    
mod 'puppet-gitlab', '10.3.0'

The puppetlabs-stdlib and puppetlabs-apt have been mentioned in the previous blogs if you have followed me. There’s no need to redeclare both of these if you already have them.

puppet-gitlab takes advantage of the provided Omnibus packages and the packagecloud package repositories to manage and install GitLab. Omnibus GitLab package, typically includes gitlab-ce or gitlab-ee.

Create a central profile class for installing GitLab

We will need to create a central profile class named profiles::gitlab. This profile class applies all GitLab-related configurations to a server that requires GitLab setup.

Still under the project root path, we create a site/profiles/manifests/gitlab.pp file. That’s where we add class profiles::gitlab {}, and we declare the following content:

# Class: profiles::gitlab
#
#  This profile use puppet-gitlab module to install and set up a GitLab server.
#  We can also include other classes (for other configurations) here 
#  that the GitLab server need
#
class profiles::gitlab {
  # Calls to puppet-gitlab module to install/setup the GitLab server.
  include gitlab
}

We don’t specify parameters that a GitLab server needs in this class. Instead, we utilise Hiera to manage these parameters.

Create Hiera data for managing GitLab parameters

We need to create a Hiera data file storing the common GitLab settings that gitlab-server server can query when the profiles::gitlab class starts running. Under the project root path, create a data/nodes/gitlab-server.srv.local.yaml file with the content:

---
# Let Hiera knows that "gitlab-server" server should use 'roles::gitlab'
server::role: 'roles::gitlab'

# The main class (init.pp) exposes the configuration sections from the gitlab.rb configuration file as hashes
# All these parameters we use below can be found in 
# https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template

# Set an external url where the GitLab server can be reached
# Change this to your own URL. 
gitlab::external_url: 'http://gitlab.turndevopseasier.com'

# Uncomment this if you need to install 'gitlab-ee'
# gitlab::manage_upstream_edition: 'ee'

# Other optional options for Rails
# You can skip these config below if you just want to use the default.
# The idea is to show you how we can change the GitLab config through Hiera.
gitlab::gitlab_rails:
  time_zone: 'Pacific/Auckland'         # Change to your time_zone
  gitlab_email_enabled: false           # disable email as I'm running locally
  gitlab_default_theme: 2               # Use Dark theme
  gitlab_email_display_name: 'GitLab'   
gitlab::sidekiq:
  shutdown_timeout: 5

Create a role class for GitLab

Next, we will create a role class named roles::gitlab that the gitlab-server.srv.local server should apply to. This role calls the profiles::gitlab class to apply the defined settings. We create the site/roles/manifests/gitlab.pp file and add the following content:

# Class: roles::gitlab
#
#   This role is in charge of installing all necessary GitLab server's components
#
class roles::gitlab inherits roles::base {
  include profiles::gitlab
}

When we run puppet agent -t command on gitlab-server, it looks up Hiera at data/nodes/gitlab-server.srv.local.yaml to search for the server::role key. This key returns 'roles::gitlab' to tell the server which role class it should use.

Ok, that’s all we need to set up. Now commit all your changes and push this branch to the remote project on GitLab.

git add .
git commit -m "#feat - Automate GitLab setup"
git push origin feat_gitlab_setup

Testing

Once you push all your changes to the remote branch, log in to your puppet-master server, and run

sudo /opt/puppetlabs/puppet/bin/r10k deploy environment -m -v 

This deploys your new branch to the corresponding environments.

After that, we can log in to the gitlab-server and run this command to start installing and setting up GitLab.

sudo /opt/puppetlabs/bin/puppet agent -t --environment=feat_gitlab_setup

This command rolls out the new configurations we added from the feat_gitlab_setup branch.

It’s going to take a while for Puppet to set it up (about 20 minutes, depending on your network), so feel free to grab a cup of coffee while waiting for Puppet to finish. When you see something that looks like the below, we’re done now.

Next, we run cat /etc/gitlab/initial_root_password to get the inital generated password by default.

Copy the generated password from the output. Then we open a browser and access GitLab with the configured external URL that we added. My URL: http://gitlab.turndevopseasier.com/. Log in with username/password as root/<the generated password>. For example:

Once signed in, the main dashboard will show up then:

Ok. We’re done installing GitLab CE now. You can consider changing the default root password by clicking on the profile icon, choosing the Edit profile option, and going to the password section to change the password.

It’s up to you from now on. Finally, don’t forget to merge the feat_gitlab_setup branch into production once you find everything works well for you. Later in the future, gitlab-server just use the production branch by default.

My thoughts

Because I use Vagrant to provision this gitlab-server server. When the server is launched, it sends a signing request to the Puppet Server to authenticate. The Puppet Server verifies that the gitlab-server server is a valid node and signs the request automatically. That’s because I have configured the Policy-based Autosigning feature for the Puppet server.

Once the gitlab-server is signed, it automatically pulls the Catalog (which contains configurations for a gitlab-server server) and sets up the server. I don’t even have to touch it and just wait for gitlab-server to be up and running with GitLab installed.

Every time I destroy this gitlab-server node and relaunch a fresh one, the process is repeated without manual intervention. In the production environment, the process remains the same. For example, if you have an existing infrastructure on AWS with Puppet Server already installed. When you provision your EC2 instance using Terraform or CloudFormation, you need to set up an initial script (or any other method) to let Puppet Agent nodes register with the Puppet Server. Once they’re signed as agents, they will pull the configurations corresponding to the role of the EC2 instance you provisioned. Everything will be automatically set up then.

Imagine when you have configured a bunch of settings for this gitlab-server manually? If you have to relaunch this server, it becomes a nightmare because you have to trace back all the configurations that you have set. Even if you have documentation of those steps, it is still a tedious task.

You see how useful it is when we have this configured by Puppet. All you need to do is provision a node and let Puppet automate the configuration for you. Furthermore, Puppet ensures the working state of the gitlab-server server. If you mess up with the GitLab config, you just need to re-run Puppet, which automatically brings the server to the original working state. Or, if you turn on the puppet-agent service, it runs at an interval to check whether the configuration is diverged and brings it back.

I know we still have to develop some custom scripts for Vagrant, or if you use Terraform or any provisioning tools. However, Puppet plays a role in the realm of configuration management, not provisioning, so we need this combination for us to work completely. I think Puppet itself, combined with any provisioning tool, will definitely not only reduce our time in setting this up manually but also serve as documentation that we can refer to when needed.

It’s going to take time for you to automate the whole process initially. However, in the long run, you have built a robust system where you can recycle any resource (server) in a short time without being headachy with manual configuration.