Setting up CI/CD for Ruby on Rails on GitLab

Recently I decided to exit the GitHub world and set up shop elsewhere. Nothing personal, just wanted something different and so after wandering around and testing some options I decided the way to go was going to self-host a GitLab instance.

Once I started exploring GitLab’s functionality one of the things that caught my attention almost instantly was the built-in CI/CD pipelines option. I had to try it.

Now, the process was absolutely not easy, the documentation for managing the specifics of a Rails CI/CD pipeline are not on-point, they are too scattered and in general are hard to understand because everybody assumes you know what you’re doing already. So, I had to piece things together and fail about 200 times before I finally got it to work.

Lets Start!

Few things before I explain the procedure:

• I use Hatchbox for my deployments, its just a life saver. I have nothing but love for this service. If you run your deployments in a different way then skip all the Hatchbox specifics. Alternatively, if you want an amazing guide on deploying Rails from scratch you should follow Ralf Ebert’s one, which I believe is the most compelling and up to date one out there right now.

• I deploy my servers using Vultr, but for the most part the deployment and setting on a VPS will the be same across the board among similar providers. The ones I’ve tried and know will work just the same are: UpCloud, DigitalOcean and Linode.

• I don’t have a Gitlab.com account, I’m working from a self-hosted instance so I’m not 100% sure everything I describe here will be the same while using Gitlab.com

Setup a Runner

To run a pipeline you are going to need a Runner, or more specifically, to run the pipeline that I will be trying in this post.

A runner is an external server instance that is set to perform actions delegated by the CI/CD pipeline.

Let’s just setup one right quick. This will be my first VPS. Head to your provider and spun up a fresh Linux machine. I will be using Ubuntu 20.04 x64.

Vultr’s marketplace gives me the option to deploy it with Docker by default:

I will go with that because on a further step I’m going to tell the Runner to use it to run my pipelines.

Once its all done SSH into the fresh server and follow the steps on the Install Runner Manually on GNU/Linux from the GitLab documentation, which basically translates to (I will be using the “Using binary file” method):

sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64"

Pay attention to the specific server architecture you are dealing with. Like I mentioned earlier, I’m setting things up on this machine using Ubuntu 20.04 x64.

# Make package executable
sudo chmod +x /usr/local/bin/gitlab-runner

# Create CI user
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash

# Install
sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner

# Start the Runner
sudo gitlab-runner start

OK, that’s all done now. Leave the shell session open, go back to GitLab and register the newly created Runner. Head to the repo you are trying to get a pipeline set for and from the left hand side menu select the option Settings > CI/CD.

Once in, expand the Runners section and copy your Registration Token.

Head back to your runner VPS shell, you know, the one you left open, and register a new Runner by following the Registering Runners documentation.

This will prompt a series of questions:

• GitLab instance URL.
• Registration Token (the one you just copied from GitLab).
• Executor: There are a number of options, for this instance I select Docker.
• Docker image: I’m using ruby:3.0.2 but you can replace for the one your are using, in spite of this, you can actually override it from your CI/DC yaml file later on.
• Description: Something you can use later on to refer to the pipeline from your GitLab yaml file. I will use myapp-runner
• Maintenance Note: I kept leaving it blank for lack of imagination. You go ahead and explore.
• Tag List: The tag list will help you control multiple pipelines for a project since you can make certain Runners come up for specific actions defined on the pipeline. I’m going to add a single tag ruby for this Runner.

We’re all done here. Back to GitLab.

Now we can dive into actually creating the pipeline action from GitLab. You can actually go back to Settings > CI/CD > Runners and your newly created Runner should show under the Registration Token on the Available specific runners section:

Go to the repo and select the Editor option from the CI/CD > Editor menu.

Setup a Pipeline

Once the editor opens up is when my absolute nightmare started. I was just incredibly unable to come up with a suitable configuration from all the literature I was following. To make matters worse, there was syntactic differences between each post/tutorial I followed so I was unable to get my file to pass.

One important thing to note is that you must abide by the Yaml syntax or the file will simple not pass, perhaps a the nicest thing is that as you type the textbox it will let you know if the file will pass or not.

My pipeline ended up looking like this:

image: ruby:3.0.2

default:
    tags:
        - ruby

cache:
    paths:
        - vendor/ruby

services:
    - postgres:14.0

variables:
    DATABASE_URL: postgres://USERNAME:PASSWORD@HOST/DATABASE

before_script:
    - bundle install
    - bundle exec rails db:schema:load RAILS_ENV=test

stages:
  - build
  - deploy

build-job:
    stage: build
    script:
        - echo "Building application..."

brakeman:
    extends: build-job
    stage: build
    script:
        - bundle exec brakeman --rails7 -A

deploy-job:
    stage: deploy
    script:
        - curl https://web-hook.com/qwerty?fakewh
    environment:
        name: staging-1

A bit of an explanation of each item in here:

• image: The Docker image to be used to run this pipeline. This should match the Ruby version your project uses, the one referenced on your Gemfile or .ruby-version file.
• default: Global default values for non-specified values in jobs specific to this pipeline, in this case I specified the tags: value to be ruby. Earlier, when setting up the Runner, one of the options was to add Tags, declaring a tags on this end will be used to select a Runner.
• cache: A list of files and directories to cache in between jobs.
• variables: You can define variables to be accessed by the Runner while the pipeline is being processed. You could also defined them using Gitlab’s CI/CD Variable settings (more about this later on). In this case I just pasted my database information, perhaps not the best of a more formal scenario.
• before_script: In here you can programmatically call functions to be run before the rest of your pipeline starts executing. In my case I make sure to install my gems and then load the schema.
• stages: This is a list of all the stages you want to go through with your pipeline. I’m just using build and deploy, you can have test as well but I’m not using tests so I don’t really need that.
  For each item you will then setup a block with the -job suffix, i.e build-job.
  • build-job: For my build job I’m just printing a message in this case, and then I extended the block to include Brakeman, which is a Rails vulnerability scanner, just to show how to extend jobs with additional blocks.
    • brakeman: This block runs Brakeman tests against the codebase, you need to specify what block are you extending, what stage does that block belongs. Finally we script the specific behavior we want to trigger in this action.
  • deploy-job: In here you can set any behaviors following the same template as with build. I setup a webhook using a curl call to trigger an automated deployment off to a staging environment. Typically you would only need to add something like - curl https://webhook-address.com/abcde123
  • environment: The environment keyword in this case refers to an Environment that I created on GitLab to keep track of deployments to a particular place.

For more keywords and references visit the official docs: .gitlab-ci.yml reference.

Environments

Setting an environment to reference on your pipeline to track deployments is simple. Go to the Deployments > Environments menu and click on New Environment.

Once that’s done you can simply go to back to your pipeline Yaml file and add the name of the environment to your environment tag. And you will see deployment information on the Deployments menu once a pipeline is completed.

Database for the Pipeline

I didn’t want to touch my actual database server so I created a new database I can use just for the purpose of testing the pipeline.

In the past when I needed to use disposable remote database for testing deployments and the like I’ve always used ElephantSQL, they offer a free tier for PostgreSQL databases that it is more than enough for these scenarios. Head there and create an account,

After you’re done with setting a new instance, you are going to obtain details for your newly created database, grab the URL field content and paste it to replace the DATABASE_URL variable on your GitLab CI/CD Yaml editor.

In my case, I’m the only person with access to this repository and I don’t really care about the database contents or details, but you could also use an environment variable and store the connection string in there, for added security.

Environment Variables

If you want to use special variables in your pipeline you can set them up from GitLab by going to Settings > CI/CD and expanding the Variables option.

In my case I set the RAILS_MASTER_KEY environment variable to be used when running the pipeline. It’s here where you could set your DATABASE_URL from here and then reference the environment variable on your Yaml file for the pipeline.

Done!

You should be all set to try your newly created pipeline. Remember to pull the code from the repo first to grab your new .gitlab-ci.yml file and then commit some changes. This should trigger the CI/CD pipeline which you can later review from the CI/CD > Pipelines menu.

Click the on the green check marks to inspect details of a particular job within the pipeline and see its actual output, for example my Brakeman execution:

So whats next? Well the example here is minimal really, there are so many more things that you might want to integrate in your CI/CD pipelines like RuboCop for linting, RSpec for testing, perhaps some other security tests or performance tests. I recommend having a look at the links below as well as the official GitLab documentation as they will provide a wealth of extra information to expand on this subject. I also included a video from Dean who’s got a very thorough tutorial on this same subject but for GitHub.

Sources of Information

Tech and Beyond with Moss, GitLab CI/CD | GitLab Runner Introduction | 2022. https://www.youtube.com/watch?v=-CyVpfDQAG0

Hix on Rails, Ruby on Rails set up on Gitlab with GitlabCI. https://hixonrails.com/ruby-on-rails-tutorials/ruby-on-rails-set-up-on-gitlab-with-gitlabci/

Nimble Ways, Let’s make faster GitLab CI/CD/pipelines. https://blog.nimbleways.com/let-s-make-faster-gitlab-ci-cd-pipelines/

Deanin, Rails 7 Automated Testing With GitHub Actions For CI/CD. https://www.youtube.com/watch?v=iztoY4DnSkw

Photo

Quinten de Graaf – Unsplash: https://unsplash.com/photos/L4gN0aeaPY4