Skipping Tests in GitLab CI

Posted on 08/29/2018 by Andi Scharfstein

If you are using GitLab CI as your build server, you might be familiar with the [skip ci] syntax documented in https://docs.gitlab.com/ce/ci/yaml/#skipping-jobs—it gives you the option to skip a build entirely by including this tag in your commit message, for example when you are only committing documentation changes.

However, there are scenarios when a build is still required but certain stages can be skipped. Let’s say you are preparing a new release, which requires a changelog entry. In this case, you can’t use [skip ci] because you’ll probably have the release build included in your pipeline as one of the last steps (and if you don’t, I am going to suggest that it’s high time you did). On the other hand, even if you need the release build, you can probably skip some of the earlier steps (such as test executions) if they ran successfully for previous commits.

Unfortunately, GitLab CI doesn’t offer a built-in [skip tests] tag. But you can easily build your own! As of GitLab 10.7, there is variable support for certain keywords such as ‘only’ and ‘except’. This gives us everything we need to define a pipeline where tests can be skipped on demand (of course, it goes without saying this should not become a regular operation).

The syntax is straightforward enough: you can evaluate any variable against a regular expression using the =~ operator. Here’s the expression we are currently using in my company for our own builds to allow developers to include a [skip tests] flag in their commit messages:

$CI_COMMIT_MESSAGE =~ /\[skip[ _-]tests?\]/i

Here, $CI_COMMIT_MESSAGE is a predefined variable provided by the build server. The regular expression we employ gives users a little flexibility in their choice of tag, in case somebody forgets the precise syntax.

It is tempting to try to define a variable to save the above flag into a global build script variable. However, at the time of this writing, evaluation of this expression for some reason did not work in the global variables block. This means that you can’t write something like the following (which was our first attempt):

$SKIP_TESTS: $CI_COMMIT_MESSAGE =~ /\[skip[ _-]tests?\]/i

This means that skipping multiple jobs becomes a little harder than it should be. However, fortunately you don’t need to repeat the above check for every job you want to skip. Instead, you can define a template like so:

.test-template: &test-template
  stage: tests
  except:
    variables:
    - $CI_COMMIT_MESSAGE =~ /\[skip[ _-]tests?\]/i
    - $SKIP_TESTS

This uses a non-standard YAML feature called https://docs.gitlab.com/ce/ci/yaml/#special-yaml-features, which is supported by GitLab CI’s YAML dialect. After the definition, you can just apply the template to your regular task definition:

unit-tests:
  <<: *test-template

And that’s really all you need to get going! As you can see above, we also included the variable $SKIP_TESTS in the except block of the template. This is helpful when triggering pipelines manually from GitLab’s web interface. Here’s an example:

Usage of `$SKIP_TESTS` when manually triggering a pipeline

In summary, GitLab CI gives you a powerful set of tools to control your builds. In this article, I have shown you an example of how we employ these tools at CQSE, and how you can use them as well. I hope you found this helpful—if you did, I’m always happy to hear from you in the comments!

Click to activate comments for this page.

Please note that comments use the third-party integration Disqus, which shares data with both Twitter and Facebook and has a separate privacy policy that you have to agree to before activating comments on this page.