Skipping Tests in GitLab CI
Posted on 08/29/2018 by Andi Scharfstein
If you are using GitLab CI as your build server, you might be familiar with the
[skip ci] syntax documented in https://docs.gitlab.com/ce/ci/yaml/#skipping-jobs—it gives you the option to skip a build entirely by including this tag in your commit message, for example when you are only committing documentation changes.
However, there are scenarios when a build is still required but certain stages can be skipped. Let’s say you are preparing a new release, which requires a changelog entry. In this case, you can’t use
[skip ci] because you’ll probably have the release build included in your pipeline as one of the last steps (and if you don’t, I am going to suggest that it’s high time you did). On the other hand, even if you need the release build, you can probably skip some of the earlier steps (such as test executions) if they ran successfully for previous commits.
Unfortunately, GitLab CI doesn’t offer a built-in
[skip tests] tag. But you can easily build your own! As of GitLab 10.7, there is variable support for certain keywords such as ‘only’ and ‘except’. This gives us everything we need to define a pipeline where tests can be skipped on demand (of course, it goes without saying this should not become a regular operation).
The syntax is straightforward enough: you can evaluate any variable against a regular expression using the
=~ operator. Here’s the expression we are currently using in my company for our own builds to allow developers to include a [skip tests] flag in their commit messages:
$CI_COMMIT_MESSAGE =~ /\[skip[ _-]tests?\]/i
$CI_COMMIT_MESSAGE is a predefined variable provided by the build server. The regular expression we employ gives users a little flexibility in their choice of tag, in case somebody forgets the precise syntax.
It is tempting to try to define a variable to save the above flag into a global build script variable. However, at the time of this writing, evaluation of this expression for some reason did not work in the global variables block. This means that you can’t write something like the following (which was our first attempt):
$SKIP_TESTS: $CI_COMMIT_MESSAGE =~ /\[skip[ _-]tests?\]/i
This means that skipping multiple jobs becomes a little harder than it should be. However, fortunately you don’t need to repeat the above check for every job you want to skip. Instead, you can define a template like so:
.test-template: &test-template stage: tests except: variables: - $CI_COMMIT_MESSAGE =~ /\[skip[ _-]tests?\]/i - $SKIP_TESTS
This uses a non-standard YAML feature called https://docs.gitlab.com/ce/ci/yaml/#special-yaml-features, which is supported by GitLab CI’s YAML dialect. After the definition, you can just apply the template to your regular task definition:
unit-tests: <<: *test-template
And that’s really all you need to get going! As you can see above, we also included the variable
$SKIP_TESTS in the
except block of the template. This is helpful when triggering pipelines manually from GitLab’s web interface. Here’s an example:
In summary, GitLab CI gives you a powerful set of tools to control your builds. In this article, I have shown you an example of how we employ these tools at CQSE, and how you can use them as well. I hope you found this helpful—if you did, I’m always happy to hear from you in the comments!