Pre-Commit Analysis

Posted on 02/22/2018 by Dr. Benjamin Hummel

With Teamscale 4.0 we finally released pre-commit analysis, a feature that we and many of our customers have been looking forward to for quite some time. In this post, I will give you a brief overview of the feature and explain, why we are so excited about it.

At Teamscale’s core, its incremental analysis engine updates the analysis state within seconds, based only on the changed files in each commit. This allows us to give real-time feedback to developers immediately after they have committed and enables the retrospective analysis of entire code-bases. We stress this rapid feedback a lot, as in our experience this is one of the key factors for making developers actually improve the code by using an analysis tool.

Is immediately after the commit fast enough?

Many of our customers asked whether feedback after the commit might be too late. Afterall, the code is already in the repository and all problems are recorded there as well. This also manifested in our own reporitory, where many commits are quickly followed by a second one, which fixes any quality defects found.

The obvious solution and an approach implemented by many other tools is to move the analysis to the client and into the IDE. While this sounds like a good idea in the first place, this has several serious drawbacks:

  • Missing History: Often the version history is not even present on the client machine and even if it is, the analysis tools do not respect it. This makes it hard for the developer to differentiate between new quality issues and those that were introduced by the changes.

  • Performance: Client-side analysis can consume a lot of CPU cycles. Depending on the implementation and amount of analyses, a full run on the workspace in the IDE can take many minutes. Even if there is an incremental mode that checks only changed files (which by the way is often crippled and skips global analysis completely), this can consume a lot of ressources and slow down the entire IDE.

  • Missing Context: Not every developer has checked out all of the relevant code, but maybe only the subdirectory he is working on. For global analyses, such as clone detection or security analyses, the entire code base must be present.

  • Configuration: You must somehow ensure, that all developers use the same configuration of the tools, as otherwise everyone will get different findings

  • Deployment: With every new release and bug-fix of the analysis engine, all developer machines would need the updated version, which can be quite a hassle, especially if you are behind a company firewall and can not just use the auto-update of the IDE

Our Answer: Server-Side Pre-Commit Analysis

Our answer to these challenges is what we call server-side pre-commit analysis. It works like this: First, the IDE pushes the changed files to the server. Teamscale then inserts these files into a virtual private branch that only exists within Teamscale and performs its normal branch-aware analysis. As our incremental analysis is optimized for dealing with few changed files, the feedback is ready to be displayed in the IDE within seconds.

The developer can use all the same Teamscale features and analysis as for a real commit, but without actually performing the commit. This gives the developer the chance to clean up before actually committing code to the repository. While technically doing many commits is almost never a problem, cleaning up before committing rids the developer of the social judgement he might face for a “bad commit”.

Pre-Commit in the Web UI

By using server-side pre-commit analysis, all analysis and configuration is kept in the server. The clients only need to be updated, if the upload interface would change. Additionally, as the server knows all the code and all the history of the project, both can be respected during the analysis. An added benefit is that, as the code is (temporarily) known to Teamscale, all features of the web UI can be used. This includes trends, dashboards, and delta analysis.

The 4.0 release enabled pre-commit analysis in the server and the Eclipse plugin. The next releases will then gradually roll out pre-commit support to Visual Studio, IntelliJ and our upcoming Netbeans plugin.

If you are as excited as I am or have any questions, let me know in the comments!