Talk Accelerate 2018: Test Impact Analysis: How to Find New Bugs 100x Faster

Dr. Elmar Jürgens & Dr. Elmar Jürgens

Coordinates

  • Speaker: Elmar Jürgens
  • Conference: Accelerate
  • Date: Tuesday, October 9th 2018, 15:45 — 16:10
  • Venue: Austria Center Vienna, Keynote Room F

Abstract

Large automated test suites are often too slow for CI. Since their execution is deferred, many defects tend to slip through CI — increasing the time, cost and difficulty required to find and fix them. Test Impact Analysis selects only the tests impacted by code changes made since the last test execution. It then sorts these tests according to their bug discovery potential. This way, it can discover new bugs extremely efficiently. In fact, our tests could identify 80% of failing builds in 1% of the test execution time.

This allows for fast CI builds and high bug-detection rates, even for large test suites. This session explains the concepts behind Test Impact Analysis, presents research findings from large case studies and shows how it can be integrated with Tricentis Tosca.

Slides

Recording

This talk was recorded and can be watched in this video. See below for times of specific talk contents.

Content

Part I: Introduction of Test-Impact-Analysis (TIA)

Part II: How good is TIA in practice? An Empirical Evaluation using a Mutation Testing Based Benchmark

»Test Intelligence«

References

Dr. Elmar Jürgens