Mutation testing measures the strength of unit tests. Unlike standard code coverage, it checks that tests are actually able to detect faults in the code.

It works by automatically introducing bugs, then running the tests and checking that one fails.

For example, it might make a change like this

public void businessLogic(int i) {
     //if (i >= 100) {
    if (i > 100) { // mutated
        rejectLoan();
    }
}

If a test fails all is good. If no tests fail, mutation testing has discovered that the test suite isn’t able to detects faults at this location.

History

Mutation testing was first proposed in a student paper in 1971. Since then it has been used extensively in academic research, but was not often used in industry as it is very computationally expensive. This changed when pitest was first released in 2010.

Pitest is an open source tool that improved the efficiency of mutation testing by orders of magnitude, making it practical for day to day development.

Developed by the same team, Arcmutate builds on top of pitest, adding advanced features for professional teams.

Read more about mutate testing and pitest on the pitest site.