Diff-based mutations

Normally, Mull looks for mutations in all files of a project. Depending on a project’s size, a number of mutations can be very large, so running Mull against all of them might be a rather slow process. Speed aside, an analysis of a large mutation data sets can be very time consuming work to be done by a user.

Diff-based incremental mutation testing is a feature that enables running Mull only on the mutations found in Git Diff changesets. Instead of analysing all files and functions, Mull only finds mutations in the source lines that are covered by a particular Git Diff changeset.

Example: if a Git diff is created from a project’s Git tree and the diff is only one line, Mull will only find mutations in that line and will skip everything else.

To enable incremental mutation testing, two config options have to be provided to Mull: gitDiffRef: <branch or commit> and gitProjectRoot: <path> which is a path to a project’s Git root path.

An additional debug option (gitDiff: true) can be useful for a visualization of how exactly Mull whitelists or blacklists found source lines.

Typical use cases

Under the hood, Mull runs git diff from a project’s root folder. There are at least three reasonable options for using the gitDiffRef argument:

1. gitDiffRef: origin/main

   Mull is run from a branch with a few commits against a main branch such as main, master or equivalent. This is what you get from your branch when you simply do git diff origin/master. This way you can also test your branch if you have Mull running as part of your CI workflow.

2. gitDiffRef: . (unstaged), gitDiffRef: HEAD (unstaged + staged)

   Mull is run against a diff between the “unclean” tree state and your last commit. This use case is useful when you want to check your work-in-progress code with Mull before committing your changes.

3. gitDiffRef: COMMIT^!

   Mull is run against a diff of a specific commit (see also How can I see the changes in a Git commit?).

   Note that Mull always operates on the current working tree — it does not perform a git checkout to switch to the given commit’s state. This means the diff is used only to determine which lines to mutate, while the actual code being compiled and tested is whatever is currently checked out.

   This is safe and useful when the commit you’re pointing to touched files that haven’t changed since (e.g., re-analyzing an older fix in an otherwise stable module). However, if newer commits have modified the same files, Mull will analyze the current code using the old diff’s line ranges, which can produce misleading results. Prefer cases 1 or 2 when in doubt.