fault-localization

What is this good for?

Not all failing code raises exceptions; behavioral bugs can be the hardest to diagnose. fault-localization is a pytest plugin that helps you identify and isolate the lines of code most likely to be causing test failure, using the simple rule-of-thumb that the most suspicious code is the code run most often in failing tests. Don't just rely on your tests to catch bugs - use them to pinpoint bugs.

Installation

pip install fault-localization

Usage

With fault-localization installed, running

pytest --localize {dir} [--n-hotspots {n}] [pytest args ...]

will highlight suspicious lines encountered within dir while running the test configuration specified. --n-hotspots can optionally be provided to show the top n most suspicious lines (only one line, with surrounding context, is shown by default).

If you suspect multiple sources of failure, or if there are multiple tests within your suite that exercise the area of code you're interested in, using pytest's -k flag is useful for running fault localization on only a subset of your suite.

Fault localization, as a technique, works best when areas of your codebase are exercised repeatedly across a bevy of differing cases and values. That's why fault-localization works with Python's premiere property-based testing framework, Hypothesis, out of the box - which does just that.

Contacts

• Name: H. Chase Stevens
• Twitter: @hchasestevens