Record test suit evolution

[andrewbwogi]

Characteristics

• Issue Type: feature
• Reporter: Andrew Bwogi, [email protected]

Description

In discussions by @bbaudry and @monperrus there is an idea about creating a program that would record the evolution of a test suite. The program would run in the background of the recorded application, for instance as a maven plugin, and collect data such as:

• In response to what program changes are tests added?
• What is the state of the project when a particular kind of test is added?
• How does the test suite grow in relation to itself?
• How does each test perform over time as the program matures?

This data could serve as a heuristic for how tests can be automatically amplified or perhaps used by DSpot for more directed input modification and assertion generation.

[danglotb]

Hello @andrewbwogi

Looks great, but concretely, what are the data collected? The test code diffs between version?

Questions:

• How can we classify the program changes?
• What do you mean by state of the project?
• What does perform means?

What could be also interesting is to collect:

1. the inputs data, i.e. literals, objects and method call used.
2. the state of the program at different step: at the initialization, after methods call, etc.

For this, we can reuse the code in ObjectLog and adapt a bit the Instrumentation done by DSpot.

[danglotb]

Another kind of Test Evolution:Understanding myths and realities of test-suite evolution

And the tool developed: TestEvol.