How to obtain full stacktrace with defects4j test?

[danielding110]

Question

When I run defects4j test, the console output lists only the class and method names of failing tests (e.g.,

Failing tests: 10
  - org.apache.commons.compress.archivers.jar.JarArchiveOutputStreamTest::testJarMarker
  - ...

), whereas the complete stack traces are stored separately under the project’s trigger_tests directory.

Is there a way to invoke defects4j test so that it prints the full stack trace for each failing test directly to stdout/stderr, instead of just the test identifiers?

The only method I’ve found to obtain full traces programmatically is to run JUnit manually, e.g.:

CP=$(defects4j export -p cp.test)
java -cp "$CP" org.junit.runner.JUnitCore <TestClass>

Can the same detailed output be generated via a built-in Defects4J command or flag?

[ramsey-coding]

@rjust that's a good point from @danielding110.

Do you know the solution or workaround to it?

[rjust]

Hi @danielding110,

After running defects4j test the detailed test failures are available in <working directory>/failing_tests. This file uses the same format as the ones in the trigger_tests directory.

For example:

defects4j checkout -p Lang -v21b -w /tmp/Lang-21
defects4j test -w /tmp/Lang-21
cat /tmp/Lang-21/failing_tests

Output:

--- org.apache.commons.lang3.time.DateUtilsTest::testIsSameLocalTime_Cal
junit.framework.AssertionFailedError: LANG-677
	at junit.framework.Assert.fail(Assert.java:57)
	at junit.framework.Assert.assertTrue(Assert.java:22)
	at junit.framework.Assert.assertFalse(Assert.java:39)
	at junit.framework.TestCase.assertFalse(TestCase.java:210)
	at org.apache.commons.lang3.time.DateUtilsTest.testIsSameLocalTime_Cal(DateUtilsTest.java:237)
[...]

Is this sufficient for your use case?