defects4j
defects4j copied to clipboard
Replace the regex expression to remove all string/character and comments of failing test methods
The framework/util/rm_broken_tests.pl
script parses the output of a test suite compilation/execution and fixes failing test methods by replacing each broken test method with a dummy test method in the source file of the corresponding test class. For instance, supposing the following test suite with a failing test method (test1):
public class TestFoo {
@Test
public void test0() {
assertTrue(true);
}
@Test
public void test1() {
String str = new String();
str = 123456789;
}
}
is then transformed to:
public class TestFoo {
@Test
public void test0() {
assertTrue(true);
}
@Test
public void test1() {}
// Defects4J: flaky method
// @Test
// public void test1() {
// String str = new String();
// str = 123456789;
// }
}
Basically, the script parses the compilation/execution output and identifies the line where the failing test method starts/ends and it uses a regex expression to remove all string/character and comments of the failing test method as they may contain unbalanced delimiters or brackets (which may cause problems to find the last }
of the test method). Although the script has been working fine in general, two issues (#96 and #184) have been reported related to that regex expression. Perhaps a more robust solution would be to build a parser that computes a line-number table for all methods and then use that table to comment out failing test methods.