defects4j icon indicating copy to clipboard operation
defects4j copied to clipboard

How to get the coverage of test cases at classes granularity?

Open gouxiaodong opened this issue 3 years ago • 2 comments

Hello, sir I'm trying to get the coverage of test cases at classes granularity (i.e., columns are classes and rows are test cases). Therefore, I tried the following work (taking lang-1b as an example): (1) First, I used Understand (a static analysis tool) to get all classes of Lang 1b, named 'all_calsses.txt'; (2) I used the [i] option of coverage command to get coverage of 'all_calsses.txt'. However, I got this error: % defects4j coverage -i all_classes.txt Running ant (compile.tests)................................................ OK Running ant (coverage.instrument).......................................... OK Running ant (run.dev.tests)................................................ OK Running ant (coverage.report).............................................. OK Lines total: 0 Lines covered: 0 Conditions total: 0 Conditions covered: Illegal division by zero at /Users/user/defects4j/framework/bin/d4j/d4j-coverage line 168. Compilation failed in require at /user/defects4j/framework/bin/defects4j line 195.

I tried to find the reason for this error, erhaps because there was an error in "all_classes.txt" ? For example, some classes in "all_classes. txt" are not real classes in d4j?

The attachment is my "all_classes. txt" . Can you give some suggestions? @rjust

all_classes.txt

gouxiaodong avatar Dec 25 '21 10:12 gouxiaodong

Hi @gouxiaodong,

The shared all_classes.txt file indeed includes a number of invalid entries (e.g., org.apache.commons.lang3.ArrayUtils.add.T); these should be removed. Perhaps this invalid output points to a bug in the used static analysis tool?

The reason for the division-by-zero error is that there are 0 instrumented lines, which is unexpected. Please let us know if this actually happens for a class that does exist in the source code. If so, Defects4J should handle this case (as opposed to assuming 1+ instrumented lines).

Thanks, René

rjust avatar Dec 28 '21 04:12 rjust

Thank you for your attention, @rjust, I will try it again and give you feedback in time once the reason is confirmed.

gouxiaodong avatar Dec 30 '21 08:12 gouxiaodong