Is it normal that EvoMaster combined "sucess" and "other" tests into "fault_representatives"?

[aruvic]

Hi,

Is it normal that EvoMaster combined "success" and "other" tests into "fault_representatives"?

After running for 14 hours EvoMaster finished with following stats

* Recomputing full coverage for 39 tests
* Going to save 32 tests to 
* Potential faults: 6
* Bytecode line coverage: 54% (213 out of 393 in 14 units/classes)
* Successfully executed (HTTP code 2xx) 7 endpoints out of 7 (100%)

It generated

• 16 tests in EvoMaster_successes_Test.java and
• 16 tests in EvoMaster_others_Test.java

and combined all tests (32) into EvoMaster_fault_representatives_Test.java

Is my understanding correct that:

• EvoMaster_successes_Test are sunny day scenarios
• EvoMaster_others_Test are failures which are gracefully handled by the REST API
• and EvoMaster_fault_representatives_Test should be the potential faults

Where and how can I find the "Potential faults: 6"? if EvoMaster_fault_representatives_Test contains 32 tests?

Please find attached also the statistics.csv and the tests generated: statistics.csv test_output.zip

Br, Alen

[arcuri82]

Hi @aruvic ,

many thanks for taking the time to report these bugs. What you report here is indeed a bug.

fault_representatives is a subset of faults. as you had no faults file, then there should be no fault_representatives either. however, if there is Potential faults: 6, then you should have had a faults file with between 1 and 6 tests (as a test could detect more than 1 fault).

why successes and others are merged into fault_representatives makes no sense whatsoever... i m aghast. I didn't write that part of the code, i ll have to look into it...

[arcuri82]

hi @aruvic this issue should now had been resolved in the master branch

[aruvic]

hi @aruvic this issue should now had been resolved in the master branch

thanks, I will use and test it