EMResearch
How to understand the statistics output file
Hello, I have question regarding the statistics file.
I have used evomaster to fuzz my simple application that has 3 endpoints. After the fuzzing was done, I checked my application log and there were many requests causing internal server error (500).
When I check the generated statistics file, it shows only 3 errors5xx and 3 distinct500Faults. Does this mean that even if there are many 500 errors on 1 endpoint, they are still counted as 1 error?
Here it is my statistics file content:
| Metric | Count |
|---|---|
| evaluatedTests | 167 |
| individualsWithSqlFailedWhere | 0 |
| evaluatedActions | 892 |
| elapsedSeconds | 302 |
| generatedTests | 4 |
| generatedTestTotalSize | 4 |
| coveredTargets | 14 |
| lastActionImprovement | 430 |
| distinctActions | 3 |
| endpoints | 3 |
| covered2xx | 1 |
| gqlNoErrors | 0 |
| gqlErrors | 0 |
| gqlErrorsPerLines | 0 |
| errors5xx | 3 |
| distinct500Faults | 3 |
| failedOracleExpectations | 0 |
| potentialFaults | 6 |
Hi,
from Statistics.kt:
// errors5xx - counting only the number of endpoints with 5xx, and NOT last executed line
add(Pair("errors5xx", "" + errors5xx(solution)))
//distinct500Faults - counts the number of 500 (and NOT the other in 5xx), per endpoint, and distinct based on the last
//executed line
add(Pair("distinct500Faults", "" + solution.overall.potential500Faults(idMapper).size ))
I am sorry, it is still not clear to me.
Just to be sure, so evomaster only counts the number of endpoints that have 500 error and doesn't count the number of unique requests/payloads that cause 500 error?
Or does distinct500Faults actually describe the number of unique requests/payloads that cause 500 error?