testaro icon indicating copy to clipboard operation
testaro copied to clipboard

Refine Tool Report Standardization

Open bbertucc opened this issue 11 months ago • 2 comments

If testaro is creating a standard format for automated scan reports, I would suggest a few changes to the formatting, expressed in README.md Standard Format section.

Comments:

  • pages: How are pages expressed in results?
  • "totals" could be gathered after processing results. to my knowledge scans don't output totals when scanning, and so this also would not need to be a standard output.
  • "what" is fairly abstract. Perhaps this could be replaced with something like "description"
  • "ordinalSeverity" - I would recommend expressing severity as a tag since, as you mentioned, tools report this factor in different ways. After the fact processing can be used to understand statistics around severity.
  • "tagName" - This would be able to be looked up via any HTML associated with the instance, and so I don't see a reason for standardizing the inclusion of it.
  • How would relatedElements from axe fit this schema?
  • Where would links to additional information go?

Additionally, I don't know if Testaro is the best vehicle for standardizing reports. The value I see is in validating reported issues. Wouldn't it be of more use (and less headache) to start from a single format? I started to lay out that idea here: https://github.com/EqualifyEverything/equalify-formatter/tree/main

Curious to hear your thoughts @jrpool

bbertucc avatar Feb 29 '24 15:02 bbertucc