aqa-test-tools
aqa-test-tools copied to clipboard
Enhancement: Include listing of test case failures within the test target when it is available
We currently track which test targets fail and keep a snippet of console output per target in the TRSS database.
For certain groups of tests like openjdk and jck we can summarize the list of testcase failures in the TAP file (and/or to console).
Currently, we have to inspect the console output to see if a particular target is failing with the same underlying set of test cases or not from run to run and across platforms.
We should consider pushing the summarized list of testcase failures to console (at the same time we are writing the list to the TAP file, see these related PRs in TKG). Then grabbing that console output and storing it in the TRSS database for querying and comparison between results in the GUI.
This may help us hone in on when root causes for certain failures are introduced.
We should consider pushing the summarized list of testcase failures to console
This should be done in TKG. https://github.com/adoptium/TKG/issues/536