David Spickett

Notes to reviewers: * This is based on https://github.com/llvm/llvm-project/pull/113160. * The first commit is that previous PR. * The second commit is the new script. * The third commit is...

Example reports can be seen here - https://buildkite.com/llvm-project/github-pull-requests/builds/112660 (make sure to switch to "All" instead of "Failures")

> Basically, the sort of "plain text" report that gets added is the difference. Is that correct? Yes, there is also a Linux report there that shows that it succeeded,...

> Do we make any guarantees about supporting lit flags in the future? It's mostly an internal tool, and I would personally be in support of removing flags that are...

The version that does not use the lit feature: https://github.com/llvm/llvm-project/pull/113660 It's not complete but it's good enough to compare approaches I think. Another, probably objectionable choice, is to write a...

I tried a few more things, so I will summarise everything up until this point. There are two layers here: 1. Making test reports. 2. Producing the XML test results....

I also looked through GitHub code search to see if anyone else was hitting this same issue. ROCm avoids it by splitting each check target into its own step: https://github.com/ROCm/ROCm/blob/bce439ecacc86b7f48934f5cc6be7ce2044412cf/.azuredevops/components/llvm-project.yml#L87...

> FWIW, I didn't mean to derail this effort or create significant additional complexity by saying what I said above. My intention was only to point out the tradeoff of...

I have updated this PR to include marking the option as hidden. If folks would be ok with that approach, I will break this up into individual parts for review...

I've split up the code: Hide the lit option - https://github.com/llvm/llvm-project/pull/114812 Write results to unique file names - https://github.com/llvm/llvm-project/pull/113160 This PR, which adds the reporting script.