buildkit icon indicating copy to clipboard operation
buildkit copied to clipboard

ci: test report summary output

Open crazy-max opened this issue 3 years ago • 1 comments

Aggregate data from JSON reports of Go tests using gotestsum and adds a step for unit and integration tests that will render the output as markdown and create a GitHub summary in our workflow using teststat:

image

Also some useful stats about slow tests:

image

Was also thinking using tparse to have a better overview: https://github.com/crazy-max/buildkit/actions/runs/2453700879#summary-6772166685

image

But output is quite huge.

@thaJeztah Maybe gotestsum could also generate a markdown summary output so we don't need extra tools. WDYT?

crazy-max avatar Jun 07 '22 11:06 crazy-max

As discussed with @tonistiigi, resulting summary might not be useful in the current state. What would be interesting is to capture popular failure points over time to catch performance regressions. Without dedicated nodes it looks difficult to catch atm.

crazy-max avatar Jul 30 '22 22:07 crazy-max