dotnet
Only write test output in GitHub actions for test failures
Integration tests have recently moved to GitHub actions. All tests include their output in the console logs. That makes it hard to see details about failures, which are what you really care about.
The test runner should be configured to only include test output in console logs for failures.
I did this in grpc/grpc-dotnet repo, which also uses GH actions, and found it made using console logs much easier. Unfortunatly that repo uses nunit, not xunit, for tests so we'll need to figure out how to configure it in dotnet/aspire.
cc @davidfowl
Just FYI you can see failures in a summary instead of sifting through the console logs.
That's useful to see the failing test and exception message. Is there a way to view that test's logs from the summary?
In grpc/grpc-dotnet I found only showing failed test's output in GH actions console helpful for quick debugging.
Hmm I don't see any options for that on https://github.com/Tyrrrz/GitHubActionsTestLogger. The TRX file does have all tests/failing and passing and the stdout per test, so the data is available...
Aren't trx and that summary logger for different purposes?
There is a console logger:
Is that what write's the console output for GHA? Its verbosity setting might impact what is written.
I'm experimenting here yeah https://github.com/dotnet/aspire/pull/7460. Trying to see if we can still capture everything and not bloat the run.
I think there are two issues here that needs addressing:
- restrict the verbosity of the console logs and move to the logging model used by the Arcade (where the output is captured in a log file), so that the logs can be downloaded and inspected locally (fed to an LLM).
- do not upload logs for successful tests.
The outerloop workflow is doing both. It is something I've been thinking to roll out to other test runs.
@RussKie is this on track? can you mark in progress for 9.3 if appropriate?
It's covered in https://github.com/dotnet/aspire/pull/8811, waiting on @radical's review.
No, we still need to switch to the runsheet-based test workflows. I'm working with @radical to get us there.