zig build: add option to only print failed steps

[Vexu]

The motivating case for this is that currently when a test fails the CI log will include ~5k lines of listing steps that succeeded.

[Vexu]

--summary failures and default:

$ zig build test
steps [3/8] zig build-exe test-runner Debug native... LLVM Emit Object... Test [1/235] standard-place
Test [34/235] initializers... 3 tests skipped
Test [75/235] #pragma pack msvc... 1 test skipped
Test [88/235] stringify invalid... 1 test skipped
Test [94/235] attributes... 1 test skipped
Testeps [6/8] run testrun test: error: 'test.destringify' failed: /home/vexu/Documents/zig/sfcc/src/Preprocessor.zig:2569:15: 0x44c888 in test.destringify (test)
    if (true) return error.HiAndrew;
              ^
run test: error: while executing test 'test.minSignedBits', the following test command failed:
/home/vexu/Documents/zig/sfcc/zig-cache/o/67672da2416b9636f08607564af18b08/test --listen=- 
Test [108/235] msvc boolean bitfield... 1 test skipped
Test [120/235] statement expressions... 1 test skipped
Test [129/235] declspec... 2 tests skipped
Test [131/235] standard attributes... 2 tests skipped
Test [145/235] imaginary constants... 1 test skipped
Test [165/235] invalid types... 7 tests skipped
Test [188/235] redefinitions... 1 test skipped
Test [199/235] #pragma pack... 1 test skipped
Test [218/235] constexpr... 1 test skipped
234 passed; 23 skipped.
Build Summary: 6/8 steps succeeded; 1 failed; 23/24 tests passed; 1 failed (disable with --summary none)
test transitive failure
└─ run test 23/24 passed, 1 failed
error: the following build command failed with exit code 1:
/home/vexu/Documents/zig/sfcc/zig-cache/o/ad9866c916aa0bf3d2b552837daaa3de/build /home/vexu/Documents/zig/zig/zig-out/bin/zig /home/vexu/Documents/zig/sfcc /home/vexu/Documents/zig/sfcc/zig-cache /home/vexu/.cache/zig test

--summary all:

$ zig build test --summary all
Test [34/235] initializers... 3 tests skipped
Test [75/235] #pragma pack msvc... 1 test skipped
Test [88/235] stringify invalid... 1 test skipped
Test [94/235] attributes... 1 test skipped
Test [108/235] msvc boolean bitfield... 1 test skipped
Test [120/235] statement expressions... 1 test skipped
Test [129/235] declspec... 2 tests skipped
Test [131/235] standard attributes... 2 tests skipped
Test [145/235] imaginary constants... 1 test skipped
Test [146/235] macro backtraTest [152/run test: error: 'test.destringify' failed: /home/vexu/Documents/zig/sfcc/src/Preprocessor.zig:2569:15: 0x44c888 in test.destringify (test)
    if (true) return error.HiAndrew;
              ^
run test: error: while executing test 'test.minSignedBits', the following test command failed:
/home/vexu/Documents/zig/sfcc/zig-cache/o/67672da2416b9636f08607564af18b08/test --listen=- 
Test [165/235] invalid types... 7 tests skipped
Test [188/235] redefinitions... 1 test skipped
Test [199/235] #pragma pack... 1 test skipped
Test [218/235] constexpr... 1 test skipped
234 passed; 23 skipped.
Build Summary: 6/8 steps succeeded; 1 failed; 23/24 tests passed; 1 failed
test transitive failure
├─ zig build-exe arocc Debug native cached 6ms MaxRSS:35M
│  └─ options cached
├─ run test 23/24 passed, 1 failed
│  └─ zig test Debug native cached 6ms MaxRSS:34M
└─ run test-runner success 1s MaxRSS:47M
   └─ zig build-exe test-runner Debug native cached 6ms MaxRSS:35M
      └─ options cached
error: the following build command failed with exit code 1:
/home/vexu/Documents/zig/sfcc/zig-cache/o/ad9866c916aa0bf3d2b552837daaa3de/build /home/vexu/Documents/zig/zig/zig-out/bin/zig /home/vexu/Documents/zig/sfcc /home/vexu/Documents/zig/sfcc/zig-cache /home/vexu/.cache/zig test --summary all

I think you can guess how --summary none will look like.

[Vexu]

For the change in Zig CI failure see https://github.com/ziglang/zig/actions/runs/5227592809/jobs/9439336808

[andrewrk]

Thanks, looks great!