NixOS
Close the gap between installer platforms generated by CI and release?
https://github.com/NixOS/nix/pull/4549 introduced Nix installer tests to the CI run.
The list of platforms that the installers are generated for is fine for the CI test (which is currently just covering x86_64-linux and x86_64-darwin), but there is a gap between this and the platforms hydra builds:
https://github.com/NixOS/nix/blob/e66550c917cc60113fa10db66b7b751ffe399214/flake.nix#L443-L444
A big benefit of generating installers in CI is being able to get timely, direct reporter feedback about whether a given PR fixes a given problem. (I recently opened #5951 to fix a bash compatibility problem some macOS installs are hitting. The reporter was eager to help out by giving the installer a go, but promptly ran into a hash-mismatch since they're on M1/aarch64-darwin.)
For now, just hoping to discuss whether it's tractable or what the best approach is...
Not certain who all to cc, but @NixOS/infra and @domenkozar seem like a start.
Problem is that it already takes 40 minutes or so to test a PR. Building the installer for all platforms (even if we could get that to work with GitHub actions) might take too long.
Yes. I agree it's not good to further bog down CI times for an occasionally-used feature (unless the mechanism is driving additional value beyond it).
I wasn't quite sure how to phrase it without muddying the point, but I don't have any specific solution in mind...
A bot command that generated a full installer and responded with invocation instructions when it was ready would be a few wins in one.
It wouldn't add much to the timing because those are separate platforms anyway and can be parallelized.
