nixos-generators icon indicating copy to clipboard operation
nixos-generators copied to clipboard

GitHub Actions image format builds

Open tomeon opened this issue 2 years ago • 5 comments

Re this comment.

This PR introduces:

  1. Nix flake check outputs for x86_64-linux (almost everything but the sd-aarch64 and sd-aarch64-installer formats) and aarch64-linux (only the sd-aarch64 and sd-aarch64-installer formats), and
  2. A set of GitHub Actions jobs that (a) evaluate the flake checks, (b) attempt to build various formats using nixos-generate with appropriate arguments, and (c) attempt to build the Nix flake check outputs.

Submitting as a draft for feedback on scope, design, and plain old fitness for purpose before proceeding further.

Link to a passing GitHub actions run is here (66th time's the charm :smiling_face_with_tear:).

tomeon avatar May 21 '22 19:05 tomeon

TODO

When to run

Building (almost) all formats is time-consuming, even when running all the builds in parallel. Maybe these jobs should run only upon pull request activity, and not upon every push?

Build artifacts

The GitLab CI setup included exposing the results of the vm and vm-nogui builds as artifacts. I've ported this over to the updated GitHub Actions workflow. Not sure what value, if any, this provides.

Caching

Would it be worthwhile to cache build results? With, say, Cachix?

Useless work

Some jobs (e.g., anything involving the cloudstack format) do a bunch of work -- checking out the repo, installing Nix, etc. -- only to exit without attempting to build anything because the Nix flake doesn't define .#checks.<system>.cloudstack-<nixpkgs>. This is wasteful.

This useless work could be cut out by duplicating some of the format-exclusion logic from the Nix flake check definitions as exclude entries in the relevant job matrix.

Complexity

The flake check stuff is pretty hairy. Would welcome feedback on improving the architecture.

tomeon avatar May 21 '22 19:05 tomeon

Btw. we have also have a self-hosted hercules ci setup in this organisation, which does not need to download all derivations. This might be faster than github actions. There is also an aarch64 builder included for cross-ci builds.

Mic92 avatar Jun 10 '22 07:06 Mic92

generally this goes in the right direction. I guess we want to build just 3-4 different images on every push (difference between push or PR is not big enough to matter) I guess we should build something like kexec-bundle, container, amazon, iso ? since those are the most commonly used I would guess

Lassulus avatar Jun 10 '22 08:06 Lassulus

Could you maybe move some of your commits not related to CI to a new PR? i.e. https://github.com/nix-community/nixos-generators/pull/155/commits/45504ad7f9aa220fa5dbad661798cead3e2fcd8a looks like no-brainer that we can just merge as is. Than reviewing this will become faster.

Mic92 avatar Jun 10 '22 08:06 Mic92

Could you maybe move some of your commits not related to CI to a new PR?

Done, done, and done :+1:

tomeon avatar Jun 17 '22 03:06 tomeon