infra

infra copied to clipboard

Is your feature request related to a problem? Please describe.

At my work we often use Armv7 devices like Raspberry Pi. Often my colleagues complain about issues with outdated packages in the default Debian systems these devices use by default and the lack of declarative management of their deployments. Naturally Nix is the answer. However, these devices usually have low memory & CPU power, making it hard to build full systems on them.

Describe the solution you'd like

Taking https://github.com/NixOS/nixos-org-configurations/pull/142 as an example, is it that simple and legitimate to add a channel for an architecture? Having an armv7 channel building and caching armv7 builds would enable me to dramatically change the experience with such devices.

Describe alternatives you've considered

nixbuild seems like an awesome service that could benefit teams such as those I occupy. However, I still think that a tier 1 support for armv7 could motivate many big software companies to use Nix, see e.g this. Also, nixbuild's pricing is the fairest I could have imagined, but as a junior in a non Nix team, it's hard to convince people to use such a service which even myself is not experienced with.

Additional context

https://github.com/NixOS/nixpkgs/issues/83049

The main problem is having build machine to build for armv7l.

Most AArch64 providers only have machines supporting 64 bit systems.

Furthermore, the builds have to happen on an armv7l kernel, not AArch64 since the personality subsystems in Linux are not as robust for ARM, and ambiant impurities leaks into the build. So for the few systems allowing 32 bit execution, we need to "burn" the virtualisation capabilities by making armv7l VMs on the AArch64 host.

There has been some exploratory work for this earlier, I don't know if there was anything conclusions coming from that or if it was simply not completed due to lack of people-power to work on that.

The main problem is having build machine to build for armv7l.

So is this a problem of merely finding a provider for armv7l? Perhaps the provider of nixbuild.net would fit for nixos.org?

Mainly yes. It depends how nixbuild.net does it. If they simply flipped the option to run 32 bit on aarch64 systems there's always some possibility of getting ambiant impurities in. (Though less so than doing the same with armv6l in mind.)

That is for the technical aspect.

Though there might be other roadblocks that are more organizational and social to get it going, but nothing major I hope. One of these is supporting the architecture. Though tbf it can't happen without first getting good regular build coverage.

nixbuild.net uses aarch64 kernel with personality patch: https://discourse.nixos.org/t/support-for-aarch64-linux-armv7l-linux-and-i686-linux-is-now-generally-available-on-nixbuild-net/21331/3 I'm also using this approach and regularly fix issues. There's issues with ambiant impurities (for example, patchelf crashes with SIGBUS error when building go-bootstrap which is not reproducible when building on armv7l-only machine), but these should be easy to spot and fix once armv7l-linux is added to the hydra. It's a lot less work than cross-compiling IMO.

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/extend-cross-compilation-support-in-hydra/26783/1