Build `nixos-rebuild`'s `--target-host` and `--build-host` into Nix.

[Atemu]

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Remote builds require a ton of setup, come with incredibly unhelpful error messages, are generally not very nice to work with and seem to be tailored towards build farms, not interactive use, making them inefficient to use.

For example, I often find myself in a situation where I want to instantiate locally and then realise on a remote machine; without copying sources or output paths between them. Nix' remote builders are not able to cover that use-case.

Describe the solution you'd like A clear and concise description of what you want to happen.

nixos-rebuild has --target-host and --build-host options that allow you to specify which host the output path should end up on (target host) and which one should realise it (build host).

It achieves this through a relatively simple script:

https://github.com/NixOS/nixpkgs/blob/d6417bd3069397ee3a7bf20a7704c0d7325b39f7/pkgs/os-specific/linux/nixos-rebuild/nixos-rebuild.sh#L174-L190

This actually works very well! I want to be able to do the same with regular nix and nix- commands.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

The default for both options should be the local machine. If I wanted to realise on a powerful machine and then copy the paths back instead of realising locally, I should be able to achieve that with --build-host powerful.local.
Nix should then make the remote daemon download sources and execute all builds and then copy the closure(s) back. (This would fail of course if they lacked a trusted signature.)

Alternatively, I should be able to run the build on the remote machine and not copy the paths back by specifying --build-host powerful.local --target-host powerful.local. Eval would still happen locally but only derivations are copied, no sources or output-paths.

IFD would probably need special handling; the derivations required for eval either need to be built with --build-host powerful.local (so, built remotely and copied back) or built locally. I'd say since you want everything to be realised on the build-host, it that should happen on the build-host by default with paths being copied.

No idea how Flakes would play into this but probably not at all as they're only concerned with instantiation which still happens locally (modulo IFD).

An open question is how to handle the realisation of paths that are present on the eval machine but not present on the build machine. Should it copy them over (might fail due to signatures?) or realise them again?

[ElvishJerricco]

At least --build-host seems to be covered by --store ssh-ng://builder. And --target-host is at least easy to do manually with nix copy --from ssh-ng://builder --to ssh-ng://target

[Atemu]

At least --build-host seems to be covered by --store ssh-ng://builder. And --target-host is at least easy to do manually with nix copy --from ssh-ng://builder --to ssh-ng://target

Good points and thanks for the pointers!

It'd still be nice to have one integrated reliable discoverable standard solution rather than everyone needing to script around it.

[ImBearChild]

reliable discoverable standard

Well, at least there is a low-quality hack for ssh-ng. You can replace your nixos-rebuild with following line: sed "s/ssh:\/\//ssh-ng:\/\//g" $(which nixos-rebuild) | bash -s -- <options for nixos-rebuild>.