deploy-rs icon indicating copy to clipboard operation
deploy-rs copied to clipboard

Build on target server?

Open Mic92 opened this issue 3 years ago • 10 comments

Is it possible to build the flake on the targets server instead of my laptop? I don't want to upload big build closures to my machines all the time.

Mic92 avatar Nov 25 '20 19:11 Mic92

I suspect this would also help with the separation between target/deploying system as in #13! Very interested in this, especially for machines that are more powerful than distributed builders / my laptop.

antifuchs avatar Nov 25 '20 22:11 antifuchs

Not yet, but this is definitely on our roadmap.

balsoft avatar Nov 25 '20 22:11 balsoft

Ok. Apart from that the feature set looks interesting. If someone else is looking for this feature as well: Krops does it by default and can be extended to support flakes: https://github.com/Mic92/dotfiles/blob/master/nixos/krops.nix It is also possible to have a krops running the deploy on a remote machine and push the build nixos closure to a another machine.

Mic92 avatar Nov 26 '20 07:11 Mic92

Last but not least, it would also enable darwin users to use this!

happysalada avatar Mar 16 '21 03:03 happysalada

Also very interested in this.

It would be nice to not need Nix installed at all on eg CI servers that deploy.

theduke avatar Apr 13 '21 08:04 theduke

https://github.com/arnarg/pushnix supports this feature.

Mic92 avatar Apr 13 '21 08:04 Mic92

Here is my workaround. I set up a git bare repo on my server. Add a hook to copy the committed nix config to the /etc/nixos folder and trigger a rebuild in the hook. (template for setting up a bare repo with a push hook This is basically what pushnix is doing.

happysalada avatar Apr 13 '21 10:04 happysalada

ssh-ng could help here. NIX_REMOTE=ssh-ng://myserver nix build .#pkg will evaluate .#pkg locally, send the derivation (and its dependency derivations) to myserver and build it there without copying the result back to the local store. Seems to be perfect for this use case.

deploy-rs theoretically supports this today with NIX_REMOTE=ssh-ng://myserver deploy -s. Problems: For some reason the call to nix eval causes evaluation to happen on the remote side, which will probably immediately fail because it can't find the flake's source path (nix bug?). If the flake's source path does exist on the remote, the eval step works but takes an eternity to complete. It then fails when trying to push the system closure to the remote, because the path is never realized locally.

TL;DR set NIX_REMOTE=ssh-ng://{deployment user}@{target hostname} for the build step, skip the copy/push step.

max-privatevoid avatar Nov 25 '21 20:11 max-privatevoid

So, I've tried to build it via -- --store somewhere. It kind of works, but you will need to copy some paths it will complain about back, and I only have tested it with single target server.

cab404 avatar Nov 25 '21 23:11 cab404

I am using pyinvoke now with a custom task that uploads configuration with rsync and than just run nixos-rebuild in parallel for all my hosts:

https://github.com/nix-community/infra#deployment-commands https://github.com/nix-community/infra/blob/a3dd018c629edfa16edf71901b77112e7de282ca/tasks.py#L17

Mic92 avatar Nov 26 '21 07:11 Mic92