serokell
Keep a registry of profiles
How about having a directory profiles with symlinks to deployed profiles for each node, e.g. web-system? Basically very similar to Nix profiles, with generations as well.
This would allow checking nix diff-closures with the previous profile so you can verify the change before deploying.
It would also allow doing build, push and activate in separate steps without having to evaluate the configuration each time. This means that related systems can be activated very close to each other.
I'm against having it as state. One of the nicest properties of deploy-rs compared to nixops is that it's "stateless" (as in, all the state is stored on the targets). However, having it as a sort of evaluation cache (similar to what Nix itself does) would be fine by me.
Ah yes indeed good point - so actually my problem is that deploy-rs is a bit slow?
When I run nix build multiple times on the system configuration, it takes 2 seconds, but deploy -s --dry-instantiate takes 11 seconds. For a lot of hosts, that adds up.
I ran it with strace -f -e trace=execve and it seems that the longest time is spent before it outputs "the following profiles are going to be deployed:". It creates about 5465 processes (or threads?) after calling git rev-parse and before outputting that line. The calls up to and including the git rev-parse take less than 0.5s.
Is that expected?
So, ideally, there'd be a --build-only flag that evals and builds, to verify that everything builds before pushing, and then it would be nice if --dry-instantiate also outputs nix diff-closures after pushing.
I just realized that I always deploy a single host at a time. I tried deploy . and it kills my system because nix tries to eval all the nodes at the same time. But I think that's a separate issue, that could be solved by eval-ing the nodes one by one.
I tried looking at the source but I can't figure out what is doing all those forks or thread creations. I do know that the nix and git interactions are fine (apart from evaling the whole fleet), it's something in deploy-rs being slow.