feat: allow optional moving to nix store

[yajo]

According to https://github.com/NixOS/nix/issues/3121#issuecomment-1050510805, flake-compat should be that issue's workaround.

However, it tries to move src to the nix store always. That is a sane default, as it imitates closer what flakes do. However, it does not let this tool to be used comfortably as a workaround for that issue.

With this change, the default behaviour is preserved. However, the user can pass moveSrcToStore = false to avoid it.

@moduon MT-83

[edolstra]

flake-compat is intended to emulate the regular flake interface, so I'd prefer not to add functionality that isn't provided by flakes (like this moveSrcToStore flag). The problem is that people may come to depend on this extended interface, which would prevent them from switching to the new CLI.

[lovesegfault]

@edolstra That's totally fair IMO, but what is the solution here for people who want to use Flakes but have to deal with large repos?

It's really hard to sell Nix to my team if it means their disk is filled after a handful of builds.

I suppose I could go back to the "traditional" approach of using something like niv and ignoring flakes altogether, but that feels like a huge sacrifice. Do you have any better suggestions?

[yajo]

I've been testing these days by setting auto-optimise-store = true in nix.conf. I don't have measures to tell if it does the trick, but so far it seems to do it.

[edolstra]

@lovesegfault This will be addressed by https://github.com/NixOS/nix/issues/3121 (copying flakes to the store lazily). It will probably be hard to replicate that functionality in flake-compat though.

[lovesegfault]

@lovesegfault This will be addressed by NixOS/nix#3121 (copying flakes to the store lazily). It will probably be hard to replicate that functionality in flake-compat though.

That's a three-year-old issue, though. Do you have any expectation of when that could get done?

Just trying to figure out which direction to go.

[edolstra]

Hopefully in Nix 2.9, see https://github.com/edolstra/nix/tree/lazy-trees.

[sigprof]

One possible workaround is replacing { src = ./.; } with { src = { outPath = ./.;}; } in shell.nix — this disables source tree cleaning without adding an extra argument for that. Of course, this also disables the protection against referencing untracked files, but this might be acceptable for a compatibility mode.

Another workaround which is safer against accidentally referring to untracked files at the expense of needing to maintain an explicit list of files used by the flake (this might be acceptable if the flake is used only to provide the development environment):

let
  flakeManifest = [
    ./flake.lock
    ./flake.nix
  ];

  lock = builtins.fromJSON (builtins.readFile ./flake.lock);
  flake-compat = fetchTarball {
    url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
    sha256 = lock.nodes.flake-compat.locked.narHash;
  };

  src = builtins.path {
    path = ./.;
    name = "source";
    filter = path: type: builtins.any (x: (toString x) == path) flakeManifest;
  };
in
  (import flake-compat {inherit src;}).shellNix.default

This obviously still copies files listed in flakeManifest to the store, but at least that's not the whole repo. Although once you need to do something like nix flake update, you would again end up with the whole thing in the store until lazy trees are implemented.

[yajo]

Closing, as this will obviously not be merged, and the comments explained viable alternatives. Thanks everyone!