niv icon indicating copy to clipboard operation
niv copied to clipboard

Do not use builtin fetchers by default

Open nmattia opened this issue 4 years ago • 6 comments

Only nixpkgs should be fetched with the builtin fetchers by default.

nmattia avatar Feb 10 '20 10:02 nmattia

What's the reason behind this?

Also btw, I recently got it working to even fetch nixpkgs itself without any fetchTarball (or other eval-time fetchers):

let
  fetched = import <nix/fetchurl.nix> {
    url = "https://github.com/NixOS/nixpkgs/tarball/e94a84a144b83eebfcfb33ac3315c01d0d4b3a0a";
    hash = "sha256:0s94ygwgkzk1i4gafc51i10j99yc2qba98mg5lb89nsg24xm31lp";
  };
  config = import <nix/config.nix>;
  nixpkgs = derivation {
    name = "nixpkgs";
    system = builtins.currentSystem;
    builder = config.shell;
    args = [ "-c" "${config.gzip} -d < ${fetched} | ${config.tar} -xf - --one-top-level=$out --strip-components=1" ];
    chrootDeps = config.chrootDeps;
  };
in import nixpkgs {}

infinisil avatar Mar 05 '20 17:03 infinisil

It would be possible to use builtins.fetchTree, if it is available. Although, that appears to need a hash in SRI format, which can't obviously be obtained from the existing hashes in sources.json without something like toHash.

tomprince avatar Aug 26 '21 22:08 tomprince

What's the reason behind this?

I’d be curious to know this as well. I though the general consensus was that for new code builtin fetchers should be preferred over ones based on fixed-output derivation.

adrian-gierakowski avatar Aug 26 '21 23:08 adrian-gierakowski

The reason I'm interested in it is that the ones from pkgs will use the cached content without re-downloading, if the directory is in the store. At least builtins.fetchTarball will always query the URL, and potentially redownload. I ran into this, when sharing a nix-store between multiple docker containers that didn't share ~/.cache.nix, so each container would try to download all the files, even if they were already present.

tomprince avatar Aug 27 '21 00:08 tomprince

builtins.fetchTree does look at things in the store, which is why I suggested it as a possibility.

tomprince avatar Aug 27 '21 00:08 tomprince

What's the reason behind this?

I don't remember for sure, but I think this had to do with restricted eval (where every single URL has to be whitelisted when using builtin fetchers).

nmattia avatar Sep 06 '21 08:09 nmattia