nixie icon indicating copy to clipboard operation
nixie copied to clipboard

Pull resources from NAR directly

Open Thesola10 opened this issue 9 months ago • 1 comments

Right now the static-bins output just crossed 100MB uncompressed, which means it now has the honor of slamming into cachix/cachix#619. (UPDATE: they kindly pushed back the size limit https://github.com/cachix/cachix/issues/619#issuecomment-2697041912, but the fragility of Nixie's current approach is still evident)

We need a simpler, more Nix-y way to retrieve files from within the derivation. My idea now that #16 has been merged, is to try and write a streamed NAR parser in Amber, which would in turn provide exact offsets to download a file from within the NAR, using minimal bandwidth (and therefore memory) to perform the lookup in the first place.

The ideal implementation would be a Jura package, which implies a couple fixes to upstream projects first:

  • [ ] amber-lang/amber#681
  • [ ] lens0021/jura#4
  • [ ] lens0021/jura#10
  • [ ] Port Jura to use AMBER_PATH

Thesola10 avatar Feb 27 '25 15:02 Thesola10

Well this is annoying... Cachix only supports zstd and xz as compression methods. zstd is completely out of the question, and xz isn't much better (not preinstalled on macOS for instance).

I really don't want to give up on the Nix cache approach since it ensures any version of the script can handle its resources format, so I'll have to look into self-hosting a Harmonia cache or something.

I'll keep working on amber-nar in the mean time.

... then again the static xz executable is tiny... and the static-bins derivation is only needed on supported platforms...

Thesola10 avatar Mar 01 '25 14:03 Thesola10