[Bug]: devbox hangs after downloading nixpkgs registry

[dudicoco]

Current Behavior (bug) When running devbox install, if the nix packages registry needs to be downloaded, devbox hangs for a few minutes after downloading the registry:

$ devbox install
Ensuring packages are installed.
Ensuring nixpkgs registry is downloaded.
[33.2 MiB DL]
*hangs here*

Since nothing is printed to the screen at this point it is unclear why it hangs.

Expected Behavior (fix) Devbox should either print to the screen what it is doing at that point if this is expected behavior, or it should not hang after the registry has been downloaded.

Additional context

$ devbox version       
0.5.13

{
  "packages": [
    "[email protected]"
  ]
}

[Lagoja]

I think some of this will be resolved in an upcoming release where we plan to eliminate the need to download the nixpkgs registry, but we should print more information on what Devbox or Nix is doing during these delays.

[Almenon]

@Lagoja do you know when the upcoming release is planned? Curious what the timeline is on that, and thanks for helping make devbox!

[Almenon]

I'm running into a issue where devbox appears to be installing the nixpkgs registry repeatedly, one time for each devbox add, instead of caching it and reusing the cache. Although if there will no longer be a need to download the registry that issue becomes moot.

[mjeffryes]

I'm experiencing something similar to @Almenon as well.

[savil]

@Almenon @mjeffryes sorry about that. I think this happens when different packages you've installed are sourced from different nixpkgs commits. Each of those is treated as a separate instance of the nixpkgs registry and gets downloaded separately by nix.

We have an update coming out soon, hopefully later this week. That should make this issue moot.

[dudicoco]

@Lagoja I've just tested devbox 0.8.3 and the issue is still there. devbox hangs but seems to be for a shorter time than before, and still no console output on what it is currently doing.

[savil]

@dudicoco the limitation in this case is that we are calling into a nix command which happens to download the nixpkgs as a side-effect. Unfortunately, nix itself doesn't print any further information that we can display.

[dudicoco]

@savil can you please elaborate? it's hanging after the DL prompt is finished - [33.2 MiB DL]

[dudicoco]

Hi @savil, any updates?

[savil]

@dudicoco sorry I'd missed your question on Dec 10.

To elaborate a bit, the underlying issue is that when nix executes some command using a package from a version of nixpkgs that your system doesn't currently have, then it needs to download that version of nixpkgs in order to fully resolve that package and have it be installed. This nixpkgs download is fairly large (i forget the size offhand) but on my relatively fast internet and macbook pro takes about 30-60 seconds to download and be evaluated. This is why there's this long pause you see.

Now, unfortunately, I haven't come across an easy way of showing more print-output from nix during that pause.

To clarify: the pause itself is happening from within nix, and not something we can directly "fix" from within Devbox. At best, we can find better practices to use nix to minimize the need to experience this pause.

Late last year, in Devbox, we built machinery to bypass this nixpkgs download for most packages. It will use the Devbox search index to get the nix-store-paths of the package's binaries and libraries and directly download them from the nixos-cache. This means we can entirely avoid the nixpkgs download and evaluation, and skip this long pause for the vast majority of packages.

You can see some of these store-paths by inspecting the devbox.lock file.

Hope this helps?

[dudicoco]

Thanks for the reply @savil.

I've tested the installation of a few packages with the latest devbox version and all of them hung:

    "ripgrep@latest",
    "fd@latest",
    "bat@latest",
    "glow@latest",
    "eza@latest"

How can I know which packages have the feature which skips the nixpkgs download?

In addition, can you please instruct me on how to recreate the hanging issue with plain Nix?

[savil]

@dudicoco 🤔 those should be pretty fast. Which version of nix are you on? This feature leverages some new nix machinery introduced in nix 2.17, so I think we preserve the slower behavior for users on older nix versions.

[dudicoco]

@savil I've uninstalled devbox and nix and reinstalled devbox 0.10.1 and nix 2.21.2.

Seems like the issue has been resolved, thanks!