Better support for printing large values in `nix repl` and error messages

[9999years]

Is your feature request related to a problem? Please describe.

Currently, some error messages will print values in a "limited" fashion, after PRs like #9753. Using the value printer refactor from #9606, Nix will print at most 10 attributes, 10 list items, and 1024 bytes of each string in error messages before giving up and printing a message like «20373 attributes elided» to indicate the missing values:

$ nix repl nixpkgs
Loading installable 'flake:nixpkgs#'...
Added 5 variables.
nix-repl> builtins.map (lib.id) legacyPackages.aarch64-darwin
error:
       … while calling the 'map' builtin
         at «string»:1:1:
            1| builtins.map (lib.id) legacyPackages.aarch64-darwin
             | ^

       … while evaluating the second argument passed to builtins.map

       error: expected a list but found a set: { _type = "pkgs"; AAAAAASomeThingsFailToEvaluate = «thunk»; AMB-plugins = «thunk»; ArchiSteamFarm = «thunk»; AusweisApp2 = «thunk»; BeatSaberModManager = «thunk»; CHOWTapeModel = «thunk»; ChowCentaur = «thunk»; ChowKick = «thunk»; ChowPhaser = «thunk»; «20373 attributes elided» }

nix repl doesn't currently use this output-limiting functionality, but instead sets a maximum depth (of 1 level) to print to, with the :p command used to print the entire structure:

$ nix repl
nix-repl> { a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s.t.u.v.w.x.y.z = 1; }
{ a = { ... }; }

nix-repl> :p { a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s.t.u.v.w.x.y.z = 1; }
{ a = { b = { c = { d = { e = { f = { g = { h = { i = { j = { k = { l = { m = { n = { o = { p = { q = { r = { s = { t = { u = { v = { w = { x = { y = { z = 1; }; }; }; }; }; }; }; }; }; }; }; }; }; }; }; }; }; }; }; }; }; }; }; }; }; }

But the maximum depth of 1 still produces virtually unbounded output for large flat attribute sets like nixpkgs. Evaluating legacyPackages.aarch64-darwin in nix repl nixpkgs will happily spend 4 minutes printing 400,000 lines of output, and from experience seems rather uninterested in stopping when Ctrl-C is pressed.

Describe the solution you'd like

I would like to set nix repl to pretty-print (as in #9931) a modest number of maximum attributes, list items, and strings. The printer will track if any values were omitted, and this information will be used to format a tip for the user. Roughly like this:

$ nix repl nixpkgs
nix-repl> legacyPackages.aarch64-darwin
{
  _type = "pkgs";
  AAAAAASomeThingsFailToEvaluate = «thunk»;
  AMB-plugins = «thunk»;
  ArchiSteamFarm = «thunk»;
  AusweisApp2 = «thunk»;
  BeatSaberModManager = «thunk»;
  CHOWTapeModel = «thunk»;
  ChowCentaur = «thunk»;
  ChowKick = «thunk»;
  ChowPhaser = «thunk»;
  «20373 attributes elided»
}
# Note: Some attributes were elided. Print the whole value with `:p` or
# increase `--option max-attrs-print ...` to see more of the value.

(NB: In practice, we'd want to print a decent amount of attributes, maybe 100, before eliding any.)

Combined with #9922 and #9944, we could also add a REPL command to change the maximum values to be printed at runtime:

nix-repl> :set max-attrs-print 100
nix-repl> legacyPackages.aarch64-darwin  # Print the first 100 attributes.

This could be used to improve error ergonomics as well, by providing an opt-in for printing more of failing values:

error:
       … while calling the 'map' builtin
         at «string»:1:1:
            1| builtins.map (lib.id) legacyPackages.aarch64-darwin
             | ^

       … while evaluating the second argument passed to builtins.map

       error: expected a list but found a set: { _type = "pkgs"; AAAAAASomeThingsFailToEvaluate = «thunk»; AMB-plugins = «thunk»; ArchiSteamFarm = «thunk»; AusweisApp2 = «thunk»; BeatSaberModManager = «thunk»; CHOWTapeModel = «thunk»; ChowCentaur = «thunk»; ChowKick = «thunk»; ChowPhaser = «thunk»; «20373 attributes elided» }

       note: Some attributes were elided. To print more attributes, use
       `--option max-attrs-print ...`.

Additional context

Why does this functionality require setting handlers (#9922)?

It may not seem obvious, but this functionality requires setting handlers (#9922). This is because C++ initializes static values in an arbitrary and undefined order before executing the main function. In print-options.hh, we have some code like this:

static PrintOptions errorPrintOptions = PrintOptions {
    .maxAttrs = 10,
    // ...
};

If we naïvely added a setting like this:

Setting<size_t> maxAttrsPrint{this, 10, "max-attrs-print",
    "The maximum number of attributes to print before eliding."};

And updated the errorPrintOptions definition to match:

static PrintOptions errorPrintOptions = PrintOptions {
    .maxAttrs = evalSettings.maxAttrsPrint,
    // ...
};

Then the compiler may very well decide to initialize errorPrintOptions before evalSettings, leading to zero attributes being printed in error messages. (Worse, the compiler could initialize the two statics in different threads, leading to a non-deterministic number of attributes being printed.) With a setting handler, we could ask the max-attrs-print setting to simply update errorPrintOptions.maxAttrs when it's set.

The alternative, which I think is probably a good idea regardless of whether settings handlers are added, is to make setting values (and many other statics in the Nix codebase) functions, so they run in a predictable, programmer-controlled order, from within the main function.

Priorities

Add :+1: to issues you find important.

[roberth]

This might be possible without having settings handlers (#9922) for settings that are localized to the repl. Some other settings might also just work, without settings handlers.

[thufschmitt]

Triaged during the Nix maintainers meeting:

Proposal looks good, especially the improved formatting

Setting configuration interactively is also nice, but non-trivial to implement correctly, and is not a priority for the team.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-04-08-nix-team-meeting-136/42963/1