Help the user debug "illegal path references in fixed-output derivation"

[bmillwood]

Is your feature request related to a problem? Please describe.

I ran into the quoted error message recently and found it pretty difficult to track down what the path references actually were. There's some evidence of other people having trouble with this too:

• https://discourse.nixos.org/t/illegal-path-references-in-fixed-output-derivation/44360
• https://phip1611.de/blog/fixing-illegal-path-references-in-fixed-output-derivation-in-nix/

I was particularly flummoxed by how when I used nix-build --keep-failed, the directory I got didn't contain any store path references; I later figured out it kept the build directory, not $out, and I was writing the path references directly to the latter. It was frustrating that nix wouldn't tell me what paths I was referencing, or where they were referenced.

Describe the solution you'd like

I just went ahead and wrote https://github.com/NixOS/nix/commit/86540620290e2dee56adc0656c48a232251f8e9e which at least gives an example path, and changes the wording so that it's more clear what's illegal about the path references. One disadvantage of changing the wording is that you'd no longer be able to google the error message to get those helpful references above, but perhaps we could leave a comment on each mentioning the new message :)

I was going to open a PR instead of an issue, especially since the proposed change is so small, but I think there's a pretty strong chance that people more familiar with nix internals might have ideas about how to do something more ambitious that would be much more helpful, so it seemed sensible to make the issue first. I basically only read the part of the nix code that generated this error message, and my C++ is very rusty, so there's lots of ways my approach might not be right. But if people do like it, even if only as a first step, happy to open a PR with that.

Describe alternatives you've considered

One alternative is to give an example path but not change the error wording, to retain googleability. I think googleability is not important enough to go this way, but happy to hear other opinions.

Another alternative I considered was to dump all the bad references, perhaps gated behind a verbosity control or something. I currently lack the knowledge to implement this, but I could probably work it out if people think it's significantly better.

Even better would be anything that helps the user know exactly which file has the bad reference in it, which would presumably mean checking for the error at a different point in the code. (Again, I lack the knowledge to do this, but I could give it a go perhaps.)

For either of the latter two ideas, we could perhaps merge my easy change first, before I embark on something more difficult :)

In either case

Additional context

Here's an example output from my commit:

error: fixed-output derivations must not reference store paths: '/nix/store/c3w5x4sgszgl9f737l4012814aw72kwy-friendica-2024.08.drv' references 3 distinct paths, e.g. '/nix/store/516kai7nl5dxr792c0nzq0jp8m4zvxpi-bash-5.2p32'

(aside: if there is only one path it will say "1 distinct paths" and then give the only path as an example, which is somewhat silly, but when I tried to do something cleverer it highlighted the "s, e.g. " in purple, I guess because it does that with all printf-formatted components)

Priorities

Add :+1: to issues you find important.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/best-way-to-manually-test-changes-to-the-nix-binary/54210/1

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/illegal-path-references-in-fixed-output-derivation/44360/4

[MatejaMaric]

I was particularly flummoxed by how when I used nix-build --keep-failed, the directory I got didn't contain any store path references; I later figured out it kept the build directory, not $out, and I was writing the path references directly to the latter.

This also confused me. It would be nice if the error message also included this tip.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/error-illegal-path-references-in-fixed-output-derivation/56749/8

[Ericson2314]

I think you could just PR your commit? Let's not make the perfect the enemy of the good :)

It seems to me like you are asking whether we have more structured errors, and I wish we did, but that is not yet the case.

[coffeeispower]

Please release this, i just ran into this issue and it would be so useful Why nix needs to have such terrible error messages?

[Artturin]

Made https://github.com/NixOS/nix/pull/12356 with the patch

[bmillwood]

I think for now I'll close this issue as complete, with the expectation that people can easily re-open it if they want further improvements and find the existing discussion helpful for that.