haskell.nix icon indicating copy to clipboard operation
haskell.nix copied to clipboard

Optimise IFD?

Open L-as opened this issue 3 years ago • 2 comments

Currently, AFAICT, haskell.nix makes extensive use of IFD in the form of essentially importing the materialisation. This is problematic because of many obvious reasons:

  • IFD blocks evaluation, so if you have (import x.outPath) + (import y.outPath), they will be built sequentially, rather than in parallel, which is a huge issue.
  • Hydra doesn't work well with it.

A very related issue on GH is https://github.com/NixOS/rfcs/pull/109

By putting all IFD into a separate derivation, e.g. Hydra could build this before building everything else, thus ensuring correct functioning. It would also fix the issue with parallelism. This seems like a big undertaking to me however?

However, it seems to me like the problem also lies with Nix. There doesn't seem to be a good reason for the sequential behavior of IFD, see https://github.com/NixOS/nix/issues/6299.

L-as avatar Jun 20 '22 14:06 L-as

By putting all IFD into a separate derivation

What does this mean? IFD is, necessarily, "in a separate derivation", it's "import-from-derivation" after all. I really don't know what you're getting at here.

michaelpj avatar Jun 20 '22 15:06 michaelpj

one derivation. The RFC elaborates more on the approach.

L-as avatar Jun 20 '22 18:06 L-as