cabal2nix

cabal2nix copied to clipboard

src/Distribution/Nixpkgs/Haskell/FromCabal/PostProcess.hs has a lot of post processing hooks which do things that would also be possible via overrides, like disabling tests, adding dependencies, adding a phase hook etc.

I've stumbled over this before since cabal2nix sometimes does “magic” stuff I didn't expect. To me it'd be clearer to move these changes into nixpkgs's configuration-*.nix files, but there is the risk of breaking stuff for users of callCabal2nix, callHackageDirect etc.

I think we should strive at being very clear cut here.

For every kind of override we do we should decide whether to do it in cabal2nix or in nixpkgs. And this distinction should be clearly documented to minimize the danger of confusion.

But I exactly agree with your analysis, that I would prefer to have as much as possible in nixpkgs and am not sure what would be optimal. Maybe a good intuition is this: Stuff that would fit into configuration-common.nix would probably not fit callCabal2nix and callHackage. But stuff that would go into configuration-nix.nix would be nice if it also applied to them (not so much for callCabal2Nix but definitely for callHackage. If we can somehow figure this out we can probably spare people a few overrides.) But how can we achieve this?

I am brainstorming for ideas here:

• A possible middle ground would be having a config file for this which we maintain in nixpkgs but which will also be used by callCabal2nix and callHackage.

• Another possibility would be to expose an override option on haskellPackages where people can somehow substitute packages before the configuration-*.nix (or at least configuration-nix.nix) overrides get applied.

• We could theoretically somehow bake the overrides into mkDerivation? (With an option to disable them?) (I know this sounds kinda crazy but I actually think this could be more flexible if done right.

Having a "INFO: cabal2nix has applied the following auto-detected fixes to this derivation" somewhere in the build log would be sweet.

I think we should strive at being very clear cut here. For every kind of override we do we should decide whether to do it in cabal2nix or in nixpkgs. And this distinction should be clearly documented to minimize the danger of confusion.

Regardless of what we eventually decide to do about this, I imagine this is the most important thing.

---

I don't personally have a strong feeling about what to do about this, but I did want to bring up a few other points:

• Having the postprocessing stuff directly in cabal2nix was nice when we had tools like stack2nix that would use cabal2nix to generate a package set from an arbitrary stack.yaml file. The way stack2nix worked by default is that it wouldn't automatically apply fixes from configuration-common.nix or configuration-nix.nix. So you only got fixes for packages that were directly happening within cabal2nix.

  There are currently no tools that I know of that attempt to generate a package set from stack.yaml or the cabal solver, so maybe it doesn't make sense to take this into account. However, if we wanted to make it easy for someone to write a tool like this in the future, we might want to put a little thought into this. It probably wouldn't be crazy to translate the output of nix-tools into something compatible with nixpkgs haskell stuff.

• From a contributor's point of view, it used to be difficult to know when changes from cabal2nix would make their way to actually take effect in nixpkgs. This is less of a problem now, since we have cabal2nix-unstable. However, it is still slightly annoying to do the whole 1) update cabal2nix, 2) bump the hash in nixpkgs, 3) regenerate the package set to make sure you fixes work dance.

  This is also still somewhat of a problem for release branches. I imagine that we won't bump the version of cabal2nix on the release branches, so if users want fixes for packages, it would be necessary to do them within Nixpkgs for the release branches.

• For Haskellers, fixing stuff in cabal2nix isn't too difficult, but for Nixers, it is generally easier to fix things in Nixpkgs by writing Nix code.

Also slightly related to this, can cabal2nix take an argument that skips the post processing hooks, I was editing wxc package on Darwin and was very surprised when cabal2nix would generate these hooks that don't work on Darwin. Also took me a long time to figure out that cabal2nix does post processing (couldn't find it documented anywhere), assumed that it's somehow merging with the derivation in hackage-packages.nix

I think we should drop postprocessing in cabal2nix as much as possible (which is completely, I believe) for the following reasons:

• Post processing hooks are very intransparent; Even I have been caught out by them on occasion when debugging or working on cabal2nix.
• Having a decent set of post processing filters would be a insurmountable task:
  • We'd need to reflect necessary changes across versions to maintain a “it just works” property.
  • We'd start to emit platform conditionals inline in cabal2nix-generated expressions to make them work properly in all occasions: See https://github.com/NixOS/cabal2nix/issues/470 and https://github.com/NixOS/cabal2nix/issues/504#issuecomment-869649561, for example.
• It is easier and faster to fix things in nixpkgs (as pointed out above).

Thus I feel like we should think of cabal2nix should do a best effort conversion and the last 0-20% are achieved by overrides either maintained in nixpkgs or in a local overlay for out of tree stuff. Since upstream cabal files are imperfect in many cases (failing to declare system, test tool dependencies correctly…), I think it is not unreasonable to not attempt a perfect conversion automatically.