haskell.nix
haskell.nix copied to clipboard
input-output-hk
build-tool-depends executables don't appear in shellFor
I want to use tasty-discover, which I have added to my .cabal file as:
build-tool-depends: tasty-discover:tasty-discover
As described in https://github.com/input-output-hk/nix-tools/issues/95, this adds a build-tools entry in the cabal-to-nix expression, which is used in setup-builder.nix to form executableToolDepends. However, it doesn't propagate correctly into the shell produced with shellFor. For build-tool-depends to work, we need a tasty-discover entry in the ghc-pkg database that also has the tasty-discover executable. As things currently stand, there is no entry at all. We can use additional = pkgs: [ pkgs.tasty-discover ], but that will only add a tasty-discover to the package database as a library - no executables will get included.
Related/dupe: https://github.com/input-output-hk/haskell.nix/issues/760 https://github.com/input-output-hk/haskell.nix/issues/231
@rvl Oh, fantastic! Do you need any help with anything? Happy to test anything, if that helps!
A similar issue seems to happen if your shell dependencies have build tool dependencies.
E.g., cardano-transactions has a test suite with a build tool dependency on cardano-tx, and when I try to create a shell on a cabal project that uses it, it errors with:
error: Neither the Haskell package set or the Nixpkgs package set contain the package: cardano-tx (build tool dependency).
If this is a system dependency:
You may need to augment the system package mapping in haskell.nix so that it can be found.
If this is a Haskell dependency:
If you are using Stackage, make sure that you are using a snapshot that contains the package. Otherwise you may need to update the Hackage snapshot you are using, usually by updating haskell.nix.
Not sure if the same thing happens on stack projects, but I would presume not considering that cardano-transactions uses stack.
I spent a little bit of time looking at this.
Firstly, I noticed that we don't pass build-tools into the fake component that we use to create the wrapped GHC for shellFor. Easy enough to fix.
But then it still doesn't appear in the package db. It turns out that makeConfigFiles doesn't use build-tools at all. So... how does anything work? My guess is that the Setup.hs interface to cabal doesn't insist on build-tool-depends being in the package db, and so we get away with just providing the executable. But probably we should register them, I'm just not sure how right now.
Yeah I don't think they should be registered in the same package db, especially if package DBs are to be per machine for cross. When you need a library, you usually need lots of information beyond the machine code (unfoldings for inlining, etc.) but for an executable dependency, one just needs to be able to run the thing, and being agnostic to how the sausage is made is probably better for sake of determinism / caching.
Yeah I don't think they should be registered in the same package db, especially if package DBs are to be per machine for cross.
Well, not having it in the package db confuses cabal, so I don't see how we can avoid it (if we want this usecase to work!).
As far as cross goes, obviously the build-tools should be in the native package db in the glorious future once we've got a separate native package db. Then I think there's no real problem with them being there.
Note that this really isn't specific shellFor, but specific to v2 cabal. You can always use v1-* cabal commands to get the old behavior (see SolveExecutables in cabal code). I think a reason this is coming up recently is that cabal 3.4.0.0 made v2 the default.
The reason this is really difficult is that both Nixpkgs & haskell.nix use the Setup.hs method of configure / install, which doesn't install a .conf file for executable components. I'm not sure if it makes more sense to patch Cabal so Setup.hs method installs executables correctly, or try to switch to the new install method.
See Cabal comment for reference:
build-tools dependencies NB: Only include these dependencies if SolveExecutables is True. It might be false in the legacy solver codepath, in which case there won't be any record of an executable we need.
https://github.com/haskell/cabal/blob/master/cabal-install-solver/src/Distribution/Solver/Modular/IndexConversion.hs#L348-L357
As far as cross goes, obviously the build-tools should be in the native package db in the glorious future once we've got a separate native package db.
v2 cabal-install recently got a --package-db flag that an be used to specify additional package dbs, though I'm not sure if it can actually help with this issue. Here's the pr for refernece: https://github.com/haskell/cabal/pull/7676
For the record, this is also an issue for llvm-hs, which needs the hsc2hs executable: https://github.com/llvm-hs/llvm-hs/blob/eda85a2bbe362a0b89df5adce0cb65e4e755eac5/llvm-hs/llvm-hs.cabal#L74.
The latest state of this issue seems to be: https://github.com/input-output-hk/haskell.nix/issues/1367#issuecomment-1207454622
No, I think that's unrelated. This isn't anything to do with source-repository-packages. This is specifically about how build-tool-depends works.
Is there any work that can be done to help solve this problem? We're trying to migrate to haskell.nix but this is a problem for us. We'd be happy to contribute fixes or consulting time to the issue.
Is there any work that can be done to help solve this problem? We're trying to migrate to
haskell.nixbut this is a problem for us. We'd be happy to contribute fixes or consulting time to the issue.
I haven't been able to look into this yet, but I really hope you are able to push this front forward.
My primary selling point for haskell.nix has been "You can avoid knowing nix and just update your cabal.project/package.cabal file but get cross compilation, reproducibility, docker containers, etc". @parsonsmatt I see in your linked issue your team is wanting the same: Use cabal.project as a source of truth.
Partial cabal.project support breaks this sellling point. I hope to deep dive into this issue and figure out haskell.nix well enough to solve some of these problems myself in the future, but I have other high priority things blocking it sadly :cry:
