haskell.nix
haskell.nix copied to clipboard
devShell inconsistencies and strange behaviour
I have a flake that looks very similar to https://github.com/Liqwid-Labs/plutus-extra/blob/master/flake.nix.
When I check out that repo (Liqwid-Labs/plutus-extra) and enter a dev shell with nix develop
, the environment is fine and I can build everything with cabal
. But I noticed something strange: All of the packages added from cabal.project
are specified in additional
. In fact, when removing additional and entering a shell with nix develop
, the specified packages disappear from the package registry (ghc-pkg list
). This is really strange; shouldn't haskell.nix auto-populate the package registry with these packages, since they are needed to build the project? I don't see this additional
being used in this way in other haskell.nix projects - just in this singular one I could find that is using flakes (most haskell.nix projects don't seem to fully use flakes yet). Perhaps flake compatibility is an issue, or devShell
is doing something different than what nix-shell
does that breaks this?
Secondly, on a closed source project, specifying the things in additional
isn't enough, because nix develop
fails, complaining about cross compilation, even though I'm not cross-compiling. Perhaps the issue is I'm using pkgsCross.musl64
?
I was about to paste the error I got - but when I went to go reproduce the error, the following sequence of events happened and I've lost the error to scrollback:
- I ran
nix develop -L
with ashell = { ...some stuff...; additional = ps: [ ps.Win32-network ]; }
, got an error about cross compilation, we'll call this error A - Commented out entirety of
shell = { ...stuff... }
, rannix develop -L
, got a different error (expected/understandable), call this error B - uncommented
shell = { ...stuff... }
, rannix develop -L
, got error B again, not error A, despite the nix expression returning the state of (1)
So now, I can't reproduce error A. But I'll paste error B since it's what's happening now, I just wanted to document the strangeness. Here is error B:
postgresql-libpq-lib-postgresql-libpq-x86_64-unknown-linux-musl> Setup: The program 'pg_config' is required but it could not be found.
error: builder for '/nix/store/4d2h50zk734qlkmi7n4yv4vp1z1z3ann-postgresql-libpq-lib-postgresql-libpq-x86_64-unknown-linux-musl-0.9.4.3.drv' failed with exit code 1;
last 10 log lines:
> source root is postgresql-libpq-0.9.4.3
> setting SOURCE_DATE_EPOCH to timestamp 1000000000 of file postgresql-libpq-0.9.4.3/test/Smoke.hs
> patching sources
> updateAutotoolsGnuConfigScriptsPhase
> configuring
> Configure flags:
> --prefix=/nix/store/nzbmi8s83p50rvl0nx9mn0mbb9n5n57s-postgresql-libpq-lib-postgresql-libpq-x86_64-unknown-linux-musl-0.9.4.3 lib:postgresql-libpq --extra-include-dirs=/nix/store/vpyznj3q6yb6sk9r9rnz0k4mzr9a88na-postgresql-x86_64-unknown-linux-musl-13.5/include --extra-lib-dirs=/nix/store/86hq4c1hb4sqpygfyzla4dhyrn6lhjjr-postgresql-x86_64-unknown-linux-musl-13.5-lib/lib --package-db=clear --package-db=/nix/store/mrdqx60xiyclqvg6q0ji4g4dx8cn50i1-x86_64-unknown-linux-musl-postgresql-libpq-lib-postgresql-libpq-0.9.4.3-config/lib/x86_64-unknown-linux-musl-ghc-8.10.7/package.conf.d --flags=-use-pkg-config --exact-configuration --dependency=rts=rts --dependency=ghc-heap=ghc-heap-8.10.7 --dependency=ghc-prim=ghc-prim-0.6.1 --dependency=integer-gmp=integer-gmp-1.0.3.0 --dependency=base=base-4.14.3.0 --dependency=deepseq=deepseq-1.4.4.0 --dependency=array=array-0.5.4.0 --dependency=ghc-boot-th=ghc-boot-th-8.10.7 --dependency=pretty=pretty-1.1.3.6 --dependency=template-haskell=template-haskell-2.16.0.0 --dependency=ghc-boot=ghc-boot-8.10.7 --dependency=ghc=ghc-8.10.7 --dependency=Cabal=Cabal-3.2.1.0 --dependency=array=array-0.5.4.0 --dependency=binary=binary-0.8.8.0 --dependency=bytestring=bytestring-0.10.12.0 --dependency=containers=containers-0.6.5.1 --dependency=directory=directory-1.3.6.0 --dependency=filepath=filepath-1.4.2.1 --dependency=ghc-boot=ghc-boot-8.10.7 --dependency=ghc-compact=ghc-compact-0.1.0.0 --dependency=ghc-prim=ghc-prim-0.6.1 --dependency=hpc=hpc-0.6.1.0 --dependency=mtl=mtl-2.2.2 --dependency=parsec=parsec-3.1.14.0 --dependency=process=process-1.6.13.2 --dependency=text=text-1.2.4.1 --dependency=time=time-1.9.3 --dependency=transformers=transformers-0.5.6.2 --dependency=unix=unix-2.7.2.2 --dependency=xhtml=xhtml-3000.2.2.1 --with-ghc=x86_64-unknown-linux-musl-ghc --with-ghc-pkg=x86_64-unknown-linux-musl-ghc-pkg --with-hsc2hs=x86_64-unknown-linux-musl-hsc2hs --with-gcc=x86_64-unknown-linux-musl-cc --with-ld=x86_64-unknown-linux-musl-ld.gold --ghc-option=-optl-fuse-ld=gold --ld-option=-fuse-ld=gold --with-ar=x86_64-unknown-linux-musl-ar --with-strip=x86_64-unknown-linux-musl-strip --disable-executable-stripping --disable-library-stripping --disable-library-profiling --disable-profiling --enable-static --enable-shared --disable-coverage --enable-library-for-ghci --enable-split-sections
> Configuring library for postgresql-libpq-0.9.4.3..
> Setup: The program 'pg_config' is required but it could not be found.
>
For full logs, run 'nix log /nix/store/4d2h50zk734qlkmi7n4yv4vp1z1z3ann-postgresql-libpq-lib-postgresql-libpq-x86_64-unknown-linux-musl-0.9.4.3.drv'.
error (ignored): error: cannot unlink '/tmp/nix-build-elfutils-x86_64-unknown-linux-musl-0.186.drv-0/elfutils-0.186/tests': Directory not empty
error: 1 dependencies of derivation '/nix/store/a7gkyj6gi14pk0aw0lx04sr653lpanxh-ghc-shell-for-packages-x86_64-unknown-linux-musl-env.drv' failed to build
Some questions:
- Why is it building anything at all? I've already run
nix build .#flake.x86_64-linux.packages."my_package"
- every dependency necessary to build the project should already exist in the nix store. - Why is
additional
necessary?
I'll work on a minimal reproduction and post when I'm done (if I'm successful).
Minimal repro at https://github.com/chessai/haskell-nix-repro/commit/df3a3d1b6771d341835bd50affcfdb017bfb2b62
Curiously, I'm getting error A, not error B, and commenting out shell = { ...stuff... }
causes nix develop
to succeed.
Curiously, changing pkgsCross.musl64
to pkgs
resolves the issue. I still need static linking for deploys, and I wanted to just use the same package set for local dev/deploys for convenience, unfortunately that isn't possible.
the fix is to change (using rough types) project :: System -> Project
to project :: StaticOrNot -> System -> Project
, using pkgs
if not static, and pkgsCross.musl64
if static.
Another unfortunate side effect of this is an immense amount of duplication in our cache.
Actually, I spoke too soon. When I add the packages back to additional
(which is still needed for some reason), nix develop
fails, failing to build one of the packages. Here is some nix repl high-level debugging:
$ nix repl
> :lf .
> :b projectStatic.x86_64-linux.hsPkgs.playground-common.components.library
<<<fails>>>
> :b projectStatic.x86_64-linux.pkgs.haskell.packages.ghc8107.playground-common
<<< `playground-common` doesn't exist>>>
> :b flakeStatic.x86_64-linux.packages."transaction-writer-utils:exe:transaction-writer"
<<<builds just fine>>>
playground-common
is a transitive dependency of transaction-writer
. So, somehow, nix develop
is using a different package set than nix build
when it comes to the dependencies needed to build the project. And playground-common
exists in packagesStatic.x86_64-linux.hsPkgs
, but it's clearly a different version, and it doesn't exist at all in packagesStatic.x86_64-linux.pkgs.haskell.packages.ghc8107
. But this leaves me wondering, where is the playground-common
(and every other package specified in the cabal.project!) my build is actually using?
Okay, an actual fix was to abandon additional
and use packages = ps: [ ps.my_package ]
. Still not sure where the discrepancies stem from, but at least I have something working now.
Maybe @hamishmack has a clue?