build issues with nix on macOS Mojave

[tscholak]

I just cloned the repo and tried to build it with nix on macOS but to no avail:

$ git clone [email protected]:dhall-lang/dhall-haskell.git

$ cd dhall-haskell

$ git --no-pager log -1 --pretty=oneline
5db1051311c34c7709b3a11803e54ce4c911f272 (HEAD -> master, origin/master, origin/HEAD) Update stack/cabal config to sync with new structure (#671)

$ uname -a
Darwin futumorphism.local 18.0.0 Darwin Kernel Version 18.0.0: Wed Aug 22 20:13:40 PDT 2018; root:xnu-4903.201.2~1/RELEASE_X86_64 x86_64

$ xcode-select --install
xcode-select: error: command line tools are already installed, use "Software Update" to install updates

$ echo $NIX_CFLAGS_COMPILE
-idirafter /usr/include

$ echo $NIX_CFLAGS_LINK
-L/usr/lib

$ nix-build
building '/nix/store/67p58mwal0lxmy16z77l6skh2p4w4z3k-cabal2nix-2.10.2.drv'...
setupCompilerEnvironmentPhase
Build with /nix/store/79qa9dkrjqh485pa888q0p2dfqznwdyp-ghc-8.4.3.
ignoring (possibly broken) abi-depends field for packages
ignoring (possibly broken) abi-depends field for packages
unpacking sources
unpacking source archive /nix/store/62hrfkr64piwp10q9pzxirvqk6n33v1f-cabal2nix-2.10.2.tar.gz
source root is cabal2nix-2.10.2
setting SOURCE_DATE_EPOCH to timestamp 1533552458 of file cabal2nix-2.10.2/test/golden-test-cases/zlib.nix.golden
patching sources
compileBuildDriverPhase
setupCompileFlags: -package-db=/private/var/folders/2y/4nx786m95275s94z8vpk5y_00000gn/T/nix-build-cabal2nix-2.10.2.drv-0/setup-package.conf.d -j8 -threaded
[1 of 1] Compiling Main             ( Setup.hs, /private/var/folders/2y/4nx786m95275s94z8vpk5y_00000gn/T/nix-build-cabal2nix-2.10.2.drv-0/Main.o )
Linking Setup ...
clang-5.0: warning: argument unused during compilation: '-nopie' [-Wunused-command-line-argument]

In file included from /private/var/folders/2y/4nx786m95275s94z8vpk5y_00000gn/T/nix-build-cabal2nix-2.10.2.drv-0/ghc58514_0/ghc_4.c:1:0: error:


In file included from /nix/store/79qa9dkrjqh485pa888q0p2dfqznwdyp-ghc-8.4.3/lib/ghc-8.4.3/include/Rts.h:29:0: error:


/nix/store/79qa9dkrjqh485pa888q0p2dfqznwdyp-ghc-8.4.3/lib/ghc-8.4.3/include/Stg.h:77:10: error:
     fatal error: 'math.h' file not found
   |
77 | #include <math.h>
   |          ^
#include <math.h>
         ^~~~~~~~
1 error generated.
`cc' failed in phase `C Compiler'. (Exit code: 1)
builder for '/nix/store/67p58mwal0lxmy16z77l6skh2p4w4z3k-cabal2nix-2.10.2.drv' failed with exit code 1
cannot build derivation '/nix/store/cjy2rkd5qdqzpkqkgrdgl92vn3qlmyfh-cabal2nix-dhall.drv': 1 dependencies couldn't be built
error: build of '/nix/store/cjy2rkd5qdqzpkqkgrdgl92vn3qlmyfh-cabal2nix-dhall.drv' failed
(use '--show-trace' to show detailed location information)

Is there a workaround for this other than https://hydra.nixos.org/build/83911384/download/1/nixpkgs/manual.html#builds-on-darwin-fail-with-math.h-not-found? It doesn't seem that this workaround actually works...

[tscholak]

I just noticed that math.h isn't in /usr/include but in /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/. Hence I set $NIX_CFLAGS_COMPILE to that value instead. Unfortunately, that doesn't seem to do the trick either :/

[Gabriella439]

@tscholak: I probably won't be of much help here. The best I can do is to document the development environment I use for OS X which is working for me when doing development on Dhall projects:

• I use OS X version 10.13 (High Sierra)
• I have never installed the XCode developer tools
• I have never installed GHC (I use the one provided by nix-shell for each project)
• My OS X computer doesn't have a /usr/include directory
• My OS X computer doesn't have a /Library/Developer/ directory

[Gabriella439]

Some other things I should probably also mention:

• I use a multi-user Nix installation on OS X
• I haven't applied any work-arounds like modifying NIX_CFLAGS_COMPILE or NIX_CFLAGS_LINK

[tscholak]

thank you @Gabriel439! I have tried to reproduce the error with a single-user installation of nix on a High Sierra machine (17.7.0), but I can't. It could be something about Mojave, or maybe this particular installation of nix is somehow broken...

[Gabriella439]

@tscholak: Could the issue be that you installed the XCode developer tools? Has anybody tested if uninstalling them fixes the problem?

Also, do you have ghc installed? Some comments on that issue seemed to indicate that this might be due to interference from a globally installed ghc

[tscholak]

both machines (darwin 17.7.0 and 18.0.0) have a global ghc:

$ ls -lah `which ghc`
lrwxr-xr-x  1 root  wheel    67B 26 May 12:49 /usr/local/bin/ghc -> /Library/Frameworks/GHC.framework/Versions/8.0.2-x86_64/usr/bin/ghc

and both machines have the xcode dev tools installed. the only difference I could find so far is the macOS version

[tscholak]

ok, let's try this again... I've upgraded to multiuser nix on Mojave:

$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-darwin"`
 - host os: `Darwin 18.2.0, macOS 10.14.1`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.1.3`
 - channels(root): `"nixpkgs-19.03pre158005.32340793aaf"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

stay tuned...

encountered so far:

• https://github.com/NixOS/nixpkgs/issues/49781 (can't install cabal2nix)
• https://github.com/NixOS/nix/issues/2244 (dyld: Library not loaded: /usr/lib/system/libsystem_network.dylib, fixed in latest nixpkgs)

[Gabriella439]

Yeah, I'm familiar with https://github.com/NixOS/nix/issues/2244. That will be trickier to fix because I'm depending on a specific revision of nixpkgs for support for static linking of binaries (See: https://github.com/NixOS/nixpkgs/issues/43795)

It sounds like this is a Mojave-specific issue, then

[joneshf]

Do we know which versions of dhall this affects? We're looking to lock down dhall versions across High Sierra, Mojave, and Linux (using the pre-compiled binaries on Linux).

We were on 1.15.1, but we can't find the linux binaries anymore so we've gotta upgrade to something newer. 1.15.1 seems to still build on Mojave though. We tried 1.19.1, but on Mojave it gives the failures explained in this issue.

[Gabriella439]

@joneshf: My understanding is that this only affects things built by Dhall's CI since it uses a very specific version of Nixpkgs to build static executables. Building with cabal/stack/brew should still work on all platforms

If you want to fix Dhall's CI to build it for you then you will need to forward port the static Haskell executables work to work against newer versions of Nixpkgs/Cabal. Specifically, you would need to forward port these two forks of Nixpkgs and Cabal to rebase those sets of changes onto newer versions:

https://github.com/dhall-lang/dhall-haskell/blob/0460f28a382a24970d9e93e0aef495509b2639f4/nix/shared.nix#L379-L384

https://github.com/dhall-lang/dhall-haskell/blob/0460f28a382a24970d9e93e0aef495509b2639f4/nix/shared.nix#L459-L467

Alternatively, you could politely nudge people to merge the necessary changes to complete the static executables work:

https://github.com/NixOS/nixpkgs/issues/43795

[joneshf]

Oops, sorry for being confusing. Let me provide more information.

N.B. Before reading further: we're not really looking for a fix to our situation. We're fine with any version of Dhall (we've been using 1.15.1 for a while now, but that's mostly out of laziness). So don't feel like you have to go out of your way to diagnose or help or anything like that. If you have a fix, great! We'll gladly take help, but it's not expected nor being asked.

We use nix on High Sierra and Mojave. When we tried to build 1.19.1 we did something like this:

let
  dhall = import dhallTarball;

  dhallTarball = builtins.fetchTarball {
    sha256 = "0fwz1kz9d0ny4209cxwj1zl7hqglxphyfvpf9mcy8c11b9zhdgdv";
    url = "https://github.com/dhall-lang/dhall-haskell/archive/1.19.1.tar.gz";
  };

  nixpkgs = import nixpkgsTarball;

  nixpkgsTarball = builtins.fetchTarball {
    sha256 = "1ib96has10v5nr6bzf7v8kw7yzww8zanxgw2qi1ll1sbv6kj6zpd";
    url = "https://github.com/NixOS/nixpkgs/archive/6a3f5bcb061e1822f50e299f5616a0731636e4e7.tar.gz";
  };
in
  nixpkgs.mkShell {
    buildInputs = [ dhall.dhall ];
  }

That worked on High Sierra, but failed on Mojave. I have High Sierra, co-worker has Mojave, so I can't validate the failures. But, they mentioned that this issue is what they run into.

We can build dhall fine on either High Sierra or Mojave with something like:

let
  nixpkgs = import nixpkgsTarball;

  nixpkgsTarball = builtins.fetchTarball {
    sha256 = "1ib96has10v5nr6bzf7v8kw7yzww8zanxgw2qi1ll1sbv6kj6zpd";
    url = "https://github.com/NixOS/nixpkgs/archive/6a3f5bcb061e1822f50e299f5616a0731636e4e7.tar.gz";
  };
in
  nixpkgs.mkShell {
    buildInputs = [ nixpkgs.dhall ];
  }

This gives us 1.15.1. This has worked well for a long time.

Recently, we wanted to use dhall lint in CI. Our CI is Linux based. We can't run nix on our CI currently, and we don't want to build dhall on our CI machines. We can't find the dhall 1.15.1 static binaries anymore, so we have to use a version that has static binaries available. That leaves us with deciding between: 1.16.1, 1.17.0, 1.18.0, 1.19.0, 1.19.1 (which we know won't work), 1.20.0, and 1.20.1.

I was hoping to figure out which of these versions are affected by the issue laid out here so we could use one that will build on Mojave. If we don't know which versions are affected, that's fine. I figured it was worth asking in case the information was readily available.

[Gabriella439]

@joneshf: My guess is that this would probably affect all versions that were built using static linking. The reason why is that I believe this is likely an issue with Nixpkgs and Dhall hasn't changed the revision of Nixpkgs that it depends on since static linking was introduced.

[joneshf]

Ah. That makes sense! Thanks!

[Gabriella439]

You're welcome! 🙂

[ari-becker]

I have a teammate who got bit by this, on both 1.30.0 and 1.31.0.

Yeah, I'm familiar with NixOS/nix#2244. That will be trickier to fix because I'm depending on a specific revision of nixpkgs for support for static linking of binaries (See: NixOS/nixpkgs#43795)

It sounds like this is a Mojave-specific issue, then

@Gabriel439 I see that in the interim, static-haskell-nix is now based on nixpkgs unstable again? https://github.com/NixOS/nixpkgs/issues/43795#issuecomment-582213348

If so, would it be possible to update the nixpkgs pin and release a 1.31.1?

[Gabriella439]

@ari-becker: I tried to update a couple of months ago and ran into this issue: https://github.com/nh2/static-haskell-nix/issues/73

... but I can try again and see if anything changed since then

[Gabriella439]

@ari-becker: Yeah, I updated to the latest static-haskell-nix and got the same result as https://github.com/nh2/static-haskell-nix/issues/73#issuecomment-580499876

[ari-becker]

@Gabriel439 I see, thanks for trying again.

I guess that my current workaround is to use https://github.com/justinwoo/easy-dhall-nix considering that my use-case only requires the final binaries. I'm not sure why the Darwin binaries aren't being fetched from cache (I verified with my teammate that he does have both the dhall-lang.org and the Cachix caches set up in his nix.conf), but well, it's rather difficult to debug these issues when I'm not running Darwin, everybody's working from home, and my teammate isn't yet sold on using Nix everywhere to begin with :sweat_smile:

[Gabriella439]

@ari-becker: Note that none of the caches contain pre-built Darwin build products. They are currently Linux only

[ari-becker]

@Gabriel439 ah, I was under the impression that you were putting the OS X binaries in Cachix, at least? https://github.com/dhall-lang/dhall-haskell/pull/1666#issuecomment-586837089

[Gabriella439]

@ari-becker: Whoops! I forgot about that 🙂

I just uploaded the OS X binaries for 1.31.1 and I'm working on building and uploading them for 1.31.0, too

[Gabriella439]

Alright, 1.30.0 should be in the cache now

Part of the reason I forgot about these is that I run the release script from a Linux machine, so I have to remember to separately build and upload OS X build products