Cross-compiling to armv6/7 seems to be broken

[afojhr]

Cross compiling to armv7l-hf-multiplatform / raspberryPi (armv7l / armv6l, dynamically linked, GHC 9.2.3, nixpkgs-unstable) from x86_64-linux works, but when I include a package which depends on TH (e.g. ghc-typelits-knownnat) and / or ghc-prim (e.g. JuicyPixels) it always fails while running remote-iserv (minimal reproducible example is here (full log):

---> Starting remote-iserv on port 8742
---| remote-iserv should have started on 8742
Listening on port 8742
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
iserv-proxy: <socket: 4>: hGetBufSome: resource vanished (Connection reset by peer)
/nix/store/gsk695dac5ryvp079bvrcm5i5m760prm-iserv-wrapper/bin/iserv-wrapper: line 10:   583 Segmentation fault      (core dumped) /nix/store/pdijlpvlixscsph1dvwn8914lsmx2zmz-qemu-6.2.0/bin/qemu-arm /nix/store/5ldy010djc29fc443i6izgky26fpkpj2-remote-iserv-exe-remote-iserv-armv7l-unknown-linux-gnueabihf-9.2.3/bin/remote-iserv tmp $PORT
ghc: ghc-iserv terminated (1)

I tried fixes from #441 and changed ghcFlavour to quick-cross (#411, #387) but that doesn't change anything.

I also tried cross and static with muslpi and the following flags, but here I can't even get GHC to compile because rt-compiler or llvm don't build (selected processor does not support 'dmb' in Thumb mode, full log here)

packages.hello.components.exes.hello = {
  configureFlags = pkgs.lib.optionals pkgs.stdenv.targetPlatform.isMusl [
    "--disable-executable-dynamic"
    "--disable-shared"
    "--ghc-option=-optl=-pthread"
    "--ghc-option=-optl=-static"
    "--ghc-option=-optl=-L${prev.gmp6.override { withStatic = true; }}/lib"
    "--ghc-option=-optl=-L${prev.zlib}/lib"
    "--ghc-option=-optl=-L${prev.libffi.overrideAttrs (old: { dontDisableStatic = true; })}/lib"
    "--ghc-option=-optl=-L${prev.numactl.overrideAttrs (old: { dontDisableStatic = true; })}/lib"
  ];
};

Only when using the exact commit as in https://github.com/michivi/cross-haskell-app (GHC 8.6.5, nixpkgs 20.03, haskell.nix: 21a6d6090a64ef5956c9625bcf00a15045d65042) cross and static builds, but then it also fails with TH when running remote-iserv (can't find ghc-prim). Here I tried playing around with reinstallableLibGhc and nonReinstallablePkgs, but this doesn't help either (Maybe the haskell.nix rev is just too old).

Thank you for your help

[angerman]

What you are seeing is effecitvely the cross compilation pipeline for TH to fail. Unless you are absolutely in need to target 32bit raspberries, I'd sugest to just go with 64bit (aarch64); most raspberries are aarch64 anyway. The reason is simply that the 32bit support in GHC is very poor. I would also not like to provide any (even remote) statements around the fitness of the aarch32 backend in GHC. If you truely intent on going for 32bit arm, please be aware that you'll likely have to hack on GHC to fix some bugs.

I did write about iOS and Android with GHC, where I did show how the kind of debugging involved. Agains the lack of proper concurrency and maintainence of the aarch32 backend in GHC (as well as the assumption that systems are 64bit in almost any modern haskell code on hackage) would mean you'll have to debug and fix quite a bit.

The actual failure you are seeing most likely stems from the following, iserv has to load the relevant haskell code into memory to link it and run it to evaluate the TH splice. Something during the loading/linking/evaluation phase is broken and as such the iserv process segfaults. The result is that GHC sees it's remote TH evaluation process to die and end the compilation. It could be a missing libgcc symbol, or some relocation in the object files that isn't handled properly, or something else. The steps from here to fix this are most likely:

• debug the iserv invocation
• and fix the relevant GHC rts
• rebuild GHC
• repeat until done.

[ghost]

Thanks for the quick reply.

Unfortunately I have to target 32bit because the board has an ARM Cortex-A9 (armv7) chip. Since the debugging seems to be very time consuming I'm continuing to use my halfway working cross & static GHC / switch to another language. It took me a very long time to figure this out, so maybe it would be good to document cross compiling a bit more for other beginners (for example which architectures are supported by haskell.nix).

Regarding compiling against musl: Is there a reason why the current branch of haskell.nix doesn't build GHC 9.2.3 with muslpi (armv6-linux-musleabihf)? Here it already fails when building compiler-rt / llvm. As I've stated this works on the old branch with GHC 8.6.5.