Segmentation fault on initial run for project using Template Haskell / Yesod

[skress]

I had initially opened #1207 which was closed later in favor of #1520. My initial problem was that HIE did not work on macOS for a simple yesod project. At that time my main focus was on the "temp object not found", like

hie: loadObj: /private/var/folders/4z/qvjhwky51571tqd352nmqmx00000gn/T/ghc89268_0/ghc_3.o: file doesn't exist

I mentioned a seg fault already in https://github.com/haskell/haskell-ide-engine/issues/1520#issuecomment-578490576 but I think it makes sense to open a separate issue for that.

Before continuing with the seg fault problem: I think that "my" problem with the temp object that cannot be found is that my initial run of HIE was terminated by the seg fault and the data in ~/.cache/hie-bios was left in an inconsistent state. When running hie without deleting ~/.cache/hie-bios a second time I will always get the temp file not found. After deleting ~/.cache/hie-bios I will get the seg fault again.

Back to the seg fault: as mentioned I started with a simple yesod project. One part of the demo app uses TH to create a list of static files including hash digests. I created a sample project, inlined the code from yesod to get rid of yesod as a dependency and tried to find the part where the seg fault occurs.

You can find the project at https://github.com/skress/hie-test/tree/71b67ae9c826fd95702e198c86493aa9dfa79796

I have included the sources of cryptonite (Crypto.Hash) and hs-memory (Data.ByteArray) into that project. stack build runs successfully on macOS and Linux. hie --debug src/Settings/StaticFiles.hs works on Linux but fails with a seg fault on macOS.

I am not sure if my trace messages really point to the problem or if laziness plays a role, but the last place that I can trace is https://github.com/skress/hie-test/blob/71b67ae9c826fd95702e198c86493aa9dfa79796/submodules/hs-memory/Data/ByteArray/Bytes.hs#L89-93

It is withPtr in Data.ByteArray.Bytes:

withPtr :: Bytes -> (Ptr p -> IO a) -> IO a
withPtr b@(Bytes mba) f = do
    a <- trace "Data.ByteArray Bytes withPtr 1" $ f (Ptr (byteArrayContents# (unsafeCoerce# mba)))
    trace "Data.ByteArray Bytes withPtr 2" $ touchBytes b
    trace "Data.ByteArray Bytes withPtr 3" $ return a

stack build runs through all those lines without problems, but when using hie --debug the trace output looks like this:

Crypto.Hash hashUpdate
Crypto.Hash hashUpdates - otherwise
Crypto.Hash hashUpdates - otherwise - copyAndFreeze
Data.ByteArray copyAndFreeze
Data.ByteArray Bytes bytesAllocRet
Data.ByteArray Bytes bytesAllocRet - 2
Data.ByteArray Bytes withPtr 1
[1]    89638 segmentation fault  hie --debug src/Settings/StaticFiles.hs

I can't trace this any further because I don't know how to print trace messages from within functions like byteArrayContents#. I am still trying to learn Haskell and primitive types/operations are way beyond my comfort zone.

I am also not sure if tracing it to the exact place in the source is the right approach. Another question might be: why is stack build ok but hie not? And why does it work on Linux but not on macOS. What is HIE doing to setup the GHC session that leads to this segmentation fault only on macOS?

Would be great if someone on macOS who has more knowledge about HIE/Haskell/GHC could pick this up ... or some hints what to try next would also be appreciated.

P.S.: I am relatively sure that this is a macOS specific problem. I have run hie --debug many times (even scripted +500 times) and it always worked on Linux and it always failed on macOS. Running HIE withing VS Code on Linux I had once the situation that was mentioned in #1520 where a temp object file was not found and after deleting some caches and recompiling again it started to work again at some point in time. So that might have been the race condition mentioned there.