segfault related to TemplateHaskell

[gelisam]

Building the following code (e.g. with stack build) and then running haskell-code-explorer results in a segfault (yes, a segfault in Haskell, yikes!):

module Lib where

import Prelude

import qualified Data.Yaml as Yaml
import Language.Haskell.TH (Exp, Q, runIO)
import Language.Haskell.TH.Syntax (addDependentFile)

migrations :: ()
migrations = $(do
  _ <- runIO (Yaml.decodeFileEither "file.txt" :: IO (Either Yaml.ParseException Yaml.Value))
  [|()|])

The above snippet is of course silly, but I encountered the bug in a real project and the above is the result of deleting as much as possible from that project in order to create a minimal repro case. I have put the code in a git repository for your convenience: https://github.com/gelisam/haskell-code-explorer-bug

Note that since this is TemplateHaskell code, the Yaml.decodeFileEither "file.txt" is executed at compile time. Building the project succeeds, so it doesn't seem to be a bug in the yaml package. However, replacing Yaml.decodeFileEither "file.txt" with the simpler readFile "file.txt" does not trigger the bug, so there must be some weird interaction between the yaml library and haskell-code-explorer. Since the symptom is a segfault, I suspect it has to do with the libyaml C library it uses under the hood.

[alexwl]

Interesting,

There was a bug in yaml library that caused a segfault (https://github.com/snoyberg/yaml/issues/5), but that bug has been fixed since version 0.4.1.2.

I successfully indexed the package from https://github.com/gelisam/haskell-code-explorer-bug using haskell-code-indexer built with ghc-8.6.3.

I also tried: -haskell-code-indexer built with ghc-8.6.4 (changed resolver in stack.yaml to lts-13.12): no errors -haskell-code-indexer built with ghc-8.4.4 (changed resolver in stack.yaml to lts-12.16): no errors -haskell-code-indexer built with ghc-8.2.2 (changed resolver in stack.yaml to lts-11.3): got "unhandled ELF relocation(RelA) type 42" (it's a known GHC bug that has already been fixed).

It doesn't matter whether file.txt exists or not, still no segfault.

At the moment it is not clear how to reproduce the bug.

Can you post the full output of haskell-code-indexer -v?

[gelisam]

Oh! I forgot to mention something important: we tried it on two other macOS machines, and the segfault only occurs on Mojave.

[alexwl]

It may be related to https://gitlab.haskell.org/ghc/ghc/issues/16117 (Random segfaults at runtime on macOS Mojave caused by GHC 8.6.3 linker misaligning sections)

[gelisam]

Huh, that looks pretty nasty. Since you cannot reproduce it, I guess I'll just have to wait until that bug is fixed and then check if the segfault still occurs. Thanks!

[gelisam]

Still, here is the full output of haskell-code-indexer -v:

$ stack install
haskell-code-explorer-bug-0.0.0: configure (lib)
Configuring haskell-code-explorer-bug-0.0.0...
haskell-code-explorer-bug-0.0.0: build (lib)
Preprocessing library for haskell-code-explorer-bug-0.0.0..
Building library for haskell-code-explorer-bug-0.0.0..
[1 of 2] Compiling Lib              ( src/Lib.hs, .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/Lib.o )
[2 of 2] Compiling Paths_haskell_code_explorer_bug ( .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen/Paths_haskell_code_explorer_bug.hs, .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/Paths_haskell_code_explorer_bug.o )
haskell-code-explorer-bug-0.0.0: copy/register
Installing library in /Users/gelisam/working/haskell/haskell-code-explorer-bug/.stack-work/install/x86_64-osx/lts-13.5/8.6.3/lib/x86_64-osx-ghc-8.6.3/haskell-code-explorer-bug-0.0.0-LClv1IjW8llBci7bWj9Oxx
Registering library for haskell-code-explorer-bug-0.0.0..
$ haskell-code-indexer -v
2019-04-30 10:26:51.834241 EDT : [info] haskell-code-indexer version 0.1.0.0, GHC version 8.6.3.0
2019-04-30 10:26:51.83692 EDT : [debug] IndexerConfig {configPackageDirectoryPath = ".", configPackageDistDirRelativePath = Nothing, configOutputDirectoryName = Nothing, configLog = StdOut, configMinLogLevel = LevelDebug, configSourceCodePreprocessing = AfterPreprocessing, configCompression = Gzip, configGhcOptions = [], configIgnoreDirectories = []}
2019-04-30 10:26:51.839883 EDT : [info] Found stack.yaml. Executing "stack path --dist-dir" to get dist directory.
2019-04-30 10:26:52.050873 EDT : [info] Stack dist directory : .stack-work/dist/x86_64-osx/Cabal-2.4.0.1
2019-04-30 10:26:53.328096 EDT : [info] Indexing haskell-code-explorer-bug-0.0.0
2019-04-30 10:26:53.335399 EDT : [debug] Component id : setup
2019-04-30 10:26:53.335457 EDT : [debug] Modules : []
2019-04-30 10:26:53.33548 EDT : [debug] GHC command line options : 
2019-04-30 10:26:53.684626 EDT : [debug] Component id : lib
2019-04-30 10:26:53.684683 EDT : [debug] Modules : ["Lib","Paths_haskell_code_explorer_bug"]
2019-04-30 10:26:53.684704 EDT : [debug] GHC command line options : -fbuilding-cabal-package -O -outputdir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -odir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -hidir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -stubdir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -i -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -isrc -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/global-autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/global-autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -optP-include -optP.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen/cabal_macros.h -this-unit-id haskell-code-explorer-bug-0.0.0-LClv1IjW8llBci7bWj9Oxx -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db /Users/gelisam/.stack/snapshots/x86_64-osx/lts-13.5/8.6.3/pkgdb -package-db /Users/gelisam/working/haskell/haskell-code-explorer-bug/.stack-work/install/x86_64-osx/lts-13.5/8.6.3/pkgdb -package-id base-4.12.0.0 -package-id template-haskell-2.14.0.0 -package-id yaml-0.11.0.0-GAWf3vpOhGMK7o9d6zMyJu -XHaskell2010 -XTemplateHaskell Lib Paths_haskell_code_explorer_bug
Segmentation fault: 11

[alexwl]

Thanks!

I tried indexing on macOS Mojave (10.14) and also got "Segmentation fault: 11" with both GHG 8.6.3 and GHC 8.6.4.

Now I can investigate this issue.

There is a bug in persistent library that might have the same cause: https://github.com/yesodweb/persistent/issues/830 (Segmentation fault: 11 on macOS).

I'll try to reproduce the segfault without the involvement of haskell-code-indexer.

[alexwl]

I've managed to reproduce the segfault on macOS Mojave using only GHC API.

This code segfaults with "Segmentation fault: 11" message:

import Control.Monad.IO.Class
import qualified Data.List as L
import GHC
import Packages
import DynFlags
import GHC.Paths (libdir)
import Digraph (flattenSCCs)

ghcFlags = "-fbuilding-cabal-package -O -outputdir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -odir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -hidir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -stubdir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -i -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -isrc -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/global-autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/global-autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -optP-include -optP.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen/cabal_macros.h -this-unit-id haskell-code-explorer-bug-0.0.0-34p2k9iR4eu61IAyrOu1BO -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db /Users/alexwl/.stack/snapshots/x86_64-osx/lts-13.5/8.6.3/pkgdb -package-db /Users/alexwl/Documents/haskell-code-explorer-bug/.stack-work/install/x86_64-osx/lts-13.5/8.6.3/pkgdb -package-id base-4.12.0.0 -package-id yaml-0.11.0.0-GAWf3vpOhGMK7o9d6zMyJu -package-id template-haskell-2.14.0.0 -XHaskell2010 -XTemplateHaskell"

main = do
  runGhcT (Just libdir) $ do
    flags <- getSessionDynFlags
    (flags', _, _) <-
      parseDynamicFlagsCmdLine flags (map noLoc $ words ghcFlags)
    (flags'', _) <- liftIO $ initPackages flags'
    setSessionDynFlags $
      L.foldl'
        gopt_set
        (flags''
           {hscTarget = HscAsm, ghcLink = LinkInMemory, ghcMode = CompManager})
        [Opt_Haddock]
    let modules = ["Lib"] -- file from  https://github.com/gelisam/haskell-code-explorer-bug
    targets <- mapM (`guessTarget` Nothing) modules
    setTargets targets
    _ <- load LoadAllTargets
    modGraph <- getModuleGraph
    let topSortMods = flattenSCCs (topSortModuleGraph False modGraph Nothing)
        modSum = head topSortMods
    parsedModule <- parseModule modSum
    typecheckedModule <- typecheckModule parsedModule
    liftIO $ print "ok"

[gelisam]

The LinkInMemory bit reminds me of the https://gitlab.haskell.org/ghc/ghc/issues/16117 bug you linked to earlier, is there an alternative value ghcLink can be given?

[alexwl]

is there an alternative value ghcLink can be given?

There are 5 possible values (https://hackage.haskell.org/package/ghc-8.6.4/docs/DynFlags.html#t:GhcLink): NoLink Don't link at all LinkBinary Link object code into a binary LinkInMemory Use the in-memory dynamic linker (works for both bytecode and object code). LinkDynLib Link objects into a dynamic lib (DLL on Windows, DSO on ELF platforms) LinkStaticLib Link objects into a static lib

I tried them all but still got the segfault.