Excessive memory usage when compiling with profiling enabled

[dgaw]

Configurator seems to cause GHC 8.0.2 to allocate a huge amount of memory during compilation with profiling enabled, specifically:

stack build --profile

I've created an minimum example to demonstrate this: https://github.com/dgaw/configurator-ghc-blowup

This example takes 6+ GB of RAM to compile and all it does is read 17 lines of a dummy config file.

[Acentelles]

Hi @dgaw , did you find out what was causing it?

[dgaw]

@Acentelles Unfortunately I didn't have time to investigate this.

[tdietert]

@dgaw I have found an ad-hoc fix for this situation. If you fix the type of require to be monomorphic in it's return type:

require :: Configurable a => Config -> Name -> IO a

to

require :: Config -> Name -> IO Text -- per your example project
require = Data.Configurator.require

then the memory explosion/simplifier ticks exhausted issue does not happen during compilation with stack build --profile. I don't really care to explore this issue any deeper, as in my main application I just defined several monomorphic versions of the require function and used them for most of the fields of my app config datatype, and I can compile the project with profiling now. Hope this helps!

[dgaw]

Thanks @tdietert. I don't use configurator anymore but I'm sure your workaround will be helpful to others.

[emhoracek]

I'm running into something that I think is related -- I'm getting this error when I enable profiling:

Simplifier ticks exhausted
  When trying UnfoldingDone lvl_s6Hv
  To increase the limit, use -fsimpl-tick-factor=N (default 100).
   
  If you need to increase the limit substantially, please file a
  bug report and indicate the factor you needed.
   
  If GHC was unable to complete compilation even with a very large factor
  (a thousand or more), please consult the "Known bugs or infelicities"
  section in the Users Guide before filing a report. There are a
  few situations unlikely to occur in practical programs for which
  simplifier non-termination has been judged acceptable.
   
  To see detailed counts use -ddump-simpl-stats
  Total ticks: 92080

I created this little program to trigger the error:

{-# LANGUAGE OverloadedStrings #-}

module Main where

import qualified Data.Configurator as C
import qualified Data.Configurator.Types as C

main = do
  bunchaStrings <- mkBunchaStrings undefined
  print bunchaStrings

data BunchaStrings = BunchaStrings String String String String String
                                   String String String String String
                                   String String String String deriving Show

mkBunchaStrings :: C.Config -> IO BunchaStrings
mkBunchaStrings conf =
  BunchaStrings
    <$> C.require conf "a"
    <*> C.require conf "b"
    <*> C.require conf "c"
    <*> C.require conf "d"
    <*> C.require conf "e"
    <*> C.require conf "f"
    <*> C.require conf "g"
    <*> C.require conf "h"
    <*> C.require conf "i"
    <*> C.require conf "j"
    <*> C.require conf "k"
    <*> C.require conf "l"
    <*> C.require conf "m"
    <*> C.require conf "n"

If I reduce the number of fields to 13 or less, the error goes away. The datatype in my real application has 26 fields 😬.

Adding the following definition suggested by @tdietert avoids the error! (In the example and in my app!)

require :: C.Config -> C.Name -> IO String
require = C.require