haskell
Inlining functions of Unboxed vector consumes a lot of memory
For quite some time test suite vector-tests-O2 was unbuildable on CI for newer ghc versions 8.2 - 8.10 See:
- #272
- https://ghc.haskell.org/trac/ghc/ticket/13535
The culprit was Unboxed vector, in particular compiling the full collection of Tests.Vector.Property.testPolymorphicFunctions properties restricted to Unboxed vector was consuming the most memory (~15Gb) (probably because it has the most number of properties). As a temporary fix #273 disables inlining of that collection of properties for Unboxed vector only. Compilation of Tests.Vector.Unboxed still takes the most amount of memory, but it is bearable ~1Gb
so to be clear, not that any specific thing is marked inline/noinline, in the test suite, BUT that our .Unboxed test clauses make the world fallover, and trigger some nonlinear blowup when other stuff is defined / optimized in the same module?
That is correct! All other vector types (Primitive, Storable and Unboxed) use the same properties and their compilation does not consume almost any memory.
i think we should still maybe keep this ticket open as a way to document/root cause analyze why that brought ghc to its knees
for the full explosion cabal get vector-0.12.0.3 ; cd vector-0.12.0.3 ; cabal configure --enable-tests ; cabal build should suffice, or cloning off the corresponding commit for that tagged release https://github.com/haskell/vector/tree/19189a884434ecd2c96212395f5705347b55775b
@GeorgeCo the module in master that just instantiates the testing properties with unboxed instances here https://github.com/haskell/vector/blob/eeb42ad42aa345ce192086baed80c805bcfc3e72/tests/Tests/Vector/Unboxed.hs is a smaller one per se.
When talking with ghc contributors about causes, notably @AndreasPK , one point thats been highlighted is that one factor is likely space leaks in the simplifier pass of ghc. (he got a tad sick which then interrupted that investigation)
on the flip side, @bgamari has pointed out that a compounding factor in the test suite prior to this split up (in V0.12.1.0 onwards) is that ghc is generally more agressive about inlinining within a single module rather than across modules (which could probably be reproduced by making certain definitions in the test suite INLINE or inlinable?)
@GeorgeCo i'm not sure if thats helpful info or just a more condensed summary of stuff you already knew. but i hope it helps none the less