conduit
conduit copied to clipboard
Running a conduit in a function with an unrelated parameter twice leaks memory
Troubleshooting my own program, I stumbled upon another instance of memory leak.
When you call twice a function which ignores its parameter, and the function runs a conduit, it leaks memory.
This is my minimized example:
#!/usr/bin/env cabal
{- cabal:
build-depends: base ^>=4.17.2.1
, conduit ^>=1.3.5
default-language: GHC2021
ghc-options: -with-rtsopts=-M500M -threaded
-}
{- project:
with-compiler: ghc-9.4
-}
import Data.Conduit ((.|), runConduit)
import Data.Conduit.Combinators qualified as CC
main :: IO ()
main = do
_ <- go ()
_ <- go ()
pure ()
where
go _ = runConduit
$ CC.yieldMany [(1 :: Integer)..]
.| CC.last
The above program exits with "Heap exhausted;" after a while thanks to the rtsopts.
Here is what I observed so far:
- Not only
CC.lastDef
but alsoCC.last
causes memory leak (unlike #510) - Moving
go
to the top level still causes memory leak - Using the parameter but irrelevantly with the conduit still causes memory leak (see below)
- Replacing
[(1 :: Integer)..]
withiterate' succ (1 :: Integer)
still causes memory leak - Calling
go
only once does not leak memory - Removing the paremeter altogether does not leak memory
- Using the parameter as part of
CC.yieldMany
does not leak memory (see below)
Just evaluating the parameter but irrelevantly with the conduit still causes the memory leak:
-- Leaks memory, aborts
main :: IO ()
main = do
_ <- go ()
_ <- go ()
pure ()
where
go x = do
print x
runConduit
$ CC.yieldMany [(1::Integer)..]
.| CC.last
But using the parameter for CC.yieldMany
prevents memory leak:
-- Doesn't leak memory, loops forever
main :: IO ()
main = do
let i = 1 :: Integer
_ <- go i
_ <- go i
pure ()
where
go i = runConduit
$ CC.yieldMany [i..]
.| CC.last
I have completely no idea what is going on. Please let me know if I misunderstand the evaluation strategy of Haskell.