Passing resource to the subsequent conduit correctly

[maksar]

Otherwise, temp file might be deleted by ResourceT m1

[snoyberg]

Can you provide a demonstration of the temp file being freed too early? I don't see how proper usage of the function would result in that.

[maksar]

:set -XOverloadedStrings

import Conduit as C
import Data.Conduit.Binary as CB

C.runConduitRes (C.yield "contents" C..| CB.sinkCacheLength) >>= \(_, c) -> C.runConduitRes (c C..| C.sinkList)

Gives:

*** Exception: /tmp/conduit59986-2.cache: openBinaryFile: does not exist (No such file or directory)

[snoyberg]

That program is behaving as expected. The idea with ResourceT is that, whenever runResourceT is called, all resources are destroyed. runConduitRes implicitly calls runResourceT, so you end up closing the temp file in the first runConduitRes call. Instead, you can wrap the whole thing in runResourceT and then use runConduit instead of runConduitRes.

I believe the change in this PR will result in not performing cleanups in cases of non-termination, such as an exception or not consuming all input.

[maksar]

I agree with "non-cleanup in case of non-consuming" argument, so we end up creating own function.

Our use-case is streaming from S3, which isn't very reliable for long-running slow consumption steams. So we decided to proxy through temp file. It is beneficial for us to terminate all resources related to S3 streaming quickly and then slowly stream from temp file.

Function signature kind of suggested that user has a freedom of terminating first resourceT before terminating second – m1 and m2 are distinct. Maybe it worth at least unifying:

(MonadResource m) => ConduitT ByteString o m (ConduitT i ByteString m ())

Which would protect from calling runResourceT twice on accident:

# this would become impossible
runResourceT (runConduitRes (yield "contents" .| sinkCacheLength) >>= \(_, c) -> runConduit (c .| sinkList))
# only that will compile
runResourceT (runConduit (yield "contents" .| sinkCacheLength) >>= \(_, c) -> runConduit (c .| sinkList))
# well, this would still compile, but ResourceT (ResourceT m) type of c would indicate the probable coding mistake
runResourceT (runConduitRes (yield "contents" .| sinkCacheLength) >>= \(_, c) -> runConduitRes (c .| sinkList))