Surprising behaviour of buildTestBracketed

[edsko]

Suppose we we something like

main = defaultMain [testA, testB]

where testA is constructed with buildTestBracketed. If we run the test suite using

test-suite -t testB

then the IO actions of testA will still be executed.

[edsko]

Actually, the problem is larger then this. When there are multiple tests constructed with buildTestBracketed, all their setup functions are run at the start, rather than one by one when each test is run.

[edsko]

I'll see if I can come up with a patch.

[edsko]

Ok, so I did look at this, but it's not clear to me how to implement this without making a lot of invasive changes. Ideally, I'd like something like

bracketTest :: IO a -> (a -> IO b) -> (a -> Test) -> Test

I thought I might be able to implement this much like mutuallyExclusive has been implemented, but I cannot figure out how to make it work. I can define something like

data BracketTest t = forall a b. BT (IO a) (a -> IO b) (a -> t)

and I can give a TestLike instance for BracketTest, but then I get stuck trying to define

bracketTest :: IO a -> (a -> IO b) -> (a -> Test) -> Test
bracketTest (Test tn t) = ???

(Note that if we did have something like this, mutuallyExclusive could easily be defined in terms of this.)

I am now using this workaround:

bracketTest :: forall a. IO a -> (a -> IO ()) -> (IO a -> Test) -> Test
bracketTest setup cleanup test = Test.buildTestBracketed $ do
  setupMVar <- newMVar Nothing

  let runSetup :: IO a
      runSetup = modifyMVar setupMVar $ \didSetup ->
        case didSetup of
          Just a  -> return (didSetup, a)
          Nothing -> setup >>= \a -> return (Just a, a)

      runCleanup :: IO ()
      runCleanup = modifyMVar_ setupMVar $ \didSetup ->
        case didSetup of
          Just st -> do cleanup st ; return Nothing
          Nothing -> return Nothing

  return (test runSetup, runCleanup)

which works without any modifications to test-framework, but the extra IO in the type is unfortunate.