sol
Intermittent failure from missing packages
I've been seeing intermittent failure from missing packages. Sometimes doctest runs fine, other times it reports hidden package failures.
Here is just such a failure: https://circleci.com/gh/freckle/bcp47/80
I'm not sure if this is a doctest issue or something arising from stack v2.
Assuming that this is the relevant package.yaml: the doctest component is missing a dependency on the library component. That means that Stack is free to run the test before the library has been built, which presumably it's doing only some of the time (hence the nondeterminism), and has no way of knowing which dependencies should be in scope for the test.
I'm not actually sure why it works some of the time - I'd've expected it to consistently fail. Maybe something weird to do with concurrent execution of the test + building of the library.
So - try adding a dependency on bcp47 in the doctest component and see if it keeps happening.
(Also, I recommend cabal-doctest for integrating doctest with build tools - getting the dependency story right is annoying to do manually and it can go wrong in all sorts of subtle ways that only turn up as failures later, in certain environments.)
That means that Stack is free to run the test before the library has been built
Oh that is a fun side effect. Thanks for the info!
Hmm, actually I just tried out adding bcp47 to the build depends of doctest and it is still giving me intermittent failure with the same errors. It seems that is not the issue.
Definitely do try cabal-doctest. The story about how to plumb dependency information around so that doctest can see it isn't trivial. (Though, I'm surprised it's still intermittent - with the sequence of components fixed, I'd expect either consistent success or failure...)
I think it could be related to package environments overwriting each other. I have a multi-package stack project and I get similar errors. I noticed that all the tests load the same package environment.
A workaround is to execute the tests with stack test -j1 (but it is much slower).
