dhall-lang
Doctest fails with cabal new-build
Doctest fails with cabal new-build; it seems it cannot find the right things (object files, cabal macros) with the new-dist folder.
@AndrasKovacs: This seems to be an issue specific to new-build. See the discussion starting here:
https://github.com/dhall-lang/dhall-haskell/pull/1023#issuecomment-506136551
I believe this is an issue with either doctest or cabal's v2 commands, although I'm not sure which. I'm not sure how I can fix things on my end to make them work together.
@quasicomputational Do you happen to know how this could be fixed? (You seem to be well-acquainted with both doctest and cabal)
It's basically that doctest needs to be pointed towards the right things - it has no idea where v2-style package databases, includes, etc are. Invoking it with cabal v2-exec is one option, which stuffs that information into the environment so that doctest (or, rather, the GHC it invokes) can find the bits it needs, but that's obviously not friendly to running the tests with other build tools. (AIUI, this works on Stack because Stack invokes test-suites with that information in the environment; cabal-install doesn't.)
The alternative is to provide the information to it directly. cabal-doctest is a convenient way of doing this plumbing today; the biggest downside is it requires build-type: Custom, but dhall already depends on packages with that. I've just pushed a PR that does this.
Eventually, hopefully, Cabal will learn how to drive doctest and do the plumbing itself, and we won't need hacks.
I have collected some advice for working around this issue in https://github.com/dhall-lang/dhall-haskell/pull/1700#issuecomment-599254595.
FWIW, there's a new tool, cabal-docspec, that I thought could possibly replace doctest for our needs, but it's not quite there yet:
$ stack test --fast -j1 dhall:doctest dhall-docs:doctest dhall-lsp-server:doctest
dhall > Examples: 257 Tried: 257 Errors: 0 Failures: 0
dhall-docs > Examples: 12 Tried: 12 Errors: 0 Failures: 0
dhall-lsp-server> Examples: 1 Tried: 1 Errors: 0 Failures: 0
$ cabal-docspec dhall dhall-docs dhall-lsp-server
Total: 214; Tried: 44; Skipped: 170; Success: 29; Errors: 15; Failures 0
Examples: 195; Tried: 31; Skipped: 164; Success: 20; Errors: 11; Failures 0
Properties: 3; Tried: 0; Skipped: 3; Success: 0; Errors: 0; Failures 0
Setup: 16; Tried: 13; Skipped: 3; Success: 9; Errors: 4; Failures 0