doctest
doctest copied to clipboard
sol
"not in scope" errors when running with GHC 9 + stack test; redux
Here are steps to reproduce the error
> git clone https://github.com/morphismtech/squeal.git
> cd squeal
> git checkout dev-ghc9
> createdb exampledb
> stack test squeal-postgresql:doctest
Here are the first two reported test failures:
src/Squeal/PostgreSQL.hs:22: failure in expression `import Squeal.PostgreSQL'
expected:
but got:
^
<interactive>:1:1: error:
attempting to use module ‘main:Squeal.PostgreSQL’ (src/Squeal/PostgreSQL.hs) which is not loaded
src/Squeal/PostgreSQL/Definition/Function.hs:69: failure in expression `type Fn = 'Function ( '[ 'Null 'PGint4, 'Null 'PGint4] :=> 'Returns ( 'Null 'PGint4))'
expected:
but got:
^
<interactive>:47:11: error:
Not in scope: data constructor ‘Function’
<interactive>:47:26: error:
Not in scope: data constructor ‘Null’
Perhaps you meant variable ‘null’ (imported from Prelude)
<interactive>:47:32: error: Not in scope: data constructor ‘PGint4’
<interactive>:47:41: error:
Not in scope: data constructor ‘Null’
Perhaps you meant variable ‘null’ (imported from Prelude)
<interactive>:47:47: error: Not in scope: data constructor ‘PGint4’
<interactive>:47:56: error:
Not in scope: type constructor or class ‘:=>’
<interactive>:47:60: error:
Not in scope: data constructor ‘Returns’
Perhaps you meant variable ‘return’ (imported from Prelude)
<interactive>:47:71: error:
Not in scope: data constructor ‘Null’
Perhaps you meant variable ‘null’ (imported from Prelude)
<interactive>:47:77: error: Not in scope: data constructor ‘PGint4’
The doctest lives in squeal-postgresql/test/Doc.hs and is very simple:
module Main (main) where
import Test.DocTest
main :: IO ()
main = doctest [ "-isrc", "src" ]
@sol I'm having the same issue, do you have any ideas what could be the root cause and if there's any workaround? Thanks a lot!
@echatav do you think it's possible to reproduce your issue to the point where it does not require a database?
@vaclavsvejcar I think your issue is due to :set -XQuasiQuotes triggering https://gitlab.haskell.org/ghc/ghc/-/issues/20670.
@vaclavsvejcar as a workaround, I think you can't set -XQuasiQuotes in Haddock examples, but you can set it in doctest/Main.hs instead.
@vaclavsvejcar as a workaround, I think you can't set
-XQuasiQuotesin Haddock examples, but you can set it indoctest/Main.hsinstead.
@sol Thanks a lot for hints. In my case, that was the issue. It still didn't work with GHC 9.0.1 due to https://gitlab.haskell.org/ghc/ghc/-/issues/19460, but works fine with 9.0.2.
@echatav do you think it's possible to reproduce your issue to the point where it does not require a database?
@sol You don't need the database to see the doctest failures, it'll just add some more failures without a database.
@vaclavsvejcar you are welcome. Ideally, we would enable -XQuasiQuotes by default (as we already do for -XTemplateHaskell). However, this will render e.g. [i|i <- [0..3]] invalid. So not sure if this is the right thing to do.
@echatav I don't have any spare cycles to investigate this myself right now. If you (or somebody else) can provide a somewhat more minimal example that reproduces the issue then I'll try to look at it.
Relude heavily uses doctest and has this issue as well. Moreover, it uses neither TemplateHaskell nor QuasiQuotes but still has this issue. CI error example:
- https://github.com/kowainik/relude/runs/6039897885?check_suite_focus=true
The current workaround is to use GHC 8.10.7 on builds using stack.
@chshersh hey👋 I'll try to look at it and see if I can help, but it may take a week or so until I get to it. If it gets lost somehow then please feel free to ping me again.
@chshersh in general, if you can provide exact steps on how to reproduce, this will save me a lot of time.
Here is what I tried:
$ git rev-parse HEAD
3011495606b8437a3ccd21d03d743f4d31e9c257
$ git diff
diff --git a/relude.cabal b/relude.cabal
index 55e085f..187e237 100644
--- a/relude.cabal
+++ b/relude.cabal
@@ -258,7 +258,7 @@ test-suite relude-doctest
main-is: Doctest.hs
build-depends: relude
- , doctest < 0.19
+ , doctest >= 0.20
, Glob
ghc-options: -threaded
diff --git a/stack.yaml b/stack.yaml
index fd834e1..db2f745 100644
--- a/stack.yaml
+++ b/stack.yaml
@@ -1 +1 @@
-resolver: lts-17.5
+resolver: nightly-2022-04-24
$ stack test relude:test:relude-doctest
...
Examples: 447 Tried: 447 Errors: 0 Failures: 0
relude> Test suite relude-doctest passed
I am way out of touch with this/these issues, but just FYI: the failures originally reported in July with hledger tests and GHC 9.0 are still happening today, but they don't happen with GHC 9.2.
I also get this error but it seems related to the OverloadedStrings extension in my case.
I made a simple repository to reproduce the issue: https://github.com/antoine-fl/doctest-bug
Using GHC 9:
$ stack test
doctest-bug> test (suite: doctest-bug-test)
src/Lib.hs:11: failure in expression `isBar "bar"'
expected: True
but got:
^
<interactive>:25:1: error: Variable not in scope: isBar :: t0 -> t
Examples: 2 Tried: 2 Errors: 0 Failures: 1
doctest-bug> Test suite doctest-bug-test failed
Test suite failure for package doctest-bug-0.1.0.0
doctest-bug-test: exited with: ExitFailure 1
Logs printed to console
Using GHC 8:
$ stack test --stack-yaml stack-8.yaml
doctest-bug> test (suite: doctest-bug-test)
Examples: 2 Tried: 2 Errors: 0 Failures: 0
doctest-bug> Test suite doctest-bug-test passed
The relevant doctest is in that file: https://github.com/antoine-fl/doctest-bug/blob/main/src/Lib.hs
With GHC 9, using Data.Text.pack instead of OverloadedStrings in the doctest fixes the issue.
Edit: testing with GHC 9.2, the tests pass again.
The same is happening to me with autodocodec: https://github.com/NorfairKing/autodocodec
I find this really sad because it forces me to turn off doctests :(
Adding some examples to this issue, hoping they will be helpful for investigating further:
Package random - branch doctests-attempt
https://github.com/haskell/random/tree/doctests-attempt
random on doctests-attempt [!?] via λ lts-18.28
❯ stack test random:test:doctests
random> build (test)
Preprocessing library for random-1.3.0..
Building library for random-1.3.0..
Preprocessing test suite 'doctests' for random-1.3.0..
Building test suite 'doctests' for random-1.3.0..
random> test (suite: doctests)
src/System/Random/Internal.hs:651: failure in expression `import GHC.Generics (Generic)'
expected:
but got:
^
<interactive>:1:32: error:
Module ‘System.Random.Stateful’ does not export ‘globalStdGen’
<interactive>:1:32: error:
Module ‘System.Random.Stateful’ does not export ‘newAtomicGenM’
<interactive>:1:46: error:
Module ‘System.Random.Stateful’ does not export ‘randomRM’
<interactive>:1:47: error:
Module ‘System.Random.Stateful’ does not export ‘applyAtomicGen’
src/System/Random/Internal.hs:691: failure in expression `import GHC.Generics (Generic)'
expected:
but got:
^
<interactive>:1:32: error:
Module ‘System.Random.Stateful’ does not export ‘globalStdGen’
<interactive>:1:32: error:
Module ‘System.Random.Stateful’ does not export ‘newAtomicGenM’
<interactive>:1:46: error:
Module ‘System.Random.Stateful’ does not export ‘randomRM’
<interactive>:1:47: error:
Module ‘System.Random.Stateful’ does not export ‘applyAtomicGen’
src/System/Random/GFinite.hs:88: failure in expression `import GHC.Generics (Generic)'
expected:
but got:
^
<interactive>:1:32: error:
Module ‘System.Random.Stateful’ does not export ‘globalStdGen’
<interactive>:1:32: error:
Module ‘System.Random.Stateful’ does not export ‘newAtomicGenM’
<interactive>:1:46: error:
Module ‘System.Random.Stateful’ does not export ‘randomRM’
<interactive>:1:47: error:
Module ‘System.Random.Stateful’ does not export ‘applyAtomicGen’
Examples: 183 Tried: 171 Errors: 0 Failures: 3
random> Test suite doctests failed
Completed 2 action(s).
Test suite failure for package random-1.3.0
doctests: exited with: ExitFailure 1
Logs printed to console
random on doctests-attempt [!?] via λ lts-18.28 took 53s
❯
NOTES: Notice
Examples: 183 Tried: 171 Errors: 0 Failures: 3. This is definitely weird.
Minimal example package using stack
Code/oss/doctest-stack via λ lts-20.4
❯ lsd --tree
.
├── app
│ └── Main.hs
├── CHANGELOG.md
├── doctest-example-stack.cabal
├── LICENSE
├── package.yaml
├── README.md
├── Setup.hs
├── src
│ └── Lib.hs
├── stack.yaml
├── stack.yaml.lock
└── test
├── doctests.hs
└── Spec.hs
Code/oss/doctest-stack via λ lts-20.4
❯ stack test doctest-example-stack:test:doctest
doctest-example-stack> test (suite: doctest)
Examples: 5 Tried: 5 Errors: 0 Failures: 0
doctest-example-stack> Test suite doctest passed
NOTES: This just works!
Minimal example package using cabal-install
[aniketd@nixos:~/Code/oss/doctest-example]$ tar --exclude-vcs --exclude-vcs-ignores --exclude-caches-all --exclude=.stack-work --exclude=dist-newstyle -zvcf doctest-cabal.tar.gz doctest-example
doctest-example/
doctest-example/CHANGELOG.md
doctest-example/src/
doctest-example/src/Lib.hs
doctest-example/LICENSE
doctest-example/app/
doctest-example/app/Main.hs
doctest-example/flake.lock
doctest-example/test/
doctest-example/test/doctests.hs
doctest-example/test/Main.hs
doctest-example/doctest-example.cabal
doctest-example/flake.nix
[aniketd@nixos:~/Code/oss/doctest-example]$ cabal test doctest-example:test:doctests
Resolving dependencies...
Build profile: -w ghc-8.10.7 -O1
In order, the following will be built (use -v for more details):
- doctest-example-0.1.0.0 (lib) (configuration changed)
- doctest-example-0.1.0.0 (test:doctests) (configuration changed)
Configuring library for doctest-example-0.1.0.0..
Preprocessing library for doctest-example-0.1.0.0..
Building library for doctest-example-0.1.0.0..
[1 of 1] Compiling Lib ( src/Lib.hs, /home/aniketd/Code/oss/doctest-example/dist-newstyle/build/x86_64-linux/ghc-8.10.7/doctest-example-0.1.0.0/build/Lib.o, /home/aniketd/Code/oss/doctest-example/dist-newstyle/build/x86_64-linux/ghc-8.10.7/doctest-example-0.1.0.0/build/Lib.dyn_o ) [System.Random.Stateful changed]
src/Lib.hs:18:1: warning: [-Wunused-top-binds]
Defined but not used: ‘runStateGen_’
|
18 | runStateGen_ g f = runState (f StateGenM) g
| ^^^^^^^^^^^^
src/Lib.hs:25:1: warning: [-Wunused-top-binds]
Defined but not used: ‘foo’
|
25 | foo = 42
| ^^^
src/Lib.hs:31:1: warning: [-Wunused-top-binds]
Defined but not used: ‘bar’
|
31 | bar = "bar"
| ^^^
Configuring test suite 'doctests' for doctest-example-0.1.0.0..
Preprocessing test suite 'doctests' for doctest-example-0.1.0.0..
Building test suite 'doctests' for doctest-example-0.1.0.0..
Linking /home/aniketd/Code/oss/doctest-example/dist-newstyle/build/x86_64-linux/ghc-8.10.7/doctest-example-0.1.0.0/t/doctests/build/doctests/doctests ...
Running 1 test suites...
Test suite doctests: RUNNING...
src/Lib.hs:5:1: error:
Could not find module ‘Control.Monad.State.Strict’
Perhaps you meant Control.Monad.ST.Strict (from base-4.14.3.0)
Use -v (or `:set -v` in ghci) to see a list of the files searched for.
src/Lib.hs:6:1: error:
Could not find module ‘System.Random.Stateful’
Use -v (or `:set -v` in ghci) to see a list of the files searched for.
src/Lib.hs:13: failure in expression `import System.Random.Stateful'
expected:
but got:
^
<no location info>: error:
Could not find module ‘System.Random.Stateful’
It is not a module in the current program, or in any known package.
src/Lib.hs:22: failure in expression `foo + 13'
expected: 55
but got:
^
<interactive>:33:1: error: Variable not in scope: foo
src/Lib.hs:28: failure in expression `bar'
expected: "bar"
but got:
^
<interactive>:39:1: error: Variable not in scope: bar
Examples: 5 Tried: 3 Errors: 0 Failures: 3
Test suite doctests: FAIL
Test suite logged to:
/home/aniketd/Code/oss/doctest-example/dist-newstyle/build/x86_64-linux/ghc-8.10.7/doctest-example-0.1.0.0/t/doctests/test/doctest-example-0.1.0.0-doctests.log
0 of 1 test suites (0 of 1 test cases) passed.
Error: cabal: Tests failed for test:doctests from doctest-example-0.1.0.0.
NOTES: This is the most confusing. You can see that GHC warns us of
unused-top-bindsforfooandbar, butdoctestcomplains of them not being in scope during test execution.
I had some luck with getting rid of "not in scope" errors by replacing :set -XOverloadedStrings by
:seti -XOverloadedStrings
(on GHC 9.4.4).
Strangely I get this error when running locally on trial (https://github.com/tomjaguarpaw/trial) but not when running in Github Actions CI.
