cabal
cabal copied to clipboard
haskell
cabal-install seems to pass new implicit flags to the compiler by default, which breaks doctest.
## Information
- cabal-install version 3.15.0.0 (commit 2f3a45d, Tue Apr 8 00:30:30 2025 +0000)
- doctest version 0.24.0
## Bug
doctest's manual tells users to run it as a substitute haskell compiler:
cabal repl --with-compiler=doctest
However with cabal-head, the passing of unexpected flags to the compiler breaks.
❯ cabal repl --with-compiler=doctest
Warning: this is a debug build of cabal-install with assertions enabled.
Configuration is affected by the following files:
- cabal.project
Resolving dependencies...
Build profile: -w ghc-9.10.1 -O0
In order, the following will be built (use -v for more details):
- servant-0.20.2 (interactive) (lib) (cannot read state cache)
Warning: this is a debug build of cabal-install with assertions enabled.
Configuring library for servant-0.20.2...
Warning: this is a debug build of cabal-install with assertions enabled.
Preprocessing library for servant-0.20.2...
doctest: unrecognized option `--interactive'
Try `doctest --help' for more information.
Error: [Cabal-7125]
repl failed for servant-0.20.2.
This behaviour cannot be reproduced with the `3.15.0.0.2024.10.3 pre-release. I'm surprised cabal's own doctest suite is able to run at all.
It seems that doctest is being invoked with the --interactive flag, so at a glance, seems everything is working on the cabal side there and it's doctest which has potentially changed?
@mpickering yes, doctest is being invoked by cabal with the --interactive flag since after 3.15.0.0.2024.10.3. Why would it be a doctest change?
The way that cabal starts the repl is by invoking ghc, or the command you say is ghc with the --interactive flag.
For example:
cabal repl -w/path/to/ghc
...
/path/to/ghc --interactive .. -package p1 ... etc etc
@mpickering So this behaviour has always been present? Because when I use the 3.15 pre-release, everything goes well:
❯ cabal --version
cabal-install version 3.15.0.0
compiled using version 3.15.0.0 of the Cabal library
❯ cabal repl --with-compiler=doctest-9.10.1
Build profile: -w ghc-9.10.1 -O0
In order, the following will be built (use -v for more details):
- servant-0.20.2 (interactive) (lib) (first run)
Preprocessing library for servant-0.20.2...
when making flags consistent: warning: [GHC-74335] [-Winconsistent-flags]
Ignoring optimization flags since they are experimental for the byte-code interpreter. Pass -fno-unoptimized-core-for-interpreter to enable this feature.
Examples: 233 Tried: 233 Errors: 0 Failures: 0
Same doctest version, and it does not complain about --interactive. That's why I don't understand when you suggest that something may have changed on the doctest side.
In that command you are using doctest-9.10.1 rather than doctest, are those the same thing?
Yes, it has always worked like this.
I can reproduce. The invocation has indeed slightly changed. Now we append -package-env=- to most of the compiler invocations.
before:
doctest --numeric-version
doctest --supported-languages
doctest --info
doctest --print-global-package-db
doctest --print-libdir
doctest --numeric-version
doctest --supported-languages
doctest --info
doctest --print-libdir -hide-all-packages
doctest -hide-all-packages -c /tmp/433821-7.c -o /tmp/433821-8.o -hide-all-packages
doctest --interactive -fbuilding-cabal-package -O0 -outputdir dist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build -odir dist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build -hidir dist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build -hiedir dist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build/extra-compilation-artifacts/hie -stubdir dist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build -i -isrc -idist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build -idist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build/autogen -idist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build/global-autogen -Idist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build/autogen -Idist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build/global-autogen -Idist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build -optP-include -optPdist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build/autogen/cabal_macros.h -this-unit-id cabal-package-0.1.0.0-inplace -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db /home/andrea/.local/state/cabal/store/ghc-9.10.1-803c/package.db -package-db /home/andrea/Scratchpad/cabal-package/dist-newstyle/packagedb/ghc-9.10.1 -package-db dist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/package.conf.inplace -package-id base-4.20.0.0-8212 -XHaskell2010 MyLib -Wall -hide-all-packages
after (now we use a response file which I manually expanded)
doctest --numeric-version
doctest -package-env=- --supported-languages
doctest -package-env=- --info
doctest -package-env=- --print-global-package-db
doctest -package-env=- --print-libdir
doctest --numeric-version
doctest -package-env=- --supported-languages
doctest -package-env=- --info
doctest -package-env=- --print-libdir -hide-all-packages
doctest -package-env=- -hide-all-packages -c /tmp/437571-7.c -o /tmp/437571-8.o -hide-all-packages
doctest -package-env=- --interactive -fbuilding-cabal-package -O0 -outputdir dist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build -odir dist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build -hidir dist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build -hiedir dist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build/extra-compilation-artifacts/hie -stubdir dist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build -i -isrc -idist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build -idist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build/autogen -idist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build/global-autogen -Idist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build/autogen -Idist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build/global-autogen -Idist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build -optP-include -optPdist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/build/autogen/cabal_macros.h -this-unit-id cabal-package-0.1.0.0-inplace -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db /home/andrea/.local/state/cabal/store/ghc-9.10.1-803c/package.db -package-db /home/andrea/Scratchpad/cabal-package/dist-newstyle/packagedb/ghc-9.10.1 -package-db dist-newstyle/build/x86_64-linux/ghc-9.10.1/cabal-package-0.1.0.0/package.conf.inplace -package-id base-4.20.0.0-8212 -XHaskell2010 MyLib -Wall -hide-all-packages
Invoking doctest -package-env=- --interactive ... seems to work so maybe there's a bug in doctest's handling of response files.
Ah thanks, @andreabedini for the investigation. Yes this is due to a change I made recently. (https://github.com/haskell/cabal/pull/10828)
It seems that
ghc -package-env=- --interactive
works as expected, so the issue is that the doctest shim doesn't behave as much like ghc as it should? I wonder if this also broke hie-bios. I will investigate.
Ah, I think the problem may actually in this bit of logic in the Cabal library
658 runGHCWithResponseFile fileNameTemplate encoding tempFileOptions verbosity ghcProg comp platform maybeWorkDir opts = do
659 invocation <- ghcInvocation verbosity ghcProg comp platform maybeWorkDir opts
660
661 let compilerSupportsResponseFiles =
662 case compilerCompatVersion GHC comp of
663 -- GHC 9.4 is the first version which supports response files.
664 Just version -> version >= mkVersion [9, 4]
665 Nothing -> False
666
667 args = progInvokeArgs invocation
668
669 -- Don't use response files if the first argument is `--interactive`, for
670 -- two related reasons.
671 --
672 -- `hie-bios` relies on a hack to intercept the command-line that `Cabal`
673 -- supplies to `ghc`. Specifically, `hie-bios` creates a script around
674 -- `ghc` that detects if the first option is `--interactive` and if so then
675 -- instead of running `ghc` it prints the command-line that `ghc` was given
676 -- instead of running the command:
677 --
678 -- https://github.com/haskell/hie-bios/blob/ce863dba7b57ded20160b4f11a487e4ff8372c 08/wrappers/cabal#L7
679 --
680 -- … so we can't store that flag in the response file, otherwise that will
681 -- break. However, even if we were to add a special-case to keep that flag
682 -- out of the response file things would still break because `hie-bios`
683 -- stores the arguments to `ghc` that the wrapper script outputs and reuses
684 -- them later. That breaks if you use a response file because it will
685 -- store an argument like `@…/ghc36000-0.rsp` which is a temporary path
686 -- that no longer exists after the wrapper script completes.
687 --
688 -- The work-around here is that we don't use a response file at all if the
689 -- first argument (and only the first argument) to `ghc` is
690 -- `--interactive`. This ensures that `hie-bios` and all downstream
691 -- utilities (e.g. `haskell-language-server`) continue working.
692 --
693 --
694 useResponseFile =
695 case args of
696 "--interactive" : _ -> False
697 _ -> compilerSupportsResponseFiles
Thank you for bringing this attention to our attention @Kleidukos and @andreabedini !
@sol are you aware of this problem with cabal-install master? Any thoughts?
@ulysses4ever It is something we need to resolve for 3.16.
I intend to implement #9115 before then, just lacking time to do so.
Just confirming, cabal HEAD can currently not be used with HLS due to this change.
Implementing #9115 and using the --with-repl flag in hie-bios will fix the issue, and improve HLS's cabal integration.
Just to doublecheck, the issue was that doctest did not properly handle response files, right?
Or is there anything else going on?
--with-replis a great improvement that I'm eager to utilize forcabal-doctestASAP- However,
cabal repl --with-compiler=doctestis used in the wild, so it's important that it does not break.
Just to doublecheck, the issue was that
doctestdid not properly handle response files, right?Or is there anything else going on?
* `--with-repl` is a great improvement that I'm eager to utilize for `cabal-doctest` ASAP * However, `cabal repl --with-compiler=doctest` is used in the wild, so it's important that it does not break.
Yes, the issue was that doctest did not handle response files. If it does not then --with-compiler should work.