`cabal v2-install` silently ignores `--run-tests`, but docs suggests it should work

[andreasabel]

https://github.com/haskell/cabal/blob/8365cec87198fce066bc8389604e285eddfe1d9a/doc/cabal-project-description-file.rst?plain=1#L1050-L1068

Run package test suite during installation.

I understand this sentence as cabal v2-install --run-tests should run the tests. It doesn't. It also does not alert the user that it ignores the option --run-tests. cabal v1-install --run-tests does run the tests.

Here is a reproducer for the situation (attached as zip): Make a package with a test suite that always fails and run cabal install --run-tests (the v1- command aborts with error but v2- succeeds).

.cabal:

cabal-version:      3.0
name:               install-run-tests
version:            0
build-type:         Simple

executable install-run-tests-exe
    main-is:          Main.hs
    hs-source-dirs:   .
    default-language: Haskell2010
    build-depends:    base >=4 && < 5

test-suite install-run-tests-test
    default-language: Haskell2010
    type:             exitcode-stdio-1.0
    hs-source-dirs:   test
    main-is:          Main.hs
    build-depends:    base >=4 && < 5

Testsuite test/Main.hs:

import System.Exit

main = do
  putStrLn "Test suite always fails."
  exitFailure

Executable Main.hs:

main = putStrLn "I am the executable."

Simplest fix:

• Report an error if --run-tests if passed to v2-install.
• Correct the docs.

Elaborate fix:

• Implement cabal install --run-tests also for the v2- version.

reproducer.zip

[geekosaur]

This may be part of the general problem that the help has v1 and v2 options intermixed and doesn't distinguish between the two. We have problems with that pretty much everywhere.

[andreasabel]

AFAIK, the documentation only documents v2 now, so this is likely a remnant that did not get cleaned up. cabal v2-build also silently ignores --run-tests. An entry run-tests: True in cabal.project is also silently ignored.

I would actually like cabal build --run-tests to work. In the situation cabal build --enable-tests ...long lists of further options... it would be easier to append --run-tests in the next step rather than navigating all the way back and replace build by test.

[andreasabel]

(Oops wrong button)

[geekosaur]

It's actually more complicated than that: not only is option parsing (and thereby help) combined, but a number of command implementations are still shared as well. So for example v1-update and v2-update are the same (I think), and v2-freeze was the same as v1-freeze until a few months ago. And this bug is v2-build using the same option parsing and then ignoring all the v1-specific options that were parsed.

[andreasabel]

So maybe the way forward here is to have a check at the beginning of v2-build for options that won't be used and emit the respective warnings.

I think programs the user interacts with should be explorative in the sense that they support the user in learning their behavior with good feedback. An essential part of this feedback is that the program can (in some form) communicate how it understood the user request.