cabal

cabal copied to clipboard

Describe the bug

Passing additional arguments to the test suite with cabal new-test / v2-test seems impossible.

To Reproduce

1. Create a cabal project with a (trivial) test suite with type: exitcode-stdio-1.0.
2. Run cabal v2-test -- --foo.

Expected behavior

The --foo argument is passed to the compiled test suite.

Observed behavior

Cabal fails with an error message:

cabal: Unknown target '--foo'.
There is no component '--foo'.
The project has no package '--foo'.

System informataion

• Debian 9, x64
• cabal: 2.4.1.0
• ghc: The Glorious Glasgow Haskell Compilation System, version 8.6.5

Additional context

Note that cabal v2-test --help has the following to say on the matter of passing arguments to test suites:

To pass command-line arguments to a test suite, see the v2-run command.

Which, in turn, suggests using --.

See https://github.com/haskell/cabal/issues/5416 for details. In summary, you need to use cabal v2-run test:TARGET -- --foo.

This isn't particularly intuitive, at the minimum the error message should be improved it link out to some external docs.

There's also a v2-test flag that adds arguments if I'm not mistaken

There's --test-option/--test-options. I use it all the time for accepting the Haddock test suite (since v2-test does some more essential stuff that v2-run doesn't do).

cabal new-test --with-ghc ghc-8.6.3 --test-options='--accept'

Just wanted to say that I'd love to see this added as a feature.

btw --test-option(s) flushes the caches... the v2-run workaround is the only way to pass parameters in anything except the smallest of codebases. See https://github.com/haskell/cabal/issues/6114

Would this also solve Perhaps that would solve https://github.com/haskell/cabal/issues/6209 ?

@ptkato this is a good one for you

I've been scouring the cabal-install codebase after an easy, yet non-disruptive, way to implement this feature using --. Turns out it is not possible for v2-test, as pointed out by @23Skidoo in https://github.com/haskell/cabal/issues/3638#issuecomment-318907673, all the arguments are just mangled together and then it's not possible anymore to know, with certainty, which argument is what. Unlike v2-run, that simply picks the first argument as a target and assume the rest as extra arguments, we can't do that for v2-test, because we have other possible flags in addition to the target.

Other than messing up with those lines in Main.hs, which could incur in a much much bigger rewrite, I'm out of ideas, anyone?

I'm afraid it does look disruptive and non-trivial, but OTOH it's obviously needed. I'm more worried about the spec, though. Do we have a uniform understanding of -- across all commands? E.g., what does expandResponse do? Are there other cases than v2-test that would treat -- specially? Does it conflict with expandResponse?

Do we have a uniform understanding of -- across all commands?

Kinda? It seems to be just ignored by every command, but other than that, not really.

E.g., what does expandResponse do? Are there other cases than v2-test that would treat -- specially? Does it conflict with expandResponse?

I don't think modifications to -- would affect expandResponse in any way, since they don't interact with each other.

I'm afraid it does look disruptive and non-trivial, but OTOH it's obviously needed. I'm more worried about the spec, though.

That's a valid concern, maybe we could rely on something like a tuple (as it is spit out by break), or another custom type tailored to our needs.

Alright, I gave using tuples a shot, and it was a terrible idea, a very big chunk of code goes around relying on the arguments as [String], and right now it wouldn't be worth it refactoring all that code; not even mentioning the unforeseen consequences of literally touching every command out there.

I was discussing with @fendor, and we concluded that the best approach would be to implement the -- behaviour in terms of --*-options, not literally, but conceptually. Considering that -- is used to pass arguments to a target executable, I propose:

• The adoption of a generic --target-options flag, which could be called upon in two ways: directly using --target-options as a flag for a command; or using --, which then would be manually wired to --target-options (in the Main function, most likely); and
• Deprecate the current --*-options flags, and while at that, they would all map to the new --target-options flag instead.

These would imply in much smaller changes, since it would rely on the flag machinery already in place.