Feature request: Add `--with-repl` option

[sol]

Proposal

We already have:

• --ghc-options specifies options that are used for compilation.
• --with-ghc specifies the executable that is used for compilation.
• --repl-options specifies options that are used when opening a REPL.

In addition to those, I would love to see:

• --with-repl specifies the executable that is used when opening a REPL.

Motivation

This can be useful for running arbitrary tools that require the same command line options as ghci.

Personally, I want to use this with doctest?

As of now, the way I recommend to run doctest with cabal is:

$ cabal repl --with-ghc=doctest

For this to this work, doctest has to proxy all invocations to ghc, unless it sees --interactive, at which point it takes over.

This mostly works, but has the following issues:

1. The need for doctest to proxy invocations to ghc complicates doctest. While doctest could technically be agnostic to command line options, it currently has to look at them to decide whether to proxy the invocation, or whether to start an API session and ghci to do its thing .
2. Some packages (notably ghc-paths) will be broken when they are built with doctest proxying to ghc. For that reason, a user needs to take great care to ensure that any such packages have already been built before they invoke doctest in such a manner.

--with-repl will address those issues.

[sol]

Would hie-bios benefit from this as well? @fendor

[fendor]

Hm, I think it would make our custom logic to figure out which ghc eactly cabal repl is going to use obsolete. That code has not been a big source of trouble so far, but I am definitely in favour of making it unnecessary.

[andreabedini]

I find the idea of with-repl quite clever but if the goal is to find what arguments to invoke ghc with... cannot we just do that instead?

(I admit I am up to date with the discussion around this topic)

[gbaz]

Note that for test stanzas, the code-generators field (https://cabal.readthedocs.io/en/stable/file-format-changelog.html#cabal-version-3-8) will produce all options that would be sent to ghc for a build, and allow invoking a custom executable (via test). It should provide (and was intended to) a very clean way to run doctest with cabal, and perhaps can be put to the purpose of some other things besides.

[sol]

Note that for test stanzas, the code-generators field (https://cabal.readthedocs.io/en/stable/file-format-changelog.html#cabal-version-3-8) will produce all options that would be sent to ghc for a build, and allow invoking a custom executable (via test). It should provide (and was intended to) a very clean way to run doctest with cabal, and perhaps can be put to the purpose of some other things besides.

Thanks @gbaz for your reply. I gave my perspective on code-generators before: https://github.com/haskell/cabal/pull/7688#issuecomment-1221229307

We can agree to disagree here, but I want things composable, Unix-style. --with-ghc (or --with-repl for that matter) does not only work for doctest, but also for hspec/sensei, and from what I understand, it is also the approach used by hie-bios. It works without the need to modify the .cabal file. It works for any components, not just libs, and so on.

I imagine that code-generators could work for doctest, and I certainly like the fact that with that approach you can rely on cabal to bring a suitable version of doctest "into scope", but personally, I still put more weight on Unix-style composability.

Finally, even if I would change my stance on doctest, I would still want to use --with-repl for tools that I use during development and that definitely don't belong into the .cabal file (think hspec/sensei, ghcid, or my experimental sol/reserve).

[sol]

I find the idea of with-repl quite clever but if the goal is to find what arguments to invoke ghc with... cannot we just do that instead?

That would mean a subcommand (e.g. repl-options) that prints all the options that cabal would pass to ghci, right?

I think that could be neat, provided that:

1. It basically behaves like --with-repl=echo would
2. We can exclude that users have to deal with quoting issues

[sol]

1. Since cabal-install-3.12.* you can use --repl-multi-file to capture the options that cabal repl would pass to GHC. This is what I currently use for cabal doctest.
2. This is still not a perfect solution for doctest as cabal repl --with-ghc=doctest <target> will change to the correct working directory for <target>, while cabal doctest <target> is not in a good position to do so. This puts the burden on the user to adjust the working directory as needed (e.g. https://github.com/haskell/cabal/pull/10231).

From the perspective of doctest a --with-repl option is still my preferred solution.

[sol]

Another issue with the --repl-multi-file workaround is that it does not bring build-tools into scope.

[mpickering]

This does seem like a pragmatic thing to add, I didn't before think about the possibility of using a different command for the repl invocation to the one used to build dependencies etc.

It would also avoid issues like #10915

[mpickering]

It does mean that you have to make sure that your --with-repl script is compatible with the --with-compiler option. But you have to do that anyway if you are messing around with proxying requests using --with-repl.

[sol]

I would also hope that this will solve the issue with build tools not being in scope for HLS. This is currently a regular annoyance I run into.

[fendor]

I wouldn't solve that issue in particular. What we do in hie-bios is we collect the compilation options but we don't keep the $PATH around. It should be possible to do this right now, the harder task is then to modify HLS to set the PATH appropriately depending on the unit.

[ulysses4ever]

Due to https://github.com/haskell/cabal/issues/10915, I mark this as high priority. See discussion there.

[mpickering]

@sol @fendor Just want to check that you are both set for 3.16 to include the --with-repl flag in your respective projects?

[fendor]

hie-bios-0.16 comes with support for --with-repl and we update HLS in https://github.com/haskell/haskell-language-server/pull/4647

We plan to do an HLS release before the exe:cabal-3.16 release, that I expect in September.

[sol]

@mpickering did you see my comment at https://github.com/haskell/cabal/issues/10915#issuecomment-3037319966.

The bottom line is that doctest users use --with-compiler. I can change the documentation, and I can change cabal-doctest. But this won't change the fact that many users still have --with-compiler in their CI scripts, etc.

[mpickering]

@mpickering did you see my comment at #10915 (comment).

The bottom line is that doctest users use --with-compiler. I can change the documentation, and I can change cabal-doctest. But this won't change the fact that many users still have --with-compiler in their CI scripts, etc.

Thanks, I didn't see that.

If doctest now works with response files then --with-compiler should work. We should test that before the release happens.

[Mikolaj]

Release is, for a very short time, blocked to let this testing happen. Who would volunteers to test that and provide reproducible instructions for the test? Later on, we should add the test to the cabal testsuite, but that doesn't block the release.

Edit: the branch to test is 3.16.

[sol]

@Mikolaj When I implemented support for response files I tested it with @mpickering's branch. I assume that we are fine.

[sol]

Anyway, I double checked again, everything seems fine.

[Mikolaj]

@sol: thank you. Release is unblocked.

TODO: Add the --with-compiler test to CI.

[fendor]

hie-bios tests with cabal HEAD in CI right now.