Is there a way to monitor test-suite AND library changes without stack?

[gromakovsky]

I have a package that has a library with sources in src/ and a test-sute with sources in test/. I want to use ghcid to get a quick response when something changes in test OR in src. With stack I can do something like ghci --command "stack ghci package:lib package:test:package-test". However, I can't use stack ghci in my case because of a bug in stack and I want to use cabal repl instead.

Unfortunately, I can't pass package:lib package:test:package-test to cabal repl, it will say:

cabal: Cannot open a repl for multiple components at once. The targets
'package' and 'package-test' refer to different components..

The reason for this limitation is that current versions of ghci do not support
loading multiple components as source. Load just one component and when you
make changes to a dependent component then quit and reload.

I tried hard to find any solution and found only this issue: https://github.com/ndmitchell/ghcid/issues/230 Some options suggested there:

1. Add :set -isrc to .ghci. It seems to work only if test-suite has all dependencies that the library has, but it's often not the case. For example, in my module I am getting this:

[  3 of 155] Compiling Michelson.Printer.Util ( src/Michelson/Printer/Util.hs, interpreted )

src/Michelson/Printer/Util.hs:28:1: error:
    Could not load module ‘Text.PrettyPrint.Leijen.Text’
    It is a member of the hidden package ‘wl-pprint-text-1.2.0.1’.
    Perhaps you need to add ‘wl-pprint-text’ to the build-depends in your .cabal file.
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
28 | import Text.PrettyPrint.Leijen.Text
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^...

2. Pass --reload src. If I change something in src it won't be taken into account, as far as I understand. I tried breaking compilation in src, but there were no errors.
3. Pass --restart src. I discarded that option quickly because each restart takes a lot of time and defeats the purpose of ghcid.

I am not sure what's the right solution to this problem. I don't know how repl commands of cabal and stack work, so it's hard for me to figure out whether this feature can be supported in cabal (since it's supported in cabal). In any case I think this use case is very common (ignoring the fact that cabal is apparently less popular than stack), so I wonder if there is any possible workaround better than --restart src. And did I get it right that in order to make :set -isrc work I have to add all deps of my library to the deps of my test-suite? Or did I do something wrong?

[ndmitchell]

Of the options 1 is valid, but 2 and 3 are not really, they might happen to work a bit, but aren't recommended. Typically, if you run cabal test once that will install enough libraries that after that :set -isrc will work. In all my repos I use neither Cabal nor Stack, but have a custom .ghci file that specifies how to load everything (there's even one in this repo), and that's a totally not recommended by everyone else approach that has worked beautifully for me.

[gromakovsky]

Typically, if you run cabal test once that will install enough libraries that after that :set -isrc will work.

For me it doesn't work this way. Maybe I am doing something wrong. Here is a minimal example to demonstrate this problem.

1. First I do cabal test, it passes successfully.
2. Then I do ghcid -c 'cabal repl test:aaaa-test' and it says:

src/Unsafe.hs:6:1: error:
    Could not load module ‘Data.Time’
    It is a member of the hidden package ‘time-1.9.3’.
    Perhaps you need to add ‘time’ to the build-depends in your .cabal file.
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
  |
6 | import Data.Time ()
  | ^^^^^^^^^^^^^^^^^^^

Or just run cabal repl test:aaaa-test, there will be the same error. Does it work for you? If it does, maybe something is wrong with my Haskell installation, I will check it further.

[ndmitchell]

I use a very weird Cabal setup, using cabal v2, with a global install of everything. And I never use cabal repl.

Using your test case, I could get it working by either:

• Add -package=time to the .ghci file.
• Use cabal exec ghci and adding :load Test.hs to the .ghci file.

[gromakovsky]

Thanks, these solutions work for me as well, but only on this simple example. Adding -package=time for each dependency in my big project would be quite painful because there are a lot of dependencies. So I started with trying cabal exec ghci. It does not work because I am using the mixins feature to hide Prelude from base. Apparently mixins thing is ignored and I am getting "Ambiguous module name ‘Prelude’" error.

I noticed cabal exec ghci generates an environment file starting with the following comment:

-- This is a GHC environment file written by cabal. This means you can
-- run ghc or ghci and get the environment of the project as a whole.
-- But you still need to use cabal repl $target to get the environment
-- of specific components (libs, exes, tests etc) because each one can
-- have its own source dirs, cpp flags etc.

From that I concluded that cabal exec is not supposed to apply mixins logic and I should still use cabal repl. So I decided to produce an environment file (passing --write-ghc-environment-files=always to cabal build) and pass it to cabal repl via the GHC_ENVIRONMENT variable. It solved the problem with missing dependencies, but I still got the "Ambiguous module name ‘Prelude’" error.

I tried doing the same on my small example. I have pushed a new commit to https://github.com/gromakovsky/ghcid-issue320, now it has an internal library and uses mixins feature. Internal library defines Numeric module that conflicts with the one from base. I am using mixins to hide Numeric. Results of my experiments:

1. cabal exec ghci does not work, complains about ambiguous Numeric module.
2. cabal repl test:aaaa-test works fine because there is :set package=time in .ghci.
3. If I do GHC_ENVIRONMENT=.ghc.environment.x86_64-linux-8.10.1 cabal repl test:aaaa-test I get "Ambiguous module name ‘Numeric’".

So my conclusion is that the only way to make it work is to add :set -package=PKG for each dependency into .ghci and maintain that file. I think this solution is not perfect because it forces me to duplicate dependencies in .ghci files and add an extra CI check to make sure they are up-to-date. But it's good that at least one solution exists. I am not sure whether mixins is so special and is the only feature that breaks cabal exec ghci in this case, or the problem is not with mixins specifically, but applies to some other things one can specify in .cabal files.

I am not sure how to proceed with this issue. If you have any ideas how to solve this problem without specifying -package for each dependency, I'd be glad to try them. If not, I can make a PR that updates documentation, but I am not sure if this question is worth mentioning. For example, I can add it to FAQ, but maybe this question is not sufficiently important to be there. So please let me know whether it's needed. Maybe I will also open an issue in cabal because I'd like to know:

1. Whether there are any fundamental obstacles to permitting cabal repl lib:library test:test-suite.
2. Whether cabal exec ghci and GHC_ENVIRONMENT=.ghc.environment.x86_64-linux-8.10.1 cabal repl test:aaaa-test are supposed to work in this case.

[ndmitchell]

I think this ultimately comes down to a Cabal problem - and underlying that a GHC problem that it can't load multiple libraries at once. Ghcid gets involved after you have a working ghci prompt, and in your case, you can't get that for both the library and test simultaneously. Agreed that openning a Cabal ticket is probably the way to go.

[cah6]

A good way around this that I found is to include src/ in the source-dirs for your test target. I'm using hpack, so my package.yaml has a section that looks like:

tests:
  test:
    main: Main.hs
    source-dirs:
      - test
      - src

and then I do ghcid test to hot-reload both. You should be able to do this without hpack though, my generated cabal looks like:

test-suite test
  main-is: Main.hs
  other-modules: ...
  hs-source-dirs:
      test
      src
  ...

[matsumonkie]

I use a very weird Cabal setup, using cabal v2, with a global install of everything. And I never use cabal repl.

Hello :wave: Do you mind explaining a bit more how you do this? Like others, my setup with Cabal isn't working nicely and I'm quite curious to see how you work around this.

[ndmitchell]

Do you mind explaining a bit more how you do this?

With newer versions of Cabal, I can't get it to work. With older versions of Cabal, I can't get newer GHC and libraries to work. So I've mostly given up and am suffering with it not working across components :(