New field on library: `test-only: Bool`

[jasagredo]

UPDATE

We seem to be converging to this plan:

• Libraries get a new field: test-only: Bool, by default False.
• Libraries with test-only: True can only be imported by other libraries with test-only: True or test-suites.
• Local libraries with test-only: True will only be enabled if --enable-tests is on, even if no other component depends on them. They will respect the usual component naming (i.e. all will build them if they are enabled).

There is however the worry of this resulting in more rebuild of the project or build-plans changing when enabling these libraries. This is not a new problem, but something that already happens with test-suites (and benchmarks).

Original

Describe the feature request

Before trying to take a stab at implementing it, I wanted to gather some feedback and opinions.

Making use of sublibraries, a Haskell package can define a sublibrary that is intended to be used only in tests, either from this package tests or by other package tests, but it is not intended to be consumed in "production code".

For a precise example, we make use of testlibs in our code (https://github.com/IntersectMBO/ouroboros-consensus, look in the cabal files for unstable-*-testlib sublibraries). We would like to not track those in the version number nor reflect changes to them in the changelog to reduce the bureaucratic side of the code.

Moreover, as our project has multiple packages, we need such sublibraries in order to share code between test-suites in different packages.

With the current mechanisms, there is no way to enable such a library only if tests are enabled. I think having the ability to do so would be very useful.

~The proposal I have is to define a new stanza testlib (or test-library). Such libraries will only be available if the building of tests is enabled.~

EDIT: The proposal is in fact that testlibs can only be depended on from test-suites or other testlibs. This also implies not building with --enable-tests won't possibly build the testlibs.

This achieves also an additional nice outcome: Packages that accidentally depend on some code that upstream dependencies thought of as just intended for testing, will fail to build if tests are disabled, correctly conveying the message that such function should not be used in production.

This mechanism can be emulated with cabal flags that expose or hide sublibraries (we used to have a flag expose-sublibs that did this) but this changes the API of a package, and as downstream components cannot specify cabal flags in their build-depends, the same problem cascades down: they have to also disable their testlibs with some trick or compilation of their testlibs will fail.

So I'm open to suggestions on how to best approach this, or whether there is already some solution that I'm not aware of.