Amend non-reinstallable packages list

[adamgundry]

cabal-install maintains a list of nonReinstallablePackages, preventing certain packages being reinstalled by the user. This list needs some updates:

• ghc-internal (released with GHC 9.10) should be considered non-reinstallable, so it needs to be added to the list.
• Following the excellent work by @TeofilC in https://gitlab.haskell.org/ghc/ghc/-/merge_requests/12479, GHC 9.12 will allow reinstallation of the template-haskell package, so it can be removed from the list (cf. #4185).
• GHC 9.12 will hopefully allow reinstallation of base; this is not yet fully implemented but is tracked in https://gitlab.haskell.org/ghc/ghc/-/issues/24903 (cc @mpickering).

The list of non-reinstallable packages does not currently depend on the compiler version. Really it should; is there an easy way to achieve this?

https://github.com/haskell/cabal/blob/dda541cebcde25b3324dac4a59c8b9eb98b01e6a/cabal-install/src/Distribution/Client/Dependency.hs#L447-L471

[adamgundry]

I've opened #10093 to add ghc-internal to the list, since that is otherwise wrong for GHC 9.10. We don't need to resolve base and template-haskell just yet.

Ideally we would keep template-haskell in the list for existing GHC versions but remove it for future versions. Perhaps GHC should expose some interface whereby cabal-install can query the list of non-reinstallable packages?

[geekosaur]

I've been thinking it would be nice of the libdir contained a file of bootlibs and dependencies, which would allow us to both get the non-reinstallable libs and treat the others as non-reinstallable if ghc is in the closure.

[("base", Nothing), -- non-reinstallable in this version
 ("ghc", Just […]) -- list of packages that are non-reinstallable if "ghc" is used
]

(quick example only, I don't know enough about the details)

[geekosaur]

Also, do we need to worry about the likes of ghc-internal? -- ETA: wrong one, I saw several new internal libs in the list. ghc-boot, ghc-boot-th, ghc-experimental.

[phadej]

Perhaps GHC should expose some interface whereby cabal-install can query the list of non-reinstallable packages?

The issue is that packages which cannot be actually built are uploaded to Hackage. AFAICT, the only reason is to provide haddocks in the common place with everything else. But I'd argue it's primarily a Hackage issue. If say base-4.20.0.0 or ghc-internal-9.1001.1 are not reinstallable packages, they shouldn't be in Hackage index (= the input to solver) to begin with.

[mpickering]

@geekosaur Package distributed with ghc are reinstallable unless the build plan contains ghc. The reason that already installed packages are preferred is #9669 which is unrelated to this ticket as far as I can tell.

[geekosaur]

"The build plan contains ghc" is exactly the case I want to capture. I think I missed a case, though: Just [] captures things like text which are reinstallable.

[TeofilC]

I think we only need the first half of your tuple @geekosaur, the second part is just the reverse dependencies of the package from the list, and I think that should be covered already.

I've made a ticket to track the GHC side of this: https://gitlab.haskell.org/ghc/ghc/-/issues/25282

[geekosaur]

The second would matter if the bootlibs started to include a package with a partial or disjoint set of dependencies; I was thinking in terms of forward compatibility.

[TeofilC]

The second would matter if the bootlibs started to include a package with a partial or disjoint set of dependencies; I was thinking in terms of forward compatibility.

I see that makes sense. I feel like maybe we could just go with the simpler thing for now. And if we need this in the future we could modify the interface. I don't imagine we will be needing this extra feature any time soon, since you normally want all the dependencies as well, otherwise you'd currently get broken packages.

[TeofilC]

To update on this. I made an initial MR here: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13297

@mpickering made some great points and it's likely that this interface will need more thought as GHC shouldn't expose information about packages, as that's outside its area of concern.

I'm taking a break from this right now (someone else is free to pick it up), but I'm hoping to get back to it before 9.14

[geekosaur]

Insifar as it's ghc that is foisting this set of packages and their special requirements on the community, I must disagree.

[TeofilC]

@mpickering suggests here

My point was

• GHC can only advertise a unit id, it wasn't clear to me about how to map this information into the cabal solver.

I suppose we will have the assumption that if GHC advertises it is linked against a specific unit id, then that unit will be in a global package database. Cabal already reads the global package database.

Perhaps the only thing we need to add to the cabal solver is the ability to express a constraint that you depend on a precise unit-id. Then when cabal-install starts the solver it can query GHC to find the unit-ids which need to be fixed, and feed those constraints into the solver. Then if users are feeling very brave, they can turn off this behaviour..

Thinking aloud here..

[TeofilC]

Another possibility would be to just make this list of packages be determined by the version of GHC. So GHC>9.12 would mean that template-haskell and base aren't included. And we could leave a principled solution to this interface until some point in the future.

[geekosaur]

I'm not entirely sure if this is related to the pitch I made to the GHC devs last year about shipping a list of non-reinstallable libraries + libraries pinned by the ghc package, but it kind of sounds like it. I personally think that this constitutes a list of ghc's invariants, and as such is part of its purview.

[TeofilC]

I'm not entirely sure if this is related to the pitch I made to the GHC devs last year about shipping a list of non-reinstallable libraries + libraries pinned by the ghc package, but it kind of sounds like it. I personally think that this constitutes a list of ghc's invariants, and as such is part of its purview.

I initially implemented a limited version of your suggestion but it wasnt merged. See the MR: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13297

I think a proposal that needs GHC to expose a list of package names is unlikely to be accepted by the GHC folks.

Ultimately the problem I would like to solve is that the current list is overly pessimistic. template-haskell and perhaps also base can be reinstalled with GHC-9.12 onwards