Consider dropping all GHC patches by default

[ocharles]

I find it very strange that using Haskell.nix - a Nix build system for Haskell - ends up with a entirely custom version of GHC. Are these patches strictly necessary, or are they for input-output-hk's needs? If so, could they be parameterized out?

[michaelpj]

I've got a lot of sympathy for this argument. I think most of the patches are backports, but then we're effectively maintaining patch-level versions of the released minor versions of GHC, which maybe we don't need to do. Or maybe we could have one and call it 8.10.7.1 or something.

The cost of making the patches swappable is of course that we make it even less likely that you'll get a cache hit. We probably can't afford to build all versions with and without patches.

So for what it's worth, my straw proposal is:

• All released versions of GHC are unpatched (except for stuff that's needed to fix e.g. build issues with Nix)
• We have special patched versions that have a bunch of backports. Maybe just one or two, variants of the latest minor version in the series?

[michaelpj]

Oh, and of course, the answer to "why is it this way?" is that it's for IOHK's needs, but this would be a more useful project if we didn't force that on everyone.

[ocharles]

So for what it's worth, my straw proposal is

I'm perfectly fine with this. I think having IOHK specific code (that's essentially what these patches are) live in this project seems a bit weird, but hey - it's your project, so do what you want!

I'm not very convinced about having backported patches - does any other GHC packaging do this? E.g., do Ubuntu or Arch apply extra patches, or do they just distribute whatever is upstream? What about Haskell specific tools like ghcup?

[michaelpj]

I think having IOHK specific code (that's essentially what these patches are) live in this project seems a bit weird, but hey - it's your project, so do what you want!

Yeah, but we do actually want this to be useful to other people :)

I'm not very convinced about having backported patches - does any other GHC packaging do this?

Right, I agree it's a bit weird on reflection. I'm not 100% sure why we ended up doing this, probably we wanted to actually fix them in a version of GHC used by someone at IOHK, and then we thought "might as well make the others less broken". But I think we could equally well have said "use 8.10.7.1" or whatever we call the frankenpatched version. Opinion from @angerman probably needed.

[gilligan]

So for what it's worth, my straw proposal is:

Definitely also happy with this suggestion 👍

As a user outside of the IOHK universe I'd like to be able to get a vanilla GHC experience (minus any fixes to actually make it work with nix in the first place). The more conveniently I can get that the better.

[gilligan]

@michaelpj any way to move forward on this? (Also still stuck on #1201 because of this)

[angerman]

We try to not have patches in here that regress. We might be able to drop the ubxt patch, we will need to see how that impacts performance though.

For clarification though, we try to upstream every patch to upstream GHC, and a set of patches we have are around cross compilation capabilities. Either we start cutting our own releases, which is a major undertaking or patch upstream.

A more practical approach might be to provide some vanilla flavour. A major concern is that any additional flavour will incure significant CI costs.

I'll put this on the agenda for 2022.

[michaelpj]

A more practical approach might be to provide some vanilla flavour. A major concern is that any additional flavour will incure significant CI costs.

What do you think about my suggestion of having all the "normal" versions of GHC be unpatched, and then providing a special minor release that is basically the latest version with all the patches?

[angerman]

We haven't decided on the exact plan yet. Just that we want to put the unpatched ones in.

Something like ghc-NNN-iohk might be an option.

[thomasjm]

When I first saw this issue I thought that using the unpatched compiler would be equivalent to using a vanilla compiler from Nixpkgs, but that isn't the case right? I looked in the compiler/ghc folder and it looks like a fair amount of customization is still needed. Is it possible that haskell.nix could ever work with a Nixpkgs compiler from the cache.nixos.org cache?

[angerman]

When I first saw this issue I thought that using the unpatched compiler would be equivalent to using a vanilla compiler from Nixpkgs, but that isn't the case right? I looked in the compiler/ghc folder and it looks like a fair amount of customization is still needed. Is it possible that haskell.nix could ever work with a Nixpkgs compiler from the cache.nixos.org cache?

You can try, conceptually it could. But this won't be supported in haskell.nix. While I'm receptive to the idea of having vanilla builds (it hasn't happened due to lack of time really), haskell.nix used to use the upstream nixpkgs ghc expressions, however that couples haskell.nix very tightly to certain nixpkgs releases (and even revisions) as the ghc expressions in nixpkgs can change at any time and without regard for downstream uses. That's why haskell.nix ships it's own nixpgks expressions so that we have full control over them and haskell.nix is better insulated from nixpkgs.

Again, I'm not against adding vanilla ghcs to haskell.nix, it just hasn't happened due to time constraints. Whether or not they'd be the default and the current ones would be -iohk ones, or the vanilla ones would get -vanilla would probably be debatable; but fundamentally there is no reason not to have vanilla GHCs. Cross logic might break, but that would be expected.

[thomasjm]

You can try, conceptually it could.

Is there some way I can pass in my own arbitrary GHC derivation? I see that there's a compiler-nix-name argument you can pass to cabalProject etc., but presumably that refers to the ones that ship with haskell.nix. I think it would be handy to be able to choose your own GHC, with the understanding that it might break.

[michaelpj]

I think it would be handy to be able to choose your own GHC, with the understanding that it might break.

Meta: can you make another issue for that? This issue is about the patches.

[thomasjm]

Sure, will do. I just thought it might be relevant to this discussion because the ability to easily override your own GHC would make it less urgent to decide which patch(es) should be included by default.

[purefn]

On a note related to this topic, how would one go about properly patching ghc? I have a patch to add a few options to read hpc and src directories from files to avoid a "Argument list too long" error. But I'm having hard time figuring out the write way to get it applied in haskell.nix without forking haskell.nix and modifying the list of patches that get applied.

[L-as]

We try to not have patches in here that regress. We might be able to drop the ubxt patch, we will need to see how that impacts performance though.

For clarification though, we try to upstream every patch to upstream GHC, and a set of patches we have are around cross compilation capabilities. Either we start cutting our own releases, which is a major undertaking or patch upstream.

A more practical approach might be to provide some vanilla flavour. A major concern is that any additional flavour will incure significant CI costs.

I'll put this on the agenda for 2022.

So I assume most patches haven't been upstreamed? Do we have any table of mappings from patches to PRs? I'd be interested in pushing this along.