Migrate release CI back to github

[hasufell]

[hasufell]

• Example pipeline: https://github.com/haskell/cabal/actions/runs/6851573157/job/18629453051?pr=9437
• Example draft release: https://github.com/haskell/cabal/releases/tag/untagged-5a72917f7c0a5a2480ec
• Example cirrus CI pipeline: https://cirrus-ci.com/task/6646855723909120?logs=main#L11

Also includes:

• script to download artifacts
• script to create yaml snippet for ghcup

[hasufell]

Workflow:

• push tag cabal-install-v4.0.0.0
• wait for github and cirrus builds to finish
• run scripts/release/download-gh-artifacts.sh cabal-install-v4.0.0.0 <your-gpg-email>
  • upload the contents of the folder gh-release-artifacts/cabal-4.0.0.0 to https://downloads.haskell.org/~cabal/cabal-install-4.0.0.0/
• run scripts/release/create-yaml-snippet.sh cabal-install-v4.0.0.0
  • based on the output create a ghcup-metadata PR

[Kleidukos]

EDIT: I have been a bit quick to approve without giving much background, so here it is: This is appealing because:

• There is a FreeBSD job configured. When I released cabal-3.10.2.0, I had to manually create the FreeBSD bindists on my own private box.

• We offload macOS and Windows runners administration to GitHub. These runners being unstable was not the end of the world but certainly did not help.

As such I am inclined to give this pipeline a chance for these two cases. The linux jobs of the Gitlab pipeline have never been a problem when I took care of 3.10.2.0, so I don't think we need to get rid of them.

[hasufell]

The linux jobs of the Gitlab pipeline have never been a problem when I took care of 3.10.2.0, so I don't think we need to get rid of them.

We get rid of gitlab CI so we don't have to maintain two entirely separate CIs and then figure out how to mangle everything together in a big mess. That's dead code and unnecessary redundancy. Adding new linux distro/distro-versions to the current github CI release pipeline is very easy.

[hasufell]

Time comparison:

• Gitlab CI: 98m 15s
  • https://gitlab.haskell.org/haskell/cabal/-/pipelines/86388
• GitHub CI: 32m 35s
  • https://github.com/haskell/cabal/actions/runs/6851573157

[chreekat]

I think I messed up by marking my review as "needs changes". The actual change I want is the restoration of GitLab files. I don't think my other suggestions should be binding.

[chreekat]

Time comparison:

Gitlab CI: 98m 15s
    https://gitlab.haskell.org/haskell/cabal/-/pipelines/86388
GitHub CI: 32m 35s
    https://github.com/haskell/cabal/actions/runs/6851573157

Ironically, the GitLab pipeline took forever because the Darwin job had to queue for an hour for an open spot. The GitHub job runs on the same server and therefore competes for the same resource. It just gets to jump the queue since the two job runners don't know about each other.

[hasufell]

I do think the GitLab stuff should be kept as a backup as part of good change management, however, so I will request that change.

That would mean we just increased complexity, which defeats the purpose of this PR.

If anything goes extremely wrong during the next release, you can restore the gitlab code from the git history.

[chreekat]

That would mean we just increased complexity, which defeats the purpose of this PR.

If it wasn't clear, I meant to keep it around for the next release and then dropping it. I hope that doesn't defeat the purpose. This change is bigger than just reimplementing some code ,and keeping two systems running in parallel is good practice. And basically free in this case.

[Mikolaj]

keep it around for the next release and then dropping it

Regarding this one point, I'd make it two releases and I'd keep the gitlab CI running all the time to ensure we spot problems in it ASAP, just as we do today.

While I'm talking, @hasufell, thank you for the PR. I remember the big appeal of your proposal long ago, now materialised, was that we'd test what we release. Does this PR bring us closer to the point where we test daily the same binaries that will be released? In particular, are our normal GHA tests run on these artifacts, and when not applicable, on the exact commits from which the artifacts are created? What's the roadmap to unify CI and release CI, to a reasonable extent?

[hasufell]

That would mean we just increased complexity, which defeats the purpose of this PR.

If it wasn't clear, I meant to keep it around for the next release and then dropping it. I hope that doesn't defeat the purpose. This change is bigger than just reimplementing some code ,and keeping two systems running in parallel is good practice.

I don't agree with this at all. There's no indication this is necessary. In HLS we also didn't need it and it has a much more complex CI.

[chreekat]

That would mean we just increased complexity, which defeats the purpose of this PR.

If it wasn't clear, I meant to keep it around for the next release and then dropping it. I hope that doesn't defeat the purpose. This change is bigger than just reimplementing some code ,and keeping two systems running in parallel is good practice.

I don't agree with this at all. There's no indication this is necessary. In HLS we also didn't need it and it has a much more complex CI.

To avoid a deadlock I've created https://github.com/chreekat/cabal/tree/b/add-github-release-ci, which is an edited version of this branch that keeps GitLab CI. I hope it won't be necessary.

[hasufell]

I can split the PR in two commits, so that Gitlab removal is easier to revert in case it's necessary.

But I won't subscribe to "let's keep code in master, just in case". That's bad engineering practice.

[Mikolaj]

But I won't subscribe to "let's keep code in master, just in case". That's bad engineering practice.

I don't think anybody is proposing that. Gitlab CI would be running continuously so that we can spot any problems and fix them ASAP and during a release, if needed, the release manager would be able to compare the results of gitlab and github, test behaviour of the artifacts from both systems, etc.

[hasufell]

I have decided to build bindists myself for the next cabal release, so cabal's CI isn't relevant for GHCup directly anymore.

You can use this PR and the contained patch as you please.

[andreabedini]

I have reopened this to prevent us losing valuable work.

[andreabedini]

And thank you @hasufell for working on this!

[geekosaur]

I'd be willing to take this on after we get 3.12.1.0 out the door.

[andreabedini]

I'd be willing to take this on after we get 3.12.1.0 out the door.

Happy to help too.

[geekosaur]

I have temporarily reinstated the Gitlab CI stuff until we come to an agreement about it. (I'm on the side that it's always possible to arrange things so it can be reverted back in if we need it. I mean, that's what git is for, isn't it?)

(And part of the reason I did that is so I can take it out in a distinct commit that can easily be reverted.)

[geekosaur]

Manually squashed, added Authored-by: to keep original attribution.

[geekosaur]

Gitlab CI removed again, revert a12c6be to bring it back if needed.

[ulysses4ever]

This seems abandoned for now. Let's close it?