Move development workflows to GitHub

[callahad]

NGINX Unit is moving its planning and development workflows to GitHub to increase transparency, facilitate community participation, and reduce developer friction.

Like Mozilla, Python, and PyPy, we appreciate Mercurial but cannot ignore the market forces which have consolidated behind Git and GitHub.

This is part of a broader effort within F5's NGINX division; Unit is taking the lead in transitioning and intends to pave the way for NGINX and NJS once we're happy with the outcome. Other NGINX projects including Agent, Ingress Controller, and Gateway Fabric have been on GitHub-based workflows from inception; we should align with their practices where reasonable.

We are following the same process for the nginx/unit and nginx/unit-docs repos.

Process

• Phase 1: Development moves to GitHub. Mercurial becomes a read-only mirror. Packaging, CI/CD, etc. still use Mercurial as the source of truth. Developers still push to unit-try to trigger full test runs.

• Phase 2: Release Engineering moves to GitHub. All Mercurial use is discontinued. GitHub is the source of truth.

Tasks

Phase 1:

• [x] Flip the direction of our hg/git sync scripts
  • [x] Check docs, GitHub repo descriptions, etc. to remove mentions of being a read-only mirror
  • [x] Verify that branches and tags propagated correctly on both sides
  • [x] Land a test commit in GitHub and ensure it propagates back into Mercurial correctly
• [x] Discontinue use of Review Board; use GitHub Pull Requests for review
  • [ ] Resolve any remaining open Review Board tasks.

Phase 2:

• [ ] #1054
  • Open question: decommission BuildBot, or trigger BuildBot from GHA?
• [ ] Ensure packaging and release scripts use Git and GitHub
• [ ] Ensure static analysis tools (Coverity, etc.) are set to scan GitHub
• [ ] Decommission Mercurial mirrors, stop sync script?

Other Items

• [ ] Decide on merge workflows (e.g., merge, rebase, or squash and rebase)
• [ ] Implement GitHub branch protection rules

Lessons Learned

• The biggest lesson is to make small changes and verify them each step of the way. We've seen issues with personal email addresses showing up in commits, and with commit hooks in Mercurial preventing changes from syncing across from Git.

• During Phase 1, developers may still need access to the previous Mercurial repos (e.g., we push to a private unit-try repo to trigger BuiltBot runs). There are two options here:

  1. Developers can keep using hg locally, and interact with the GitHub repo via Mercurial's built-in support for Git, or using the separate hg-git extension.
  2. (Preferred by the Unit team) Developers can use git locally, and push to Mercurial via an extension like git-cinnabar or git-remote-hg.

• GitHub's default permissions may be too lax by default; branch protection rules should be configured to avoid new team members accidentally pushing where they shouldn't.

• Team members should be aware that using GitHub's tools for editing code, merging pull requests, etc. will, in most cases, overwrite some commit metadata. E.g., gpg signatures may be replaced by GitHub's own web signing key, and the Commit field will likely change to show GitHub as the committer. Authorship metadata is preserved.

• The team should decide on a code review and merge workflow and configure GitHub to enforce it (e.g., the Unit team prefers a linear history, so merge commits are disabled in our repo settings, but we're still debating whether GitHub's "Squash and Merge" is acceptable or if we'd rather have developers manually prepare their final patch.)

• The team should decide on branching philosophies before moving to GitHub. For example, are short lived feature branches allowed in the main repository, or should they be developed in a personal fork of the repo? (The Unit team prefers the latter to keep the upstream repo clean, and to eat our own dogfood with regard to external contribution workflows.)

• Tags are weird. Git stores tags as mutable pointers inside the repository, while Mercurial stores them alongside the code in a special .hgtags file. Care should be taken to ensure tags stay in sync while in Phase 1. See https://wiki.mercurial-scm.org/GitConcepts#Tag_model for more info.

[ac000]

The team should decide on a code review and merge workflow and configure GitHub to enforce it (e.g., the Unit team prefers a linear history, so merge commits are disabled in our repo settings, but we're still debating whether GitHub's "Squash and Merge" is acceptable or if we'd rather have developers manually prepare their final patch.)

My views on why for me squashing is a non-starter are well documented, but I'll just summarise it as

If a developer has N commits, that should be committed as N commits, then they should be merged as N commits.

I didn't go to the trouble of splitting commits out only to have them all munged back together. This would also be problematic for a pull request with commits from different authors...

or should they be developed in a personal fork of the repo? (The Unit team prefers the latter to keep the upstream repo clean, and to eat our own dogfood with regard to external contribution workflows.)

This. The main repository should not be polluted with random branches. Make use of the D in DVCS.

Tags are weird.

We should switch to using annotated tags to point to a version bump, rather than the currently somewhat weird

1.31.1          Generated Dockerfiles for Unit 1.31.1.
1.31.1-1        Merged with the default branch.

I.e

1.32.0        Unit 1.32.0

That then points to a commit that updates the version. Maybe it includes the above, or maybe it is simply the bump in version in version.

So when you checkout a version, that is everything for that version, e.g at least everything that would be in a .tar.gz for that release.

In case you're wandering about annotated tags, from the git-tag(1) man page

Annotated tags are meant for release while lightweight tags are meant for private or temporary object labels. For this reason, some git commands for naming objects (like git describe) will ignore lightweight tags by default.

That last sentence means this

$ git describe 
fatal: No annotated tags can describe '88854cf14688286f835f7177c6bfaa17f1962f67'.
However, there were unannotated tags: try --tags
$ git describe --tags
1.31.1-24-g88854cf1

Lightweight tag as used by Unit

$ git show --no-decorate 1.31.1
commit 09ab626b13a027a4d3f8b5dcdcdba3e8eee76977
Author: Andrei Zeliankou <[email protected]>
Date:   Tue Oct 17 14:15:38 2023 +0000

    Generated Dockerfiles for Unit 1.31.1.


...

Example of an annotated tag as used by unit-wasm

$ git show --no-decorate v0.3.0
tag v0.3.0
Tagger: Andrew Clayton <[email protected]>
Date:   Thu Oct 19 19:42:38 2023 +0100

unit-wasm 0.3.0

commit 01c43784ec53aa1ff22aca7e7ae6f18b4591b514
Author: Andrew Clayton <[email protected]>
Date:   Thu Oct 19 19:35:12 2023 +0100

    unit-wasm 0.3.0
    
    Signed-off-by: Andrew Clayton <[email protected]>


...